Started my Project on Energy-Efficient DevOps: Auto-Suspending Idle AWS Resources using ML

Week 1 – Setup Phase (Problems & Progress)

Objective

To simulate periodic DevOps workloads on the EC2 instance using cron and stress commands, and begin logging CPU usage every minute to generate time-series data for future machine learning model training.

What I Did Today:

Connected to the EC2 Instance via SSH

Connected from PowerShell on Windows using the .pem key:

Created a Bash Script to Simulate Load

A script using stress to consume CPU, memory, and I/O for 2 minutes:

File saved as simulate_load.sh.
Made it executable using:

Scheduled Workload Using Cron

Used crontab to schedule the script to run every 30 minutes:

Logged CPU Usage Every Minute

Created a second script to record CPU usage at 1-minute intervals:

Saved as log_cpu.sh.
Made it executable:

Scheduled it in the same crontab:

All Commands Used Today and Their Meaning

Command	Purpose
`ssh -i "key.pem" ubuntu@<IP>`	Securely SSH into the EC2 instance
`sudo apt install stress htop cron`	Install utilities: `stress` (simulate CPU), `htop` (monitoring), `cron` (task scheduler)
`python3 -m venv aws-env`	Create a Python virtual environment
`source aws-env/bin/activate`	Activate the Python virtual environment
`pip install awscli`	Install AWS CLI in the virtual environment
`nano simulate_load.sh`	Open editor to create the workload simulation script
`chmod +x simulate_load.sh`	Make the script executable
`crontab -e`	Edit the current user’s cron jobs
`/30 * * * /home/ubuntu/simulate_load.sh`	Run the workload script every 30 minutes
`nano log_cpu.sh`	Open editor to create CPU logging script
`chmod +x log_cpu.sh`	Make the logging script executable
`/1 * * * /home/ubuntu/log_cpu.sh`	Run CPU logger every 1 minute
`tail -n 5 filename`	View last 5 lines of a file (to verify cron execution)

Problems Faced:

While editing the crontab file, the SSH connection can time out if the command isn't added quickly, causing the session to reset unexpectedly.
Since the logging scripts run continuously in the background, it’s necessary to reconnect to the EC2 instance multiple times a day to verify that the logs (cpu_log.csv and workload_log.txt) are being updated correctly.

Setup Status

EC2 instance (Ubuntu 24.04 LTS) running
SSH working from Windows
Installed:
- stress, htop, cron, python3-pip
Workload simulation and CPU logging scripts are active
Logs are being collected in cpu_log.csv and workload_log.txt

What’s Next :

Keep instance running 24/7 to gather enough active/idle samples
Monitor logs daily to verify data
Then extract, label, and preprocess logs for ML model training

Screenshot :

Want to Follow Along?

I’ll be sharing weekly progress — issues, logs, architecture, and ML models.

If you've solved similar problems (like automated cloud optimization), I’d love to hear your insight.