Published on

Reduce Costs with Scheduled EC2 Instance Shutdowns on AWS

Photo by Eric McCowan

It is often still necessary to run certain applications on an EC2 instance, particularly if it is not easily containerised, and you do not want to pay extra for a managed service. In a non-production environment, you will be able to reduce costs by shutting down these instances when they are not needed by your developers or users.

Off-the-shelf scheduling with an AWS template

The team at AWS has already put together a CloudFormation template you can deploy. The documentation and deployment guide is available here:

It uses a new command line interface to initialise and configure a schedule. This metadata is stored in DynamoDB, and a Lambda function will be run every few minutes to ensure the rules get applied to your instances at the correct times. It is scalable but can seem a bit heavy if you don't need all the features.

A custom but lightweight alternative

The previous template may be too heavy for some. As a simple alternative, you can make something similar with your own Lambda function, albeit with reduced functionality. This approach is suited to non-production environments where developers have permission to start instances manually when they need them, and have those instances stop automatically at the end of the day.

This approach is just a serverless application deployed with AWS Serverless Application Model (SAM). The two key resources are a Lambda function, and a Cloudwatch event rule. With SAM you can specify one template.yaml and just have the one resource, a Type: AWS::Serverless::Function. This resource will neatly wrap up your function's code, event triggers, and IAM policies and make them ready for deployment. Here is the entire template used for this shutdown app:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >

  SAM Template for ec2-autoshutdown

    Type: AWS::Serverless::Function
      CodeUri: ec2autoshutdown/
      Handler: app.lambda_handler
      Runtime: python3.8
          Type: Schedule
            Description: Schedule for shutdown (7:30/UTC = 5:30pm/+10)
            Enabled: True
            Schedule: cron(30 7 * * ? *)
        - !Ref EC2AccessPolicy
          SHUTDOWN_TAG_NAME: 'AutoShutdown'
          WAIT_TIMEOUT_SECS: 10
      MemorySize: 256
      Timeout: 900

    Type: AWS::IAM::ManagedPolicy
      Description: Allows access and state management of EC2 instances
        Version: 2012-10-17
          - Effect: Allow
              - 'ec2:Get*'
              - 'ec2:Describe*'
              - 'ec2:StopInstances'
            Resource: '*'

Now onto the application itself. It gets initiated by a CloudWatch Events Rule, with a daily time configured in the template above. It will request a list of all running EC2 instances, then check each one for the tag key AutoShutdown. If its associated value is True, its ID gets saved along with any others that also match the criteria. Once all other instances have been checked, it will send a single request to stop all those instances.

The application code is written in Python and uses the standard boto3 library for interacting with AWS services. The environment variables above are given to the function below on load, so it knows which tag to look for, and how long to wait.

import json
import boto3
from time import sleep
import os

REGION = os.environ['AWS_REGION']

def lambda_handler(event, context):

    ec2 = boto3.client('ec2', region_name=REGION)

    response_describe = ec2.describe_instances(Filters=[{
        'Name': 'instance-state-name',
        'Values': ['running']

    shutdown_list = list()

    for res in response_describe:
        for instance in res.get('Instances', []):
            ec2_id = instance.get('InstanceId')
            taglist = instance.get('Tags', [])

            ec2_has_true_shutdown_tag = False
            ec2_name = None
            for tag in taglist:
                if tag.get('Key') == SHUTDOWN_TAG_NAME:
                    if tag.get('Value') == 'True':
                        ec2_has_true_shutdown_tag = True
                if tag.get('Key').startswith('Name'):
                    ec2_name = tag.get('Value')

            if not ec2_name:
                ec2_name = 'Unnamed'

            if ec2_has_true_shutdown_tag:
                print('Affected instance: {} ({})'.format(ec2_name, ec2_id))

    if shutdown_list:
        print("\nShutting down the following instances in {} seconds: {}".format(WAIT_TIMEOUT, ', '.join(shutdown_list)))

        response_shutdown = ec2.stop_instances(InstanceIds=shutdown_list)

        return {
            "statusCode": 200,
            "body": json.dumps(response_shutdown, default=str)
        print("No instances to shut down.")

To deploy using SAM, save the code blocks above to template.yaml and ec2shutdown/ respectively then run the following two commands:

sam build --use-container
sam deploy --guided

This will create the resources in AWS for you, and you can tinker with the YAML parameters without having to change the Python app code. When it runs, we get a simple event in the logs like this, followed by the actual API response from the stop_instances request:

Affected instance: ENV-TEST-APP-EC2 (i-0bc99...)
Affected instance: ENV-TEST-DATABASE-EC2 (i-0a239...)
Shutting down the following instances in 10 seconds: i-0bc99..., i-0a239...

This simple output will help developers check that the right instances are getting shut down.

But the next day when you start up the instance again, what if your application doesn't start?

Ensuring your application runs on instance boot

We also need to reduce the time spent starting instances that have been shut down, especially if the application doesn't automatically start for us.

Use systemd

If you are running a Linux application, you may be able to configure the built-in service manager systemd to run your application on boot as a system service. All you really need to do is change some sample values (Description, User/Group, ExecStart) in this example in a text editor, then save the file as /etc/systemd/system/sampleapp.service


ExecStart=/path/to/sampleapp -listen ":8080"
ExecReload=/bin/kill -HUP $MAINPID


Original source code thanks to Tristan.

To start and enable the service to run on boot, run:

sudo systemctl start sampleapp
sudo systemctl enable sampleapp

AWS Glue Development Endpoints are another good candidate to shut down using this method because of their higher hourly cost. Cleaning these up can also be done using the simple Lambda above, where you just list Dev Endpoints (with client.list_dev_endpoints) and delete them.

Consider using Cloud9 for development instances as it has an automatic shutdown based on a configurable timeout (default 30 minutes).

For a routine clean-up of your environment that removes unused resources, Marat has developed a clean-up tool, you can read more about it here.