Friday, December 8, 2017

Updating AWS Autoscaling Launch Configs with a new AMI using Lambda

Amazon provides a great article on using Lambda to automate updating the AMI of an auto scaling group's launch configuration.  The only problem with their provided code is that the existing launch configuration's storage settings (ebs volumes) are not kept, so the new launch config has no disks specified, resulting in new launches using the AMI's default settings.

Since an AMI may be generic, the storage settings may be specific to different use cases.

I took their example code and altered it slightly.  In the version below, there is an additional lookup for the existing root volume information, which is then applied to the newly generated launch config.  My change only looks for the root volume since that fits my use-case, but someone smart can adjust to loop through and keep all storage assignments, if needed.

Lambda code:

from __future__ import print_function

import json
import datetime
import time
import boto3

print('Loading function')

def lambda_handler(event, context):
    print("Received event: " + json.dumps(event, indent=2))

    # get autoscaling client
    client = boto3.client('autoscaling')
    clientEc2 = boto3.client('ec2')

    # get object for the ASG we're going to update, filter by name of target ASG
    response = client.describe_auto_scaling_groups(AutoScalingGroupNames=[event['targetASG']])

    if not response['AutoScalingGroups']:
        return 'No such ASG'

    # get name of InstanceID in current ASG that we'll use to model new Launch Configuration after
    sourceInstanceId = response.get('AutoScalingGroups')[0]['Instances'][0]['InstanceId']
    # get the snapshotID of the source AMI
    responseAmi = clientEc2.describe_images(ImageIds=[event['newAmiID']])
    sourceAmiSnapshot = responseAmi.get('Images')[0]['BlockDeviceMappings'][0]['Ebs']['SnapshotId']
    print('New source AMI: ' + event['newAmiID'] + " has snapshot ID: " + sourceAmiSnapshot)
    # get block device mapping (by default boto doesn't copy this)
    sourceLaunchConfig = response.get('AutoScalingGroups')[0]['LaunchConfigurationName']
    print('current launch config name:' + sourceLaunchConfig)
    responseLC = client.describe_launch_configurations(LaunchConfigurationNames=[sourceLaunchConfig])
    sourceBlockDevices = responseLC.get('LaunchConfigurations')[0]['BlockDeviceMappings']
    print('Current LC block devices:')
    sourceBlockDevices[0]['Ebs']['SnapshotId'] = sourceAmiSnapshot
    print('New LC block devices (snapshotID changed):')

    # create LC using instance from target ASG as a template, only diff is the name of the new LC and new AMI
    timeStamp = time.time()
    timeStampString = datetime.datetime.fromtimestamp(timeStamp).strftime('%Y-%m-%d-%H-%M-%S')
    newLaunchConfigName = event['targetASG'] + '_'+ event['newAmiID'] + '_' + timeStampString
    print('new launch config name: ' + newLaunchConfigName)
        InstanceId = sourceInstanceId,
        ImageId= event['newAmiID'],
        BlockDeviceMappings = sourceBlockDevices )

    # update ASG to use new LC
    response = client.update_auto_scaling_group(AutoScalingGroupName = event['targetASG'],LaunchConfigurationName = newLaunchConfigName)

    return 'Updated ASG `%s` with new launch configuration `%s` which includes AMI `%s`.' % (event['targetASG'], newLaunchConfigName, event['newAmiID'])

Or, see a gist

Example call:

$ aws lambda invoke --invocation-type RequestResponse --function-name autoscaling_update_ami --log-type Tail --region us-west-2 --payload '{"newAmiID": "ami-123456", "targetASG": "my-fun-asg"}' 

Monday, March 6, 2017

Calling AWS for current nodes in a group instead of hardcoding public IPs

When integrating CI/CD with cloud instances, the old-school method of specifying a server IP is problematic since a well-architected cloud solution allows for instance to be replaced as needed.  Instead, Jenkins or other processes should verify the current running nodes before issuing a connection attempt.

Below is a sample query that return the public DNS names of servers tagged with a certain value (Group=fancyapp1).

aws ec2 describe-instances  --region us-east-1 --filters "Name=tag:Group,Values=fancyapp1" --output json --query 'Reservations[*].Instances[*].{Name:Tags[?Key==`Name`].Value,PublicIP:PublicIpAddress}'

see gist

The response would look like:

            "Name": [
            "PublicIP": ""
            "Name": [
            "PublicIP": ""

Or, if you just want the first node, change Reservations[*] to Reservations[0].  And if you only want the public IP, remove the Name: part of the query and change output to --text:

aws ec2 describe-instances  --region us-east-1 --filters "Name=tag:Group,Values=myfancyapp1" --output text --query 'Reservations[0].Instances[*].{PublicIP:PublicIpAddress}'

In this case the output would be:

From the source script, you could just set the result of the above to a variable for the server to connect to.