Mydex CIC shows how to use Python to automate the creation of an AWS volume from the most recent snapshot using boto3.

# Introduction

Most of the time, provisioning volumes for cloud instances is a one-off ‘set and forget’ activity. The instance is provisioned, it has its disk, it does its job. However, sometimes a task involves the temporary creation of a volume, perhaps with some nuance regarding where that volume’s data came from (for example, a snapshot) or where to attach it to. As part of a disaster recovery test procedure, engineers at Mydex CIC found themselves manually restoring snapshots as volumes and carrying out manual verification tests, which took time and needless cognitive load each time. To simplify this process, we designed a Python script that automates the creation, attachment and detachment of EBS (Elastic Block Store) volumes based on the latest snapshot. This script leverages the AWS SDK for Python, known as boto3, to interact with the AWS EC2 service.

# The script

 1#!/usr/bin/env python3
 2
 3import argparse
 4import boto3
 5import time
 6
 7# Parse the args
 8parser = argparse.ArgumentParser()
 9parser.add_argument("-a", action="store_true", help="Attach volume")
10parser.add_argument("-d", action="store", help="Detach volume (volume ID must be provided)")
11args = parser.parse_args()
12
13# Initialize boto3 clients
14region = "your-aws-region"
15session = boto3.Session(profile_name="your-profile-name", region_name=region)
16ec2 = session.client("ec2", region_name=region)
17
18isinstance_id = "your-instance-id"
19
20def get_latest_snapshot():
21    """
22    Get the ID of the most recent snapshot matching the given filters.
23    """
24    snapshots = ec2.describe_snapshots(
25        OwnerIds=["your-aws-account-id"],
26        Filters=[
27            {"Name": "volume-id", "Values": ["your-volume-id"]},
28            {"Name": "description", "Values": ["your-snapshot-description"]},
29        ],
30    )
31    if snapshots["Snapshots"]:
32        # Find the snapshot with the most recent start time
33        latest_snapshot = max(snapshots["Snapshots"], key=lambda s: s["StartTime"])
34        return latest_snapshot["SnapshotId"]
35    return None
36
37
38def create_volume(snapshot_id):
39    """
40    Create a new volume from the given snapshot.
41    """
42    response = ec2.create_volume(
43        AvailabilityZone="your-aws-availability-zone",
44        SnapshotId=snapshot_id,
45        VolumeType="gp3",
46    )
47    return response["VolumeId"]
48
49
50def attach_volume(volume_id):
51    """
52    Attach the given volume to the given instance.
53    """
54    ec2.attach_volume(
55        Device="/dev/sdf", VolumeId=volume_id, InstanceId=isinstance_id
56    )
57
58
59def detach_volume(volume_id):
60    """
61    Detach the given volume from the given instance.
62    """
63    ec2.detach_volume(
64        Device="/dev/sdf", VolumeId=volume_id, InstanceId=isinstance_id
65    )
66
67
68if args.a:
69    snapshot_id = get_latest_snapshot()
70    if snapshot_id:
71        volume_id = create_volume(snapshot_id)
72        print(f"Created volume {volume_id} from snapshot {snapshot_id}. You can pass this ID with -d to detach it later.")
73        time.sleep(60)  # Wait for the volume to become available
74        attach_volume(volume_id)
75        print(f"Attached volume {volume_id} to instance {isinstance_id}")
76    else:
77        raise SystemError("Could not find latest snapshot ID")
78
79if args.d:
80    volume_id = args.d
81    detach_volume(volume_id)
82    print(f"Detached volume {volume_id} from instance {isinstance_id}")

## Explaining the flow

If the argument -a (attach) is supplied:

• The latest snapshot for the supplied volume is located
• A new volume is created from the latest snapshot ID
• The script sleeps for a minute to allow some time for the volume to finish being created (as the response to the previous step may be asynchronous)
• The volume is attached to the supplied EC2 instance. Note that it is not mounted within the OS.

If the argument -d $volume_id (detach) is supplied: The volume with the ID specified after the -d flag is detached from the EC2 instance. Note that it is not unmounted from the OS first, so you should do that yourself.

## Customisation

To use this script effectively, the following placeholders need to be customised:

• your-aws-region: Replace with your AWS region (e.g., eu-west-2).
• your-profile-name: Your AWS credentials profile name.
• your-instance-id: The ID of the EC2 instance to which you want to attach or detach volumes to/from.
• your-aws-account-id: Your AWS account ID.
• your-aws-availability-zone: Your AZ. This parameter can be particularly useful in case the volume needs to be created in a different AZ from that of the original volume. The AZ needs to match the AZ in which the EC2 instance that will be mounting the volume is running.
• your-volume-id: The ID of the volume you want to find snapshots of.
• your-snapshot-description: A description that helps identify the specific snapshot(s) of that volume (in case you want to exempt other snapshots that were taken of that volume for another purpose/at another time).

# Use Cases

Mydex CIC subscribes to the philosophy that backups are not actually backups unless they have been tested (for example, restoring the data temporarily and validating that what was restored was expected).

As a result, at Mydex CIC a script such as the above helps the SysAdmin Team run aspects of Disaster Recovery Tests by quickly creating a new volume from the latest snapshot and attaching it to an instance for testing and verification purposes.

This task can (and used to) be carried out manually. However, it’s a perfect example of a repetitive, predictable process that lends itself well to automation if possible! This reduces effort but also avoids the risk of a mistake resulting in a ‘false positive’ test outcome, which would be as bad as not having tested at all.

Of course, the script could serve other scenarios, such as Backup Automation. Depending on the infrastructure configuration, the script can be integrated into a larger backup strategy to ensure that the most recent snapshots are readily available for use.

If your use-case involves having a ‘warm standby’ volume mounted somewhere, either for disaster recovery purposes or perhaps to ’load shed’ read-only access to a secondary device, you could also set the script to be scheduled to run periodically, providing a seamless way to refresh the backup volume with a relatively recent copy from a snapshot. You would perhaps need to adjust the script to umount and then detach/delete the old volume first before attaching the new one.

# Conclusion

This script serves as a useful component for efficiently managing and leveraging snapshots.

By integrating the script into a backup strategy, organisations can automate the process of creating new volumes from the latest snapshots as part of testing that the snapshots are actually creating reliable restore points.

It may also be useful in periodically bringing a warm spare volume up to date with the latest data.

This automation eliminates the need for manual intervention in retrieving and attaching snapshots, thereby reducing the risk of human error and speeding up the test or restoration process.