Managing Storage: EBS Snapshots & S3 Versioning

Summary

The lab combines four tasks. Tasks 1–2 set up the S3 bucket / IAM role and run the manual and scheduled EBS snapshot flow. Tasks 3–4 (the challenge) cover the S3 sync cycle with versioning, including recovering an object after a delete marker is created.

Task 1 — Bucket + IAM role for the Processor

Setup task: create the S3 bucket that the challenge will use and grant the Processor instance the permissions it needs to talk to EBS and S3.

Task 2 — Initial EBS snapshot from the CLI

Connect to the Command Host via EC2 Instance Connect, identify the Processor's EBS volume, stop the instance to guarantee a clean snapshot, take the snapshot, and start the instance again.

Discover the volume + instance IDs

Two describe-instances queries with JMESPath filters returned the VolumeId and the InstanceId of the Processor. These values are the inputs for every subsequent EBS call.

aws ec2 describe-instances \
  --filter 'Name=tag:Name,Values=Processor' \
  --query 'Reservations[0].Instances[0].BlockDeviceMappings[0].Ebs.{VolumeId:VolumeId}'

aws ec2 describe-instances \
  --filters 'Name=tag:Name,Values=Processor' \
  --query 'Reservations[0].Instances[0].InstanceId'

Actual IDs returned in this run: volume vol-052774238e8c5a84b, instance i-0b40f2b484a8f4733.

Stop · snapshot · start

Stopping the instance before create-snapshot guarantees the filesystem is flushed. The wait sub-commands are blocking and return only when the resource reaches the target state, which makes the flow scriptable.

aws ec2 stop-instances           --instance-ids i-0b40f2b484a8f4733
aws ec2 wait instance-stopped    --instance-id  i-0b40f2b484a8f4733
aws ec2 create-snapshot          --volume-id    vol-052774238e8c5a84b
aws ec2 wait snapshot-completed  --snapshot-id  snap-0c318bdefed94e5c4
aws ec2 start-instances          --instance-ids i-0b40f2b484a8f4733

Task 3 — Scheduled snapshots + retention script

The lab installs a one-minute cron entry to generate snapshots automatically, then uses a Python script (snapshotter_v2.py) to keep only the two most recent ones per volume.

Schedule recurring snapshots

The cron entry redirects both stdout and stderr to /tmp/cronlog so the job runs silently while still leaving a trail for debugging. After installing the job, a single describe-snapshots call against the volume confirms that new snapshots show up every minute (one completed, one pending in the capture).

echo "* * * * *  aws ec2 create-snapshot --volume-id vol-052774238e8c5a84b 2>&1 >> /tmp/cronlog" > cronjob
crontab cronjob

aws ec2 describe-snapshots --filters "Name=volume-id,Values=vol-052774238e8c5a84b"

Retention with snapshotter_v2.py

Once enough snapshots accumulate, the cron job is removed with crontab -r and snapshotter_v2.py is run. The script uses boto3 to enumerate all volumes in the account, sort the existing snapshots by start_time, and delete anything beyond the most recent MAX_SNAPSHOTS. Only the script's relevant structure is shown below — the full code is on the Processor, not transcribed here.

#!/usr/bin/env python
import boto3

MAX_SNAPSHOTS = 2   # keep only the last 2 per volume

ec2 = boto3.resource('ec2')

for v in ec2.volumes.all():
    v.create_snapshot()
    snapshots = list(v.snapshots.all())
    if len(snapshots) > MAX_SNAPSHOTS:
        snap_sorted = sorted(
            [(s.id, s.start_time, s) for s in snapshots],
            key=lambda k: k[1]
        )
        for s in snap_sorted[:-MAX_SNAPSHOTS]:
            print("Deleting snapshot", s[0])
            s[2].delete()

The script also creates a snapshot on each run before pruning. That is the source of the new snap-080e659fbfaea38f1 visible in the post-execution list.

Task 4 — Challenge: S3 sync with versioning

Switch hosts: this task runs on the Processor instance (which now has the S3BucketAccess role). The goal is to sync a local files/ directory with s3://superbucket3k/, delete a file locally, propagate the delete to S3, and then recover the deleted object using S3 versioning.

Sample files + enable versioning

The sample files.zip was already pre-staged on the Processor. After unzip files.zip the working directory contained:

$ unzip files.zip
Archive:  files.zip
 inflating: files/file1.txt
 inflating: files/file2.txt
 inflating: files/file3.txt

$ ls
cronjob  files  files.zip  get-pip.py  snapshotter_v2.py

$ ls files
file1.txt  file2.txt  file3.txt

Versioning was enabled on the bucket before any sync — this is the critical ordering, because objects uploaded while versioning is off cannot be recovered later.

aws s3api put-bucket-versioning \
  --bucket superbucket3k \
  --versioning-configuration Status=Enabled

The command returns nothing on success, which is correct AWS CLI behavior for idempotent configuration calls.

Initial sync

A plain aws s3 sync uploaded the three files. aws s3 ls confirmed the bucket state.

$ aws s3 sync files s3://superbucket3k/files/
upload: files/file1.txt to s3://superbucket3k/files/file1.txt
upload: files/file2.txt to s3://superbucket3k/files/file2.txt
upload: files/file3.txt to s3://superbucket3k/files/file3.txt

$ aws s3 ls s3://superbucket3k/files/
2026-05-15 06:12:49      30318 file1.txt
2026-05-15 06:12:49      43784 file2.txt
2026-05-15 06:12:49      96675 file3.txt

Delete locally · propagate with --delete

file1.txt was removed from the local files/ directory and a second sync was run with the --delete flag. Without --delete, aws s3 sync is upload-only and would have left the file in S3.

$ rm files/file1.txt
$ ls files
file2.txt  file3.txt

$ aws s3 sync files s3://superbucket3k/files/ --delete
delete: s3://superbucket3k/files/file1.txt

$ aws s3 ls s3://superbucket3k/files/
2026-05-15 06:12:49      43784 file2.txt
2026-05-15 06:12:49      96675 file3.txt

Inspect versions · list the delete marker

list-object-versions returns two parallel arrays. Versions[] contains the actual content versions; DeleteMarkers[] contains the tombstone entries. After the propagated delete, file1.txt has one of each:

aws s3api list-object-versions \
  --bucket superbucket3k \
  --prefix files/file1.txt

Captured VersionIds from this run, used in the next step:

Recover the previous version · re-sync

There is no single "restore" call in S3; the workflow is to download the previous version by VersionId straight to the original local path, then sync again so the bucket sees a new latest version.

$ aws s3api get-object \
  --bucket superbucket3k \
  --key files/file1.txt \
  --version-id xyHef1ZSWB5VCgi4HR6917ssBs2uWoYh \
  files/file1.txt

$ ls files
file1.txt  file2.txt  file3.txt

$ aws s3 sync files s3://superbucket3k/files/
upload: files/file1.txt to s3://superbucket3k/files/file1.txt

$ aws s3 ls s3://superbucket3k/files/
2026-05-15 06:18:12      30318 file1.txt
2026-05-15 06:12:49      43784 file2.txt
2026-05-15 06:12:49      96675 file3.txt

Notice the new LastModified on file1.txt: 06:18:12 instead of the original 06:12:49. The bucket now has three "live" objects again, plus the historical version chain underneath.

Key concepts

Three ideas that the lab cements through hands-on commands rather than theory.

S3 versioning lifecycle (this lab in 4 states)

Command reference

Quick scan of every CLI verb used in this lab, grouped by phase.