Downloading the latest file in an S3 bucket using AWS CLI?

Question

I have an S3 bucket that contains database backups. I am creating a script that I would like to download the latest backup (and eventually restore it somewhere else), but I'm not sure how to go about only grabbing the most recent file from a bucket.

Is it possible to copy only the most recent file from an s3 bucket to a local directory using AWS CLI tools?

Answer 1

You can download the latest object from s3 using the following commands:

$ KEY=`aws s3 ls $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'` $ aws s3 cp s3://$BUCKET/$KEY ./latest-object

Following is how these commands are working.

List all of the objects present in the bucket using the following command:

aws S3 ls $ BUCKET --recursive

Once you have all the object listed, sort them by date using the following command:

aws S3 ls $ BUCKET --recursive | sort

Now the list of objects will be sorted by date in descending order, so the most recent object will be at the end. Run the following command to select the last file:

$ aws s3 $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'

'tail -n 1' will select the last row(the most recent object) and awk '{print $4}' will extract the name of the object.

Finally, drop what you extracted in the previous command into aws s3 cp in order to download the object.

$ aws s3 cp s3://$BUCKET/$KEY ./latest-object

To learn more about Amazon S3, you can enroll in AWS Training.