Downloading the latest file in an S3 bucket using AWS CLI?

Question

I have an S3 bucket that contains database backups. I am creating a script that I would like to download the latest backup (and eventually restore it somewhere else), but I'm not sure how to go about only grabbing the most recent file from a bucket.

Is it possible to copy only the most recent file from an s3 bucket to a local directory using AWS CLI tools?

Answer 1

You can download the latest object from s3 using the following commands:

$ KEY=`aws s3 ls $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'` $ aws s3 cp s3://$BUCKET/$KEY ./latest-object

Following is how these commands are working.

List all of the objects present in the bucket using the following command:

aws S3 ls $ BUCKET --recursive

Once you have all the object listed, sort them by date using the following command:

aws S3 ls $ BUCKET --recursive | sort

Now the list of objects will be sorted by date in descending order, so the most recent object will be at the end. Run the following command to select the last file:

$ aws s3 $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'

'tail -n 1' will select the last row(the most recent object) and awk '{print $4}' will extract the name of the object.

Finally, drop what you extracted in the previous command into aws s3 cp in order to download the object.

$ aws s3 cp s3://$BUCKET/$KEY ./latest-object

To learn more about Amazon S3, you can enroll in AWS Training.

Answer 2

You can copy the latest object in S3 with the following commands

KEY=`aws s3 ls $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'`\\

$ aws s3 cp s3://$BUCKET/$KEY./latest-object

This is what these commands do.

Download first all objects in the bucket by running

.aws s3 ls $BUCKET --recursive

Sort next the objects chronologically by running

 The sorted output will put the most recently uploaded object at the end.

Command downloading the last uploaded file it should be

$ aws s3 ls $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'

tail -n 1 prints last line awk '{print $4}' prints the name of the object.

Now use the above obtained object name with "aws s3 cp" command for downloading the requested object.

$ aws s3 cp s3://$BUCKET/$KEY./latest-ob