How can I use wildcards to `cp` a group of files with the AWS CLI

Question

I'm having trouble using * in the AWS CLI to select a subset of files from a certain bucket.

Adding * to the path like this does not seem to work

aws s3 cp s3://data/2016-08*.

Answer 1

Downloading multiple files to your current directory from an aws bucket can be done by using recursive, exclude, and include flags like this:

aws s3 cp s3://data/ . --recursive --exclude "*" --include "*.filetype"

Where "filetype" is the extension, such as .jpg.

This below statement will exclude all files except that file type:

--exclude "*" --include "*.filetype"

This below statement will exclude all the files so keep do not do this

--include "*.txt" --exclude "*" 

For more information, check out this document from Amazon - https://docs.aws.amazon.com/cli/latest/reference/s3/#use-of-exclude-and-include-filters

Answer 2

If you are receiving an error when using '*', and if you want to download multiple files from an aws bucket to your current directory, you can use recursive, exclude, and include flags like this:

aws s3 cp s3://myfiles/ . --recursive --exclude "*" --include "file*"`

Hope this helps you!