How to use AWS S3 CLI to dump files to stdout in BASH?

Question

I'm starting a bash script which will take a path in S3 (as specified to the ls command) and dump the contents of all of the file objects to stdout. Essentially I'd like to replicate cat /path/to/files/* except for S3, e.g. s3cat '/bucket/path/to/files/*'. My first inclination looking at the options is to use the cp command to a temporary file and then cat that.

Has anyone tried this or similar or is there already a command I'm not finding which does it?

Answer 1

The contents of all the file objects have to be dumped to stdout

Pass “-” for destination of aws s3 cp command to obtain this. For instance,  $ aws s3 cp s3://mybucket/stream.txt 

What you're trying to do is something like this? ::

#!/bin/bash

BUCKET=your-bucket-name

for key in `aws s3api list-objects --bucket $BUCKET --prefix bucket/path/to/files/ | jq -r '.Contents[].Key'`

do

  echo $key

  aws s3 cp s3://$BUCKET/$key - | md5sum

Done