0 votes
1 view
in AWS by (18.2k points)

I'm starting a bash script which will take a path in S3 (as specified to the ls command) and dump the contents of all of the file objects to stdout. Essentially I'd like to replicate cat /path/to/files/* except for S3, e.g. s3cat '/bucket/path/to/files/*'. My first inclination looking at the options is to use the cp command to a temporary file and then cat that.

Has anyone tried this or similar or is there already a command I'm not finding which does it?

1 Answer

0 votes
by (42.5k points)

The contents of all the file objects have to be dumped to stdout

Pass “-” for destination of aws s3 cp command to obtain this. For instance,  $ aws s3 cp s3://mybucket/stream.txt 

What you're trying to do is something like this? ::

#!/bin/bash

BUCKET=your-bucket-name

for key in `aws s3api list-objects --bucket $BUCKET --prefix bucket/path/to/files/ | jq -r '.Contents[].Key'`

do

  echo $key

  aws s3 cp s3://$BUCKET/$key - | md5sum

Done

Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...