How to do parallel uploads to the same s3 bucket directory with s3 cmd?

Question

When I tried to run the following code, the index.js turned out to be corrupted and if the upload is unparalleled, the process of uploading looks fine. Please help me with a solution.

gzip dist/production/index.js

mv dist/production/index.js.gz dist/production/index.js

s3cmd --access_key="$S3_ACCESS_KEY" --secret_key="$S3_SECRET_KEY" \

      --acl-public --no-mime-magic --progress --recursive         \

      --exclude "dist/production/index.js" \

      put dist/production/ 

      "s3://${BUCKET}/something/${BUILD_IDENTIFIER}/production/" &

s3cmd --access_key="$S3_ACCESS_KEY" --secret_key="$S3_SECRET_KEY" \

      --acl-public --no-mime-magic --progress --recursive         \

      --add-header="Content-Encoding:gzip" \

      put dist/production/index.js 

      "s3://${BUCKET}/something/${BUILD_IDENTIFIER}/production/" &

wait

Answer 1

Use s3cmd-modified, which allows you to carry on parallel put/get/sync operations with multiple AWS s3 bucket.

$ git clone https://github.com/pcorliss/s3cmd-modification.git

$ cd s3cmd-modification

$ python setup.py install

$ s3cmd --parallel --workers=4 sync /source/path s3://target/path