How to get contents of a text file from AWS s3 using a lambda function?

Question

I was wondering if I could set up a lambda function for AWS, triggered whenever a new text file is uploaded into an s3 bucket. In the function, I would like to get the contents of the text file and process it somehow. I was wondering if this was possible...?

For example, if I upload foo.txt, with contents foobarbaz, I would like to somehow get foobarbaz in my lambda function so I can do stuff with it. I know I can get metadata from getObject, or a similar method.

Answer 1

Basic code to retrieve bucket and object key from the Lambda event is as follows:

Using the event parameter, you can obtain the S3 object and then read it’s contents. This is the code to retrieve a bucket and the object key using a Lambda event:

exports.handler = function(event, context, callback) {

   var src_bkt = event.Records[0].s3.bucket.name;

   var src_key = event.Records[0].s3.object.key;

};

After getting the bucket, now you use getObject and retrieve the required object:

var AWS = require('aws-sdk');

var s3 = new AWS.S3();

exports.handler = function(event, context, callback) {

    // Retrieve the bucket & key for the uploaded S3 object that

    // caused this Lambda function to be triggered

    var src_bkt = event.Records[0].s3.bucket.name;

    var src_key = event.Records[0].s3.object.key;

    // Retrieve the object

    s3.getObject({

        Bucket: src_bkt,

        Key: src_key

    }, function(err, data) {

        if (err) {

            console.log(err, err.stack);

            callback(err);

        } else {

            console.log("Raw text:\n" + data.Body.toString('ascii'));

            callback(null, null);

        }

    });

};

This is a Java equivalent of the above code, here is the code snippet:

package example;

import java.net.URLDecoder;

import com.amazonaws.services.lambda.runtime.Context;

import com.amazonaws.services.lambda.runtime.RequestHandler;

import com.amazonaws.services.lambda.runtime.events.S3Event;

import com.amazonaws.services.s3.AmazonS3;

import com.amazonaws.services.s3.AmazonS3Client;

import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;

public class S3GetTextBody implements RequestHandler<S3Event, String> {

    public String handleRequest(S3Event s3event, Context context) {

        try {

            S3EventNotificationRecord record = s3event.getRecords().get(0);

            // Retrieve the bucket & key for the uploaded S3 object that

            // caused this Lambda function to be triggered

            String bkt = record.getS3().getBucket().getName();

            String key = record.getS3().getObject().getKey().replace('+', ' ');

            key = URLDecoder.decode(key, "UTF-8");

            // Read the source file as text

            AmazonS3 s3Client = new AmazonS3Client();

            String body = s3Client.getObjectAsString(bkt, key);

            System.out.println("Body: " + body);

            return "ok";

        } catch (Exception e) {

            System.err.println("Exception: " + e);

            return "error";

        }

    }

}