CloudWatch logs stream to Lambda python

Question

I have created a subscription filter in CloudWatch log group and made it stream to my lambda function, but am getting an error in my lambda function.

Code:

import boto3

import binascii

import json

import base64

import zlib

def stream_gzip_decompress(stream):

    dec = zlib.decompressobj(32 + zlib.MAX_WBITS)  # offset 32 to skip the header

    foo=''

    for chunk in stream:

        rv = dec.decompress(chunk)

        if rv:

            foo += rv

    return foo

def lambda_handler(event, context):

    # Decode and decompress the AWS Log stream to extract json object

    stream=json.dumps(event['awslogs']['data'])

    f = base64.b64decode(stream)

    payload=json.loads(stream_gzip_decompress(f.decode(f)))

    print(payload)

Error:

Response:

{

  "errorMessage": "decode() argument 1 must be str, not bytes",

  "errorType": "TypeError",

  "stackTrace": [

    [

      "/var/task/lambda_function.py",

      34,

      "lambda_handler",

      "payload=json.loads(stream_gzip_decompress(f.decode(f)))"

    ]

  ]

}

Any help or clue would be greatly appreciated! If you have any alternative solution please suggest. My requirement is to handle logs from CloudWatch using lambda.

Answer 1

Using Python 3.6 runtime in Lambda, this code ran successfully which triggered Lambda when a CloudWatch event occurred.

import gzip

import json

import base64

 

def lambda_handler(event, context):

    print(f'Logging Event: {event}')

    print(f"Awslog: {event['awslogs']}")

    cw_data = event['awslogs']['data']

    print(f'data: {cw_data}')

    print(f'type: {type(cw_data)}')

    compressed_payload = base64.b64decode(cw_data)

    uncompressed_payload = gzip.decompress(compressed_payload)

    payload = json.loads(uncompressed_payload)

    log_events = payload['logEvents']

    for log_event in log_events:

        print(f'LogEvent: {log_event}')