Connecting to DocumentDB from AWS Glue

Question

I'm creating a python shell job in Glue, but I'm not able to access the DocumentDB from Glue. So, how can I connect to DocumentDB to Glue?

Answer 1

You can create a dummy JDBC connection in Glue, which will allow ENI to be created in VPC, and attach this to the python shell job. This will make you interact with the resources.

Or you can use CSV in s3 to connect to DocumentDB with Glue, below providing the Script:

# Constants

data_catalog_database = 'sample-db'

data_catalog_table = 'data'

## @params: [JOB_NAME]

args = getResolvedOptions(sys.argv, ['JOB_NAME'])

spark_context = SparkContext()

glue_context = GlueContext(spark_context)

job = Job(glue_context)

job.init(args['JOB_NAME'], args)

# Read from data source

## @type: DataSource

## @args: [database = "glue-gzip", table_name = "glue_gzip"]

## @return: dynamic_frame

## @inputs: []

dynamic_frame = glue_context.create_dynamic_frame.from_catalog(

    database=data_catalog_database,

    table_name=data_catalog_table

)

documentdb_write_uri = 'mongodb://yourdocumentdbcluster.amazonaws.com:27017'

write_documentdb_options = {

    "uri": documentdb_write_uri,

    "database": "yourdbname",

    "collection": "yourcollectionname",

    "username": "###",

    "password": "###"

}

# Write DynamicFrame to MongoDB and DocumentDB

glue_context.write_dynamic_frame.from_options(dynamic_frame, connection_type="documentdb",                                             connection_options=write_documentdb_options)

Interested in learning AWS? Check out: AWS Online training.