What is driver memory in Spark?

Question

Can anyone explain the driver memory in Spark?

Answer 1

Driver memory in Spark is allocated to be 1 GB by default, but this can be altered using the -driver -memory flag. The driver memory is the memory that stores RDDs created during the Spark job, and the executor memory is the memory allocated within the worker nodes to perform job execution.

Kickstart your career by enrolling in the Apache Spark course in Singapore.

If you are looking for an online course to learn Spark, check out this Spark Training program by Intellipaat.