Is it possible to get the current spark context settings in PySpark?

Question

I'm trying to get the path to spark.worker.dir for the current sparkcontext.

Answer 1

Yes, I would suggest you to try the method below in order to get the current spark context settings.

SparkConf.getAll()

as accessed by

SparkContext.sc._conf

Another method specifically for Spark 2.1+

spark.sparkContext.getConf().getAll() 

where spark is your sparksession (gives you a dict with all configured settings)