What does setMaster `local[*]` mean in spark?

Question

I found some code to start spark locally with:

val conf = new SparkConf().setAppName("test").setMaster("local[*]")
val ctx = new SparkContext(conf)

What does the [*] mean?

Answer 1

[*]  means for all, as many threads as there are in your machine. Local  keyword is used to run spark locally. There are different parameters that can be used with it .

According to spark.apache.org 

Local[*] is used to run spark locally with  as many worker threads as logical cores on your machine.

The code you have given will specify a master url for a local to run run locally with all the threads on your machine.

Answer 2

I just want to add some additional information:

Don't run Spark Streaming programs locally with master configured as "local" or "local[ 1]". This allows only one CPU for tasks and if a receiver is working on it, there is no resource left to process the obtained data. Use at least "local[ 2]" to have more centers.

If you want to know more about Spark, then do check out this awesome video tutorial: