How to check status of Spark applications from the command line?

Question

To check running applications in Apache spark, one can check them from the web interface on the URL:

http://<master>:8080

My question how we can check running applications from terminal, is there any command that returns applications status?

Answer 1

I would suggest you to use spark-submit --status (as described in Mastering Apache Spark 2.0).

spark-submit --status [submission ID]

And for reference just see the code of spark-submit:

if (!master.startsWith("spark://") && !master.startsWith("mesos://")) {

  SparkSubmit.printErrorAndExit(

    "Requesting submission statuses is only supported in standalone or Mesos mode!")

}