Spark: How to kill running process without exiting shell?

Question

How can I kill a running process in the Spark shell on my local OSX machine without exiting?

For example, if I just do a simple .count() on an RDD, it can take a while and sometimes I want to kill it.

However, if I do Ctrl C then it kills the whole shell.

Is there a way to kill the process but not the shell?

Answer 1

You can simply use the Master Web Interface to kill or Visualize the Job. 

Also, in order to kill the process that is failing repeatedly,you can do this: 

./bin/spark-class org.apache.spark.deploy.Client kill <master url> <driver ID>

Reference: http://spark.apache.org/docs/latest/spark-standalone.html