Intellipaat Back

Explore Courses Blog Tutorials Interview Questions
+1 vote
2 views
in Big Data Hadoop & Spark by (1k points)

as titled, how do I know which version of spark has been installed in the CentOS?

The current system has installed cdh5.1.0.

3 Answers

0 votes
by (13.2k points)

 Open Spark shell Terminal and enter command

sc.version  Or  spark-submit --version

 The easiest way is to just launch “spark-shell” in command line. It will display the   

 current active version of Spark.

0 votes
by (33.1k points)
edited by

You can get the spark version by using the following command:

spark-submit --version

spark-shell --version

spark-sql --version

Hope this answer helps you!

If you want more information regarding the same, refer the following video:

0 votes
ago by (1.1k points)

1.Using Command Line

spark-submit --version

2.Using PySpark and In Spark shell

   i. Import pyspark 

     print(pyspark.__version__)

  ii. Sc.version

Related questions

31k questions

32.8k answers

501 comments

693 users

Browse Categories

...