How to install Spark on Ubuntu 16.04?

Question

Can anyone tell me how to install Spark on Ubuntu 16.04?

Answer 1

Apache Spark can be installed on Ubuntu by following the below steps:

• Download the latest stable version of Apache Spark
• Unpack the compressed archive:

$ tar -xvf spark-2.1.1-bin-hadoop2.7.tgz

• Rename the resulting folder with a shorter name and move it to another directory:

$ sudo mv spark-2.1.1-bin-hadoop2.7 /usr/local/

$ sudo ln -s /usr/local/spark-2.1.1-bin-hadoop2.7/ /usr/local/spark

$ cd /usr/local/spark

• Export the SPARK_HOME variable:

$ export SPARK_HOME=/usr/local/spark

• Start a standalone master server:

$ $SPARK_HOME/sbin/start-master.sh

• Start a standalone worker process:

$ $SPARK_HOME/sbin/start-slave.sh spark://ethane:7077

• Start the spark-shell:

$ spark-shell"

If you are looking for an online course to learn Spark, I recommend this Apache Spark Certification program by Intellipaat.