Two of the earlier posts are discussing installing Apache Spark-0.8.0 and Apache Spark-1.1.0 on Ubuntu-12.04 and Ubuntu-16.04 respectively. In this post you can discover necessary actions to set up Apache Spark-2.0.2 on Ubuntu 16.04 and Mac OS X Sierra. For enhanced guidance refer above mentioned posts.
Java should be installed in the machine to run Apache Spark. The subsequent commands help quickly install Java in Ubuntu machine.
1 2 3
To install Java in Mac OS X visit oracle website under
Java SE Development Kit 7u79 find and download
jdk-7u79-macosx-x64.dmg file after accepting the license agreement, double click the downloaded dmg file and follow the instructions.
To check the Java installation is successful run following command in the terminal
It exhibits installed java version
java version "1.7.0_72"_
Java(TM) SE Runtime Environment (build 1.7.0_72-b14)_
Java HotSpot(TM) 64-Bit Server VM (build 24.72-b04, mixed mode)
install git. Spark build depends on git. on Ubuntu run
on Mac OS X
Finally, downloaded and untar the apache spark 2 distribution to some location, for example
Maven is used for building Spark, which is bundled with it. To build the apache spark run the following
1 2 3
The building needs some time. After successfully packing you can test a sample program
Then you get the output as
Pi is roughly 3.14634 along with the log.
To build the apache spark for the particular version of hadoop use below command
For more details about buiding apache spark again specific version of hadoop refer
For step by step install of the Apache Spark on Ubuntu refer my previous post