#Install spark locally mac install

Setup the SPARK_HOME now: vi ~/.bashrcĮxport SPARK_HOME=/usr/loca/Cellar/apache-spark/$version/libexecĮxport PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin You spark binaries/package gets installed in /usr/local/Cellar/apache-spark folder.

#Install spark locally mac install

If you have brew configured then all you need to do is just run: brew install apache-spark We will setup a cluster which has 2 slave nodes. We will need the spark cluster setup as we will be submitting our Java Spark jobs to the cluster. Again, there are plenty of good blogs covering this topic, please refer one of them. If you wish to run your pom.xml from command line then you need it on your OS as well. You are good if you have Maven installed in your Eclipse alone.

There are plenty of Java install blogs, please refer one of them for installing and configuring Java either on Mac or Windows.Īs we will be focussing on Java API of Spark, I’d recommend installing latest Eclipse IDE and Maven packages too. Scala install is not needed for spark shell to run as the binaries are included in the prebuilt spark package. spark-shell.cmd and If everything goes fine you have installed Spark successfully. spark-shell and you should be in the scala command prompt as shown in the following pictureįor windows, you will need to extract the tgz spark package using 7zip, which can be downloaded freely. Either double click the package or run tar -xvzf /path/to/yourfile.tgz command which will extract the spark package. Copy the downloaded tgz file to a folder where you want it to reside.Download the version of Spark that you want to work on from here.