JAVA from http://www.oracle.com/technetwork/java/javase/downloads/index.html
SCALA from http://www.scala-lang.org/download/
Prebuilt Spark from http://spark.apache.org/downloads.html
WINUTILS from https://github.com/steveloughran/winutils
# Installation
Install Java and Scala
Unzip prebuilt Spark into C:\Usr
Unzip Winutils(hadoop 2.7.1/bin) into C:\Usr\spark-2.10-bin-hadoop2.7\bin
# Set environment variables
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_111
_JAVA_OPTIONS=-Xmx512M -Xms512M
SCALA_HOME=C:\Program Files\scala
SPARK_HOME=C:\Usr\spark-2.1.0-bin-hadoop2.7
HADOOP_HOME=C:\Usr\spark-2.1.0-bin-hadoop2.7
# Add path
%SCALA_HOME%\bin
%SPARK_HOME%\bin
# Grant permissions
winutils.exe chmod 777 C:\tmp\hive
# Run command
> scala -version
Picked up _JAVA_OPTIONS: -Xmx512M -Xms512M
Scala code runner version 2.12.1 -- Copyright 2002-2016, LAMP/EPFL and Lightbend, Inc.
> spark-shell
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.1.0
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111)
Type in expressions to have them evaluated.
Type :help for more information.
scala>
Thank you so much. This was concise and very helpful.
ReplyDelete