Add DYLD_LIBRARY_PATH under Environment Variables
2017-01-31
How to set DYLD_LIBRARY_PATH in Xcode
Open Product > Scheme > Edit Scheme > Run > Arguments
Add DYLD_LIBRARY_PATH under Environment Variables
Add DYLD_LIBRARY_PATH under Environment Variables
dyld: Library not loaded: libboost_system.dylib
Got an error when compile boost library and run a sample program.
Check libraries then tried install_name_tool, but failed.
$ ./PROGRAM
dyld: Library not loaded: libboost_system.dylib
Referenced from: /PATH/TO/PROGRAM
Reason: image not found
Abort trap: 6
$ otool -L PROGRAMSolution, add the following to .profile or .bash_profile
PROGRAM:
libboost_system.dylib (compatibility version 0.0.0, current version 0.0.0)
libboost_thread.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 307.4.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.0.0)
$ install_name_tool -change libboost_system.dylib /PATH/TO/libboost_system.dylib PROGRAM
$ install_name_tool -change libboost_thread.dylib /PATH/TO/libboost_thread.dylib PROGRAM
export DYLD_LIBRARY_PATH=/PATH/TO/BOOST/stage/lib
2017-01-11
Install Spark on Ubuntu
# Install Java
# Install Scala
Download prebuilt Spark from http://spark.apache.org/downloads.html
$ sudo apt install python-software-properties $ sudo add-apt-repository ppa:webupd&team/java $ sudo apt update $ sudo apt install oracle-java8-installer
# Install Scala
# Install Spark$ sudo apt install scala
Download prebuilt Spark from http://spark.apache.org/downloads.html
$ wget http://d3kbcqa49mib13.cloudfront.net/spark-2.1.0-bin-hadoop2.7.tgz $ sudo mv spark-2.1.0-bin-hadoop2.7.tgz /opt $ sudo tar xfz spark-2.1.9-bin-hadoop2.7.tgz $ sudo ln -s spark-2.1.9-bin-hadooop2.7 spark
# Add environment variables and add path
$ vi ~/.profile
JAVA_HOME=/usr/lib/jvm/java-8-oracle SCALA_HOME=/usr/share/scala SPARK_HOME=/opt/spark PYTHONPATH=$SPARK_HOME/python/lib/pyspark.zip:$SPARK_HOME/python/lib/py4j-0.10-4-src.zip export JAVA_HOME SPARK_HOME PYTHONPATH PATH="$HOME/bin:$HOME/.local/bin:$PATH:$SPARK_HOME/bin"
# Edit configuration(Optioinal) to reduce logs displayed
$ cd $SPARK_HOME/conf $ sudo cp log4j.properties.template log4j.properties $ sudo vi log4j.properties
FIND : log4j.rootCategory=INFO, console REPLACE : log4j.rootCategory=WARN, console
# Test if Spark works
$ run-example SparkPi 10
Pi is roughly 3.140963140963141
# Launch Spark Shell(run scala language) or PySpark(run python language)
$ spark-shell Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /___/ .__/\_,_/_/ /_/\_\ version 2.1.0 /_/ Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111) Type in expressions to have them evaluated. Type :help for more information. scala >
$ pyspark Welcome to ____ __ / __/__ ___ _____/ /__ _\ \/ _ \/ _ `/ __/ '_/ /__ / .__/\_,_/_/ /_/\_\ version 2.1.0 /_/ Using Python version 2.7.12 (default, Nov 19 2016 06:48:10) SparkSession available as 'spark'. > > >
$ Spark UI
http://localhost:4040
Install Spark on Windows 10
# Download
JAVA from http://www.oracle.com/technetwork/java/javase/downloads/index.html
SCALA from http://www.scala-lang.org/download/
Prebuilt Spark from http://spark.apache.org/downloads.html
WINUTILS from https://github.com/steveloughran/winutils
# Installation
Install Java and Scala
Unzip prebuilt Spark into C:\Usr
Unzip Winutils(hadoop 2.7.1/bin) into C:\Usr\spark-2.10-bin-hadoop2.7\bin
# Set environment variables
JAVA_HOME=C:\Program Files\Java\jdk1.8.0_111
_JAVA_OPTIONS=-Xmx512M -Xms512M
SCALA_HOME=C:\Program Files\scala
SPARK_HOME=C:\Usr\spark-2.1.0-bin-hadoop2.7
HADOOP_HOME=C:\Usr\spark-2.1.0-bin-hadoop2.7
# Add path
%SCALA_HOME%\bin
%SPARK_HOME%\bin
# Grant permissions
winutils.exe chmod 777 C:\tmp\hive
# Run command
> scala -version
Picked up _JAVA_OPTIONS: -Xmx512M -Xms512M
Scala code runner version 2.12.1 -- Copyright 2002-2016, LAMP/EPFL and Lightbend, Inc.
> spark-shell
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.1.0
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_111)
Type in expressions to have them evaluated.
Type :help for more information.
scala>
Subscribe to:
Posts (Atom)