Cacher is the code snippet organizer for pro developers

Machine

• Launch ami-d05e75b8 from here
• Choose a GPU instance type: g2.2xlarge or g2.8xlarge
• Increase the size of the storage (this depends on what else you plan to install, I’d suggest at least 20 GB)

Install Cuda 7.0

wget http://developer.download.nvidia.com/compute/cuda/7_0/Prod/local_installers/rpmdeb/cuda-repo-ubuntu1404-7-0-local_7.0-28_amd64.deb

sudo dpkg -i cuda-repo-ubuntu1404-7-0-local_7.0-28_amd64.deb

sudo apt-get update
sudo apt-get upgrade -y

sudo apt-get install -y opencl-headers build-essential protobuf-compiler     libprotoc-dev libboost-all-dev libleveldb-dev hdf5-tools libhdf5-serial-dev     libopencv-core-dev  libopencv-highgui-dev libsnappy-dev libsnappy1     libatlas-base-dev cmake libstdc++6-4.8-dbg libgoogle-glog0 libgoogle-glog-dev     libgflags-dev liblmdb-dev git python-pip gfortran

sudo apt-get clean

You will get a dialog regarding the menu.lst file, just choose the default option it gives you.

For an explanation of why this is needed, see Caffe on EC2 Ubuntu 14.04 Cuda 7 and search for this command.

sudo apt-get install -y linux-image-extra-`uname -r` linux-headers-`uname -r` linux-image-`uname -r`

sudo apt-get install -y cuda
sudo apt-get clean

Verify

nvidia-smi

Add to ~/.bashrc

export PATH=$PATH:/usr/local/cuda-7.0/bin
export LD_LIBRARY_PATH=:/usr/local/cuda-7.0/lib64

Install cdnn

Download cudnn then

tar -zxf cudnn-7.0-linux-x64-v3.0-prod.tgz
cd cuda
sudo cp lib64/* /usr/local/cuda/lib64/
sudo cp include/cudnn.h /usr/local/cuda/include/
sudo ldconfig /usr/local/cuda/lib64/

Installing Caffe

Install the dependencies:

sudo apt-get install -y libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev protobuf-compiler gfortran libjpeg62 libfreeimage-dev libatlas-base-dev git python-dev python-pip libgoogle-glog-dev libbz2-dev libxml2-dev libxslt-dev libffi-dev libssl-dev libgflags-dev liblmdb-dev python-yaml python-numpy

Then run:

sudo easy_install pillow

You could have "TypeError: 'NoneType' object is not callable" error when installing pillow, then try:

sudo apt-get install pypy-dev

Now we can download Caffe. Navigate to the directory of your choice for the cloning.

cd ~
git clone https://github.com/BVLC/caffe.git

Add to ~/.bashrc

export CAFFE_ROOT=/home/username/caffe

We now install more dependencies. Warning: This takes 10-30 minutes.

cd caffe
cat python/requirements.txt | xargs -L 1 sudo pip install

Now we update the Makefile:

cp Makefile.config.example Makefile.config
vi Makefile.config

Uncomment the line: USE_CUDNN := 1 Make sure the CUDA_DIR correctly points to our CUDA installation. If you want the Matlab wrapper, uncomment the appropriate MATLAB_DIR line. If you add a layer in python, uncomment the line WITH_PYTHON_LAYER=true

Now we build Caffe. Set X to the number of CPU threads (or cores) on your machine. Use the command htop to check how many CPU threads you have.

make all -jX
make pycaffe -jX
make test -jX

Add this line to ~/.bashrc to use python from caffe

export PYTHONPATH=/path/to/caffe/python:$PYTHONPATH

then make ~/.bashrc valid.

source ~/.bashrc

Verify

python -c 'import caffe'

If something goes wrong with matplotlib, reinstall it

sudo pip uninstall matplotlib
sudo apt-get install python-matplotlib

References

http://tleyden.github.io/blog/2015/11/22/cuda-7-dot-5-on-aws-gpu-instance-running-ubuntu-14-dot-04/

https://iqbalnaved.wordpress.com/2016/05/08/installing-caffe-and-pycaffe-on-ubuntu-14-04/

https://github.com/BVLC/caffe/wiki/Install-Caffe-on-EC2-from-scratch-(Ubuntu,-CUDA-7,-cuDNN-3)

https://github.com/BVLC/caffe/wiki/Caffe-on-EC2-Ubuntu-14.04-Cuda-7

http://tleyden.github.io/blog/2015/11/22/running-neural-style-on-an-aws-gpu-instance/