Running Jupyter notebooks

This provides a system similar to the one we had in atticus, but in your own computer. You can use this everywhere, the only requirement is that you need to have permissions to run Docker. (This means you can do it in macOS if you have an admin account, but you cannot do it in the Linux machines at work).

Install Docker

You should have it installed in your Macs, but just in case you should install Docker.

You probably want to set it up so it starts when you log in. Lauch Docker for the first time like any other application and then set the preferences by clicking in the whale icon in the macOS menu bar (close to the clock).

Copy your old notebooks in your make

You should have a backup copy of your files from atticus. Say you want to copy your files in a folder called ~/Projects (change below for whatever name you want):

$ mkdir ~/Projects
$ cp ~/Downloads/jupyter-user-iglpdc-backup-2018-05-31.tar.bzip ~/Projects
$ cd ~/Projects
$ tar -xvjf jupyter-user-iglpdc-backup-2018-05-31.tar.bzip
$ ls
./notebooks

Your files should be in the ~/Projects/notebooks folder, exactly as they were in atticus.

Run Jupyter

You can use any of these commands. Pick you favorite and save it into a script to avoid mistakes (see below).

Run Jupyterlab

A newer interface for Jupyter (I think it's better, but you will have to get used to the new menus)

$ docker run -it --rm -v ~/Projects/notebooks:/work -p 8888:8888 jupyter/datascience-notebook start.sh jupyter lab 
Execute the command: jupyter lab
[I 19:11:58.529 LabApp] Writing notebook server cookie secret to /home/jovyan/.local/share/jupyter/runtime/notebook_cookie_secret
[W 19:12:01.356 LabApp] WARNING: The notebook server is listening on all IP addresses and not using encryption. This is not recommended.
[I 19:12:01.413 LabApp] JupyterLab alpha preview extension loaded from /opt/conda/lib/python3.6/site-packages/jupyterlab
[I 19:12:01.446 LabApp] Serving notebooks from local directory: /home/jovyan
[I 19:12:01.446 LabApp] 0 active kernels
[I 19:12:01.446 LabApp] The Jupyter Notebook is running at:
[I 19:12:01.446 LabApp] http://[all ip addresses on your system]:8888/?token=d9a226a357704a5698a136902c0ed7e3ed98188330556aa5
[I 19:12:01.446 LabApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 19:12:01.447 LabApp] 
    
    Copy/paste this URL into your browser when you connect for the first time,
    to login with a token:
        http://localhost:8888/?token=d9a226a357704a5698a136902c0ed7e3ed98188330556aa5

The last line has a link. Click on it and a jupyter will open in your browser. To close it, go back to the terminal and type ^C (that's the Ctrl and the c key pressed at the same time.)

If you have place your stuff in a different place other than ~/Projects/notebooks, you have to change in the command above -v ~/Projects/notebooks to wherever path your notebooks are.

Run Jupyter as in atticus

$ docker run -it --rm -v ~/Projects/notebooks:/work -p 8888:8888 jupyter/datascience-notebook
...
(Very similar output also with a link at the end.)
...

Use data in another folder/computer (advanced)

Works with any of the above.

Say you have data in another machine, you need to work with this data, but don't want to copy it over to your Mac. You need to do two things.

First, mount the other computer's filesystem in your Mac. You need to ask helpdesk for that. Tell them you need "to mount the hard drive of your Linux workstation into your Mac". They will set it up and tell you how to connect to it.

Second, make this mounted folder available to Docker. Your workstation folder will show up in your Mac in /some/path/in/your/Mac after the first step above. To make these files available in your notebooks add another -v option to your docker run command:

$ docker run -it --rm -v ~/Projects/notebooks:/work /path/to/your/data:/my_data -p 8888:8888 jupyter/datascience-notebook start.sh jupyter lab 

Inside the notebook, your data files will show up in the /my_data folder.

Running more than one server at once

Say that you are working on a project and what to work on a second project that requires some data somewhere else. If you try to run the commands above twice, docker complain becuase you are trying to use the same port (the URL you jupyter gives you) for two different things. The solution is to run the second instance in a different port. You just have to change the first 8888 in the command to some other number. Again, change only the first, the second 8888 has to stay 8888. You can choose any number that is not taken and is larger than 1028, so go - p 9999:8888 or -p 8989:8888 or whatever.

Save the command into a script

Create a file with Atom (or other text editor) with the following (or any of the options above):

#!/usr/bin/env bash
docker run -it --rm -v ~/Projects/notebooks:/work -p 8888:8888 jupyter/datascience-notebook start.sh jupyter lab 

Save it as run-jupyter-lab.sh``. You can put it anywhere, but the best is to put it into abinfolder in your$HOME`:

$ mkdir -p ~/bin # Create a bin folder in your home dir if you don't have it
$ mv run-jupyter-lab.sh ~/bin
$ chmod 755 ~/bin/run-jupyter-lab.sh # Make the file executable
$ echo 'export PATH="~/bin:$PATH"' >> ~/.bash_profile  # Add ~/bin to your path

(You could also copy the script attached set-up-jupyterlab-with-docker.sh into your computer annd run bash set-up-jupyterlab-with-docker.sh to do all above.)

Now you can do:

$ run-jupyter-lab.sh

from anywhere in your computer and will open jupyter with all the notebooks loaded.