Skip to content

Jupyter Notebook#

Before Starting#

Jupyter Notebook is not an HPC application and no support beyond this documentation will be provided by SCITAS.

The Jupyter documentation can be found here

TL;DR#

To run interactively Jupyter Notebook on a node it boils down to this operations

<front-node>$ Sinteract 
<node>$ ipnport=$(shuf -i8000-9999 -n1)
<node>$ jupyter-notebook --no-browser --port=${ipnport} --ip=$(hostname -i)

Installing Jupyter Notebook#

The key to successfully installing the tools is using python virtual environments. Please note that you will need to install it for each cluster

Load a compiler and python using modules#

$ module load gcc python
or 
$ module load intel python

Create and activate a virtual environment#

Environment

Shown here for GCC but the same applies for the Intel compiler.

$ virtualenv --system-site-packages opt/`hostname`/venv-gcc
$ source opt/`hostname`/venv-gcc/bin/activate
# for example on izar this will create a venv in opt/izar/venv-gcc
(venv-gcc) [user@izar ~]$
A more detailed article about Python virtual environments is available here.

Install Jupyter#

(venv-gcc) [user@izar ~]$ pip install jupyter

Running Jupyter notebook#

The easiest way to use Jupyter Notebook is via the Sinteract tool which gives interactive access to a compute node:

Getting access to a node interactivaly#

On Izar
[user@izar ~]$ Sinteract -g gpu:1 -c10  -t 1:0:0 -m 32G
[user@<node> ~]$ source opt/izar/venv-gcc/bin/activate
On Jed / Helvetios
[user@jed ~]$ Sinteract -c 18 -t 01:00:00 -m 120G
[user@<node> ~]$ source opt/jed/venv-gcc/bin/activate

Running Jupyter Notebook on the node#

(venv-gcc) [user@<node> ~]$ ipnport=$(shuf -i8000-9999 -n1)
(venv-gcc) [user@<node> ~]$ jupyter-notebook --no-browser --port=${ipnport} --ip=$(hostname -i)

This will start a Jupyter Notebook on the node and expose 2 addresses with the following format: http://<IP ADDRESS>:<PORT NUMBER/?token=<TOKEN>. For example,

Jupyter Notebook 6.5.4 is running at:
http://10.91.27.45:9116/?token=f124bfbf16824f8848d367943a2ac5f5404dabcd2136e5d4
 or http://127.0.0.1:9116/?token=f124bfbf16824f8848d367943a2ac5f5404dabcd2136e5d4

Making Jupyter Notebook accessible from the local machine#

local-machine$ ssh -L <PORT NUMBER>:<IP ADDRESS>:<PORT NUMBER> -l <USERNAME> <front-node> -f -N

If we take the example of the notebook from before on izar 

ssh -L 9116:10.91.27.45:9116 -l <user> izar.hpc.epfl.ch -f -N

At this point you should be able to open http://127.0.0.1:9116/?token=f124bfbf16824f8848d367943a2ac5f5404dabcd2136e5d4 in a local web browser