Skip to content

Jupyter Notebook#

Before Starting#

Jupyter Notebook is not an HPC application and no support beyond this documentation will be provided by SCITAS.

The Jupyter documentation can be found here

In Brief#

To run interactively Jupyter Notebook on a node it boils down to this operations

<front-node>$ Sinteract
<node>$ ipnport=$(shuf -i8000-9999 -n1)
<node>$ jupyter notebook --no-browser --port=${ipnport} --ip=$(hostname -i)

Installing Jupyter Notebook#

The key to successfully installing the tools is using python virtual environments. Please note that you will need to install it for each cluster

Load a compiler and python using modules#

module load gcc python

or

module load intel python

Create and activate a virtual environment#

Environment

Shown here for GCC but the same applies for the Intel compiler.

$ virtualenv --system-site-packages opt/`hostname`/venv-gcc
$ source opt/`hostname`/venv-gcc/bin/activate
# for example on izar this will create a venv in opt/izar/venv-gcc
(venv-gcc) [user@izar ~]$

A more detailed article about Python virtual environments is available here.

Install Jupyter#

(venv-gcc) [user@izar ~]$ pip install jupyter

Running Jupyter notebook#

The easiest way to use Jupyter Notebook is via the Sinteract tool which gives interactive access to a compute node:

Getting access to a node interactivaly#

On Izar
[user@izar ~]$ Sinteract -g gpu:1 -c10  -t 1:0:0 -m 32G
[user@<node> ~]$ source opt/izar/venv-gcc/bin/activate
On Jed / Helvetios
[user@jed ~]$ Sinteract -c 18 -t 01:00:00 -m 120G
[user@<node> ~]$ source opt/jed/venv-gcc/bin/activate
On Kuma
[user@kuma1 ~]$ Sinteract -c 16 -p h100 -g gpu:1 -m 64GB -t 01:00:00
[user@<node> ~]$ source opt/kuma1/venv-gcc/bin/activate

Running Jupyter Notebook on the node#

(venv-gcc) [user@<node> ~]$ ipnport=$(shuf -i8000-9999 -n1)
(venv-gcc) [user@<node> ~]$ jupyter notebook --no-browser --port=${ipnport} --ip=$(hostname -i)

This will start a Jupyter Notebook on the node and expose 2 addresses with the following format: http://<IP ADDRESS>:<PORT NUMBER/?token=<TOKEN>. For example:

To access the server, open this file in a browser:
    file:///home/peyberne/.local/share/jupyter/runtime/jpserver-1293548-open.html
Or copy and paste one of these URLs:
    http://10.91.54.11:8138/tree?token=e5bf66fbeafaf6a70b93513e39afb257351f9c5630f34688
    http://127.0.0.1:8138/tree?token=e5bf66fbeafaf6a70b93513e39afb257351f9c5630f34688

Making Jupyter Notebook accessible from the local machine#

local-machine$ ssh -L <PORT NUMBER>:<IP ADDRESS>:<PORT NUMBER> -l <USERNAME> <front-node> -f -N

If we take the example of the notebook from before on kuma

ssh -L 9116:10.91.27.45:9116 -l <user> kuma.hpc.epfl.ch -f -N

At this point you should be able to open http://127.0.0.1:9116/?token=f124bfbf16824f8848d367943a2ac5f5404dabcd2136e5d4 in a local web browser

Jupyter Lab instead of Jupyter Notebook#

If you want to Jupyter Lab instead of Jupyter Notebook, there are only two minor changes to the process described above:

  1. The command uses lab instead of notebook:

    jupyter lab --no-browser --port ${ipnport} --ip=$(hostname -i)

  2. You need to add lab to the URL:

    http://127.0.0.1:9116/lab?token=<session token>