Jupyter Notebooks – High Performance Computing Group

How to run Jupyter Notebooks on the IDUN HPC cluster?

1. Use Open OnDemand https://apps.hpc.ntnu.no

We recommend to use https://apps.hpc.ntnu.no to run Jupyter Notebook:

2. For users with experience

This video shows more flexible steps from this article:

Use terminal to connect idun-login1.hpc.ntnu.no or idun-login2.hpc.ntnu.no via ssh:

ssh USERNAME@idun-login1.hpc.ntnu.no

Create Slurm job file jupyter.slurm for CPU job:

#!/bin/sh
#SBATCH --partition=short
#SBATCH --account=YOUR-ACCOUNT
#SBATCH --time=08:00:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --mem=4G
#SBATCH --job-name="jupyter"
#SBATCH --output=jupyter.out
module load Anaconda3/2022.05
jupyter notebook --no-browser

Or create Slurm job file jupyter.slurm for GPU job:

#!/bin/sh
#SBATCH --partition=GPUQ
#SBATCH --account=YOUR-ACCOUNT
#SBATCH --time=08:00:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --mem=4G
#SBATCH --job-name="jupyter"
#SBATCH --output=jupyter.out
#SBATCH --gres=gpu:1
module load Anaconda3/2022.05
jupyter notebook --no-browser

Submit this job :

$ sbatch jupyter.slurm
Submitted batch job 17679480

Show job details to get node name and log file location:

$ scontrol show job 17679480
JobId=17679480 JobName=jupyter
. . .
   JobState=RUNNING Reason=None Dependency=(null)
. . .
   NodeList=idun-90-01
. . .
   StdOut=/cluster/home/USER/jupyter.out

Print log file on the screen to copy last line with link and port number:

$ cat /cluster/home/USER/jupyter.out
. . .
http://127.0.0.1:8889/?token=1c85c0d74a8a0bc. . . .183b136342f680ad1

Next setep is to create port forwarding with ssh jump.

Via VPN or via the university network

Open new terminal window and use this command. Replace hilighted text with your data.

ssh -L 88xx:127.0.0.1:88xx -J USER@idun-login1.hpc.ntnu.no USER@NODE-NAME

Without VPN or outside of the university network

For students:

ssh -L 88xx:127.0.0.1:88xx -J USER@login.stud.ntnu.no,USER@idun-login1.hpc.ntnu.no USER@NODE-NAME

For employees:

ssh -L 88xx:127.0.0.1:88xx -J USER@login.ansatt.ntnu.no,USER@idun-login1.hpc.ntnu.no USER@NODE-NAME

Do not close this terminal window. It will keep port forwarding active.

Open the web browser and use the link from the job log file: