Apptainer (former Singularity)

Apptainer/Singularity is the widely used container system for HPC. You can use Docker container to create Apptainer containers.

Apptainer documentation is here: https://apptainer.org/docs/user/latest/build_a_container.html

1. Create on login node

You can create apptainer on your computer and upload it to the IDUN. But this example shows how to create an Apptainer container on IDUN login node.

The goal: I have a python script "myscript.py" and I need to run it on Ubuntu Linux for some reason 🙂

1.1. Create sandbox container and install missing software.

On login node you need to use options "--no-mount hostfs" to ignore other filesystems on login nodes. And you need to use option "--fakeroot" that allows an unprivileged user to run a container with the appearance of running as root.

$ apptainer build --sandbox myapp1 docker://ubuntu
$ apptainer shell --no-mount hostfs --fakeroot --writable myapp1
> apt update
> apt install python3
> apt install python3-pip
> pip3 install pandas
> exit

1.2. Convert sandbox container to SIF format and run test

$ apptainer build myapp1.sif myapp1
$ apptainer run myapp1.sif python3 -V
Python 3.10.6

1.3. This is python script and Slurm job file

Python script: myscript.py

import pandas as pd
data = pd.DataFrame({"x1":["y", "x", "y", "x", "x", "y"],
                     "x2":range(16, 22),
                     "x3":range(1, 7),
                     "x4":["a", "b", "c", "d", "e", "f"],
                     "x5":range(30, 24, - 1)})
print(data)

Slurm job file: apptainer.slurm

#!/bin/sh
#SBATCH --partition=CPUQ
#SBATCH --account=<Your_Account>
#SBATCH --time=0-00:15:00     # 0 days and 15 minutes limit
#SBATCH --nodes=1             # 1 compute nodes
#SBATCH --cpus-per-task=1     # 1 CPU cores
#SBATCH --mem=5G              # 5 gigabytes memory
#SBATCH --output=apptainer.txt
apptainer exec myapp1.sif python3 myscript.py

1.4. Run the job and check the result

$ sbatch apptainer.slurm 
Submitted batch job 18636396

Wait and check job status:

$ scontrol show job 18636396
. . .
   JobState=COMPLETED Reason=None Dependency=(null)

Check output log file:

$ cat apptainer.txt
  x1  x2  x3 x4  x5
0  y  16   1  a  30
1  x  17   2  b  29
2  y  18   3  c  28
3  x  19   4  d  27
4  x  20   5  e  26
5  y  21   6  f  25
Scroll to Top