NGC Container Environment Modules

NGC container environment modules are lightweight wrappers that make
it possible to transparently use NGC
containers as environment modules.

• Use familiar environment module commands, ensuring a minimal
  learning curve or change to existing workflows

• Leverage all the benefits of containers, including portability and
  reproducibility

• Take advantage of the optimized HPC and Deep Learning containers
  from NGC

Synopsis

$ git clone https://github.com/NVIDIA/ngc-container-environment-modules
$ module use $(pwd)/ngc-container-environment-modules
$ module load gromacs
$ gmx

The gmx command on the host is transparently mapped into the GROMACS
container. If the container image is not already present, then the
image will be pulled and cached on first use. Since Singularity
automatically mounts $HOME, /tmp, and the current working
directory into the container, the container environment modules provide
nearly transparent access to NGC containers.

Note: Add the path to the NGC container environment modules to
MODULEPATH to skip the module use step.

Prerequisites

• Lmod
• Singularity 3.4.1 or later

Modification

The NGC container environment modules are a reference. It is expected
that they will need some modification for the local environment.

Some of the expected changes are:

1. The name of the Singularity module. The container environment
   modules try to load the Singularity module (note the capital
   'S'). Set the NGC_SINGULARITY_MODULE environment variable if the
   local Singularity module is named differently (set it to none if
   no Singularity module is required).

2. Module conflicts. The container environment modules set module
   conflicts based on the commands mapped into the container. Sites
   may want to modify the list of conflicting modules to prevent
   conflicts between containers or other environment modules.

3. Container image cache. The container environment modules can either
   pull NGC container images as needed or use a library of
   pre-downloaded container images. Sites may wish to modify the
   modules to only support one of these modes. Sites may also want to
   hard-code the path to the library of pre-downloaded container
   images rather than using the NGC_IMAGE_DIR environment variable.

4. Mount additional directories into the containers. Sites may have
   filesystems that should be visible to all user processes, such as a
   /scratch filesystem or a set of shared datasets. In this case,
   set SINGULARITY_BINDPATH, either globally, or in the container
   environment modules.

Examples

Basic

Download a GROMACS benchmark
to run this example.

$ module load gromacs/2020.2
$ gmx mdrun -ntmpi 1 -ntomp 40 -v -pin on -nb gpu --pme gpu --resetstep 12000 -nsteps 20000 -nstlist 400 -noconfout -s topol.tpr

Interactive

$ module load pytorch/20.02-py3
$ python3
>>> import torch
>>> x = torch.randn(2,3)

Jupyter notebooks

$ module load rapidsai
$ jupyter notebook --ip 0.0.0.0 --no-browser --notebook-dir /rapids/notebooks

Multi-node MPI

Download the LAMMPS Lennard Jones
fluid dataset to run this
example.

$ module load lammps/15Jun2020
$ mpirun -n 2 lmp -in in.lj.txt -var x 8 -var y 8 -var z 8 -k on g 2 -sf kk -pk kokkos cuda/aware on neigh full comm device binsize 2.8