General

The NVIDIA HPC Software Development Kit (HPC-SDK) includes the proven compilers, core, math and communication libraries, profilers and debugger needed to compile a broad variety of mostly scientific software to make full, partial or hybrid use of NVIDIA GPUs in HPC environments. Apart from C, C++, Fortran and CUDA compilers, the HPC-SDK incorporates not only one or more NVIDIA CUDA Toolkits but also a full NVIDIA HPC-X stack. NVIDIA HPC-X is a comprehensive software package that includes Message Passing Interface (MPI), Symmetrical Hierarchical Memory (SHMEM) and Partitioned Global Address Space (PGAS) communications libraries, and various acceleration packages. This full-featured and tested package enables to achieve high performance, scalability, and efficiency and ensures that communication libraries are fully optimized by NVIDIA networking solutions, which are used on LiCCA.

The following component versions are included:

Compilers

nvc , nvc++ and nvfortran were formerly known as the PGI compilers before being aquired by NVIDIA.

There is also the nvcc binary, which is called a CUDA compiler driver. nvcc accepts both C/C++ as well as CUDA code but needs a host compiler (typically gcc or g++ from the GCC suite) for most applications. Further details on the nvcc compilation process can be found here. If you just need nvcc , then loading the CUDA Toolkit is a more lightweight alternative should be preferred over the HPC-SDK.

Lmod Modules

Main Modules

At the time of writing, the following modules are available:

ml nvhpc/cu11.8/23.5
ml nvhpc/cu11.8/23.11
ml nvhpc/cu11.8/24.1

ml nvhpc/cu12.1/23.5
ml nvhpc/cu12.3/23.11
ml nvhpc/cu12.3/24.1

ml nvhpc/23.5
ml nvhpc/23.11
ml nvhpc/24.1

HPC-X Submodules

By default the HPC-X module hpcx is loaded automatically right after loading any of the above HPC-SDK modules. While this is the desired behaviour for most usecases, there are also some other HPC-X modules available:

• hpcx-mt This module enables multi-threading support in all of the HPC-X components. Please use this module in order to run multi-threaded applications.
• hpcx-prof This module enables UCX compiled with profiling information.
• hpcx-debug This module enables UCX/HCOLL/SHARP compiled in debug mode.
• hpcx-stack This module is the same as hpcx , however all MPI functionality is excluded. This is what is also sometimes referred as nvhpc-nompi.

MPI using HPC-X

HPC-X is using the OpenMPI MPI implementation under the hood. To ensure maximum compatibilty with our hardware and Slurm, all HPC-X OpenMPI versions have been manually recompiled to take all components into account. It is therefore also recommended to run applications using HPC-X MPI using Slurm's srun utility.