RAM disk (tmpfs)

Every Job can make use of a local RAM disk located at /dev/shm , which has a significantly higher performance (both I/O operations per seconds as well as bandwidth) as the filesystem on the local SSD disk. The usage is similar to the local SSD storage. Contrary to disk storage, RAM disk storage requirements have to be added to the requested amount of RAM. The maximum size of the RAM disk is limited to approx. 50% of the total amount of RAM per node, i.e.  500G for nodes of the epyc and epyc-gpu nodes, and 2T for epyc-mem  nodes.

Given that your application requires 4G of RAM, and up to 8G of RAM disk storage will be used, you need to request at least #SBATCH --mem=12G of RAM. Failure to do so will result in your Job being terminated by the OOM (Out-Of-Memory) handler.

This directory is a private directory, it will only be seen by your Job.

Data retention policy

Data in /dev/shm will be deleted right after your Job terminates! Make sure that you copy back important files before your Job ends.

Handling Timelimit-situations for Jobs using the RAM disk.

If you are unsure how long your Job will take, it might run into the timelimit. Make sure you implement a mechanism to copy back important intermediate results in this case, because the private /dev/shm directory will be deleted right at the end (timeout or not) of a Job.

Do not submit Jobs with significantly more that 8G per CPU core on the epyc partition. Use the epyc-mem partition for high memory applications instead.

A typical Job using the RAM diskj has at least three steps:

  1. Copy necessary data from GPFS to /dev/shm
  2. Run your calculation there.
  3. Move results from the Node back to the GPFS.
Example (Script part only) 
#!/usr/bin/env bash

#SBATCH options ...

# Step 1
TMP=/dev/shm
cp job.inp job.dat $TMP

# Step 2
### change to dir $TMP
pushd $TMP
srun your_application
### change back to the starting dir
popd

# Step 3
### move the results back, for example 'job_result.out'
mv $TMP/job_result.out .

part of icons are from social media marketing icons Social media marketing icons created by Freepik - Flaticon