2. Storage on the LRZ AI Systems
Overview
The LRZ AI Systems are integrating the full offer of LRZ Data Science Storage (DSS) systems. The following table provides an overview of the available storage options:
| Storage Pool | Designated Use | Top-level Directory | Free Size Limit | Automated Backup | Expiration | Additional Information |
|---|---|---|---|---|---|---|
| Home directory | infrequently changing user files, scripts, configuration files, etc. Not suitable for heavy and/or high-frequency I/O operations, i.e. most machine learning applications. Use the AI Systems DSS instead. | /dss/dsshome1/.../<user> | 100 GB | Yes, backup to tape and file system snapshots | Lifetime of LRZ project | File Systems and IO on Linux-Cluster |
| AI Systems DSS | high-bandwith, low-latency I/O | /dss/dssfs04 | up to 4 TB (or 20+ TB with associated costs) | No for free tier / Yes for paid option | Until further notice | see below (or DSS on demand offer with associated costs) |
Linux Cluster DSS | general purpose, long-term data storage | /dss/dssfs02 | up to 10 TB | No for free tier / Yes for paid option | Lifetime of data project | File Systems and IO on Linux-Cluster (or DSS on demand offer with associated costs) |
Exclusive/private DSS systems | specified by system owner | /dsslegfs01 | specified by system owner | specified by system owner | specified by system owner | Data Science Storage ("joint project offer") |
Use the following command on the login nodes to get an overview of all individually accessible DSS containers and their utilization:
$ dssusrinfo all
Home directory
The LRZ AI Systems provide a unified home directory together with the LRZ Linux Cluster (cf. File Systems and IO on Linux-Cluster). This is shared among all the LRZ AI Systems resources described here: Description and Resources.
User home directories can, of course, be accessed from the login.ai.lrz.de SSH-login nodes as well as the https://login.ai.lrz.de web-front end (via a web-based file manager). These directories are located in a dedicated Data Science Storage (DSS) container managed by LRZ and are created when Linux Cluster access is granted.
AI Systems DSS
There is a dedicated AI Systems DSS system available, providing access to a high-performance, SSD-based network storage. This system is intended for high-bandwidth, low-latency I/O operations, serving the demands of modern day AI applications.
Access to this storage is granted upon request by a LRZ project's Master User through the LRZ Servicedesk.
A quota of up to 4 TB, 8.000.000 files and maximum 3 DSS containers (commonly: a single container) will be assigned. Once granted, the Master User will subsequently act as DSS Data Curator and manage the assigned storage quotas for their project.
AI Systems DSS storage space can be requested by submitting a service request to LRZ Servicedesk. By using this link, you will open a ticket for the appropriate service. Select "AI topics", next "Master user only: Application for project storage space (DSS AI)" and confirm. Eventually, fill in and submit the form.
Linux Cluster DSS
Additional storage, mainly targeted at long-term storage needs, can be requested as described here File Systems and IO on Linux-Cluster (up to 10TB) or as DSS on demand offer (20TB or more with associated costs).
Exclusive/private DSS systems
As "joint project offer", DSS systems can be purchased, implemented and housed exclusively, for a private group of dedicated users, see Data Science Storage.