Deep Learning and GPU programming using OpenACC @ HLRS 2022
Participants 2022 © HLRS 2022
Overview
Learn how to accelerate your applications with OpenACC, how to train and deploy a neural network to solve real-world problems, and how to effectively parallelize training of deep neural networks on Multi-GPUs.
The workshop combines lectures about Fundamentals of Deep Learning and Fundamentals of Deep Learning for Multi-GPUs with a lecture about Accelerated Computing with OpenACC.
The lectures are interleaved with many hands-on sessions using Jupyter Notebooks. The exercises will be done on a fully configured GPU-accelerated cluster in the cloud.
The first three days are organized in cooperation with LRZ (Germany) and NVIDIA. All instructors are NVIDIA certified University Ambassadors. On the last day, you will learn more about DL on the systems at HLRS.
1st day Fundamentals of Accelerated Computing with OpenACC
Lecturer: Dr. Volker Weinberg (LRZ) / Dr. Momme Allalen (LRZ)
AGENDA (all times in CEST)
09:00-09:30 Welcome & Intro
09:30-11:00 Profiling
11:00-11:15 Coffee Break
11:15-12:15 Introduction into NVIDIA® Nsight™ Systems
12:15-12:45 Lab 1
12:45-13:15 OpenACC Directives
13:15-14:15 Lunch
14:15-14:45 Lab 2
14:45-15:15 GPU Programming
15:15-15:45 Lab 3
15:45-16:00 Coffee Break
16:00-16:30 Data Management
16:30-16:45 Lab 4
16:45-17:00 Q&A, Final Remarks
SLIDES
- dli-intro-hlrs-2022-v1.pdf
- Intro_Nsight Systems.pdf
- Module1.pdf
- Module2.pdf
- Module3.pdf
- Module4.pdf
- Module5.pdf
- Module6.pdf (for self-study)
DOCUMENTATION
2nd day: Fundamentals of Deep Learning
Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)
AGENDA (all times in CEST)
09:00-09:20 Welcome and Intro
09:20-11:00 Introduction to Deep Learning and Convolutional Neural Networks
11:00-11:15 Coffee Break
11:00-12:00 How a Neural Network Trains
12:00-13:15 Convolutional Neural Networks
13:15-14:15 Lunch Break
14:15-15:45 Data Augmentation, Deployment and Pre-Trained Models
15:45-16:00 Coffee Break
16:00-16:45 Advanced Architectures
16:45-17:00 Q&A
SLIDES
DOCUMENTATION
3rd day: Fundamentals of Deep Learning for Multi-GPUs
Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)
AGENDA (all times in CEST)
09:00-09:15 Intro
09:15-11:00 Stochastic Gradient Descent, a focus session on how training works
11:00-11:15 Coffe Break
11:15-12:00 Hands On Session on Gradient Descent
12:00-13:15 Introduction to Distributed Training
13:15-14:15 Lunch Break
14:15-15:00 Hands On Session on Distributed Training
15:00-15:45 Algorithmic Challenges of Distributed SGD
15:45-16:00 Coffee Break
16:00-16:45 Hands on session on Algorithmic Challenges of Distributed SGD
16:45-17:00 Q&A
SLIDES
- A Casual Introduction to Neural Network Training and Implications for Multiple GPUs
- Fundamentals_of_deep_learning_for_multiple_gpus.pdf
Survey
- Please fill out the online survey under tbd.
- This helps us to
- increase the quality of the courses,
- design the future training programme at LRZ, GCS and in Europe according to your needs and wishes,
- get future funding for training events.
NEXT STEPS
Visit the NVIDIA Deep Learning lnstitute's website at https://www.nvidia.com/en-us/training/ to access more training and resources.
- Start online, self-paced training in deep learning and accelerated computing (using the account you created today).
- View upcoming workshops around the world and request an onsite workshop at your company or organization.
- Learn about the University Ambassador Program.