Deep Learning Week @ LRZ 2022

OVERVIEW

In this intensive course week, packed with lectures about Deep Learning and AI, you will learn:

• to train and deploy deep neural networks to solve computer vision problems,
• the fundamentals of machine learning for working with texts,
• to use transformer-based natural language processing models for advanced tasks involving languages (e.g., categorising documents),
• to effectively parallelize training of deep neural networks on single and Multi-GPUs, and
• how to leverage the LRZ AI Systems to perform all the above tasks. 

This online workshop combines lectures about Fundamentals of Deep Learning for single and for Multi-GPUs, Building Transformer-Based Natural Language Processing Applications and Deep Learning on LRZ systems.

The lectures are interleaved with many demos and hands-on sessions using Jupyter Notebooks. For days 1 to 4, students will have access to a fully configured GPU-accelerated workstation in the AWS cloud. On day 5, the capabilities of the LRZ AI System will be showcased. 

The workshop is co-organised by LRZ and NVIDIA Deep Learning Institute (DLI). Material developed by NVIDIA is supplemented by vendor-neutral material developed by LRZ. All instructors are NVIDIA certified University Ambassadors.

NVIDIA Deep Learning Institute

The NVIDIA Deep Learning Institute delivers hands-on training for developers, data scientists, and engineers. The program is designed to help you get started with training, optimizing, and deploying neural networks to solve real-world problems across diverse industries such as self-driving cars, healthcare, online services, and robotics.

DLI TRAINING SETUP

To get started, follow these steps:

1. Create an NVIDIA Developer account at http://courses.nvidia.com/join Select "Log in with my NVIDIA Account" and then '"Create Account".
2.  Make sure that WebSockets works for you:
   
   • Test your Laptop at http://websocketstest.com
   • Under ENVIRONMENT, confirm that '"WebSockets" is checked yes.
   • Under WEBSOCKETS (PORT 80]. confirm that "Data Receive", "Send", and "Echo Test" are checked yes.
3. lf there are issues with WebSockets, try updating your browser.
   We recommend Chrome, Firefox, or Safari for an optimal performance.
4. Visit http://courses.nvidia.com/dli-event and enter the event code provided by the instructor.
5. You're ready to get started. Please complete the survey at the end of the course to share your feedback.

HOW TO ACCESS LRZ SYSTEMS

On the last day the LRZ AI systems and the LRZ compute cloud will be used for the hands-on sessions.

For details see the slides of Day 5.

1st day: Fundamentals of Deep Learning

Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)

AGENDA (all times in CEST)

10:00-10:20  Welcome and Intro

10:20-12:00  Introduction to Deep Learning and Deep Learning Training

12:00-13:00  Lunch Break

13:00-14:20  Introduction to Convolutional Neural Networks and Data Augmentation 

14:20-14:30  Coffee Break

14:30-15:45  Deployment, Pre-Trained Models and Transfer Learning

15:45-16:00  Q&A

SLIDES

• dli-intro-lrz-2022-v1.pdf
• FDL.pdf

DOCUMENTATION

• Python

2nd day: Fundamentals of Deep Learning for Multi-GPUs

Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)

AGENDA (all times in CEST)

10:00-10:15  Intro

10:15-12:00  Stochastic Gradient Descent

12:00-13:00  Lunch Break

13:00-14:20  Introduction to Distributed Training

14:20-14:30  Coffee Break

14:30-15:45  Algorithmic Challenges of Distributed SGD

15:45-16:00  Q&A

SLIDES

• Fundamentals of Deep Learning for Multiple GPUs
• A Casual Introduction for Deep Learning Training

3rd day: Fundamentals of machine learning for working with texts and Building Transformer-Based Natural Language Processing Applications: Part 1

Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)

AGENDA (all times in CEST)

10:00-10:15  Intro

10:15-12:00 Working with text: From classifying images to generating Text

12:00-13:00  Lunch Break

13:00-14:20 Machine Learning for Natural Language Processing: Fundamentals

14:20-14:30  Coffee Break

14:30-15:45 Machine Learning for Natural Language Processing: Attention is all you need. 

15:45-16:00  Q&A

SLIDES

• Working with text: preliminaries
• Machine Learning for NLP

4th day: Fundamentals of machine learning for working with texts and Building Transformer-Based Natural Language Processing Applications: Part 2

Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)

AGENDA (all times in CEST)

10:00-10:15  Intro

10:15-12:00 Transformer Based Models for Natural Language Processing

12:00-13:00  Lunch Break

13:00-14:20 Hands On: Text Classifier and Named Entity Recognition

14:20-14:30  Coffee Break

14:30-15:45 Transformer Based Networks in Production 

15:45-16:00  Q&A

SLIDES

• Transformers_and_their_deployment.pdf

5th day:  Deep Learning on LRZ Systems

Lecturer: PD Dr. Juan Durillo Barrionuevo (LRZ)

AGENDA (all times in CEST)

10:00-10:15  Intro

10:15-12:00 Introduction to the LRZ AI Infrastructure

12:00-13:00  Lunch Break

13:00-14:20 Introduction to the Compute Cloud for AI Inference and AI development

14:20-14:30  Coffee Break

14:30-15:45 Hands on: Configuring a Cloud Instance for running AI workloads via JupyterLab

15:45-16:00  Q&A

SLIDES

• AI@LRZ_DeepLearningWeek.pdf
• An Example of Jupyter Notebook: text_classification.ipynb
• dli-wrapup-2022-v1.pdf

Survey

• Please fill out the online survey under https://survey.lrz.de/index.php/441328?lang=en 
• This helps us to
  
  • increase the quality of the courses,
  • design the future training programme at LRZ, GCS and in Europe according to your needs and wishes,
  • get future funding for training events.

NEXT STEPS

Visit the NVIDIA Deep Learning lnstitute's website at https://www.nvidia.com/en-us/training/ to access more training and resources.

• Start online, self-paced training in deep learning and accelerated computing (using the account you created today).
• View upcoming workshops around the world and request an onsite workshop at your company or organization.
• Learn about the University Ambassador Program.