Overview of CUDA architecture and programming model – GPU evolution – CUDA GPU architecture

Basic CUDA programming – Brief revise of CUDA programming model – Key principles – Introduction to the concept of threads & blocks – Host-device data transfer

Hands-on exercises on writing simple CUDA programs – Using CUDA on HPC cluster – Simple programs with C/C++

Overview of CUDA memory hierarchy – An overview of memory levels – Global memory – Registers, constant memory, texture memory – Shared memory and synchronization

Introduction to CUDA Deep Neural Network library (cuDNN) – Using cuDNN for deep neural networks – Convolutional neural networks in cuDNN – Integration with other CUDA libraries (cuBLAS, cuSOLVER, cuRAND, cuTENSOR, TensorRT)

Exercises on CUDA techniques: image processing and neural networks – Image convolution filtering – Implementation of neural network from scratch with C/C++ – Implementation of neural network using cuDNN