Introduction to Parallel Programming Using MPI

• Jakub Gałecki (ICM UW)

- Introduction: the need for message passing, the SPMD model, working with distributed memory - Basic concepts: communicator, rank, message - Point-to-point communication: the basic building block of MPI programs - Collective communication: expressing distributed parallel algorithms and common communication patterns - Non-blocking communication: how to overlap computation and communication - Hybrid parallelism: leveraging MPI to scale other parallel paradigms - Launching MPI programs on HPC machines - Hands-on exercise: distributed inner product - Hands-on exercise: halo exchange