| Home| Lecture1 | Lecture2 | Lecture3 |

Lecture 3 - Programming models and environments for petascale computing

Prof. Marc Snir (Univ. of Illinois at Urbana-Champaign -USA)

• Description

A programming model is an essential mediator between the algorithm and the architecture:  it has to hide changing details of the architecture and provide a high-level view of the machine, while enabling algorithms to be specified to a point where they can be mapped efficiently onto the hardware by a compiler and run-time. The tension between abstraction and efficiency is   inherent, and is particularly acute when performance is at premium - in particular in high-performance computing: Elegant, high-level models have not provided acceptable performance, in the past; models that did provide acceptable performance have been tedious to use. Will the situation improve in the near future? 
We shall discuss in our course the forces that influence the design and implementation of parallel programming models and outline a taxonomy of parallel programming models. We shall survey the current state of the art in parallel programming and review emerging parallel programming models. In particular, we shall discuss PGAS languages (UPC, CAF and Titanium); the use of templates and objects for parallel language extensions in TBB and other frameworks; and the use of MapReduce for data intensive parallelism.  We shall survey ongoing research in parallel programming languages and speculate about the landscape of parallel programming in coming years.
While the course will not lead to programming proficiency in any specific parallel programming model, it will help practitioners choose the best programming model for each implementation task.

 

Timetable

Lecture 3.1 (1.5 hours) Parallel Programming Patterns: What is a parallel programming model supposed to express?

We shall discuss the “building blocks” of parallel algorithms such as task granularity, locality, communication and synchronization patterns, task scheduling and load balancing, parallel composition of modules, etc. (1.5 hour– possibly distributed along the course)

Lecture 3.2 (1 hour) What is a good parallel programming model?

We shall discuss what are desired features in parallel programming models and discuss how features of applications and of parallel architectures influence our choice of a model. We shall also discuss the various mechanisms for implementing a parallel programming model: language constructs, directives, libraries, language extensions, etc. We shall present a taxonomy of parallel programming models and description of the forces driving the choice of a suitable model. (1 hour – possibly distributed along the course)

Lecture 3.3 (1 hour) The state of the art:

We shall survey currently used parallel programming models: vector and SIMD programming; shared memory programming and OpenMP (loop and task parallelism); message-passing programming and MPI (double-sided, single-sided, collective, split communications). We shall discuss how these models fit existing architectures and applications and to what extent they satisfy the requirements discusses in (1). (1 hour)

Lecture 3.4 (0.5 hour) Hybrid programming:

Most large programs are written using multiple languages and models. We shall discuss issues related to hybrid programming models, such as OpenMP+MPI, C++ + Fortran, etc. (0.5 hour)

Lecture 3.5 (1.5 hour) PGAS programming languages:

We shall discuss the overall design of PGAS languages and how they are implemented.  We shall briefly survey and contrast UPC, CAF and Titanium. (1.5 hours)

Lecture 3.6 (2 hours) Object Frameworks and parallel programming though language extensions:

We shall discuss the use of OO language extension mechanisms (templates, objects and methods) for expressing parallelism, using examples such as A++/P++, Cactus and TBB; we shall briefly touch on implementation issues for such frameworks. (2 hours)

Lecture 3.7 (1 hour) Data-intensive parallelism:

We shall discuss the use of MapReduce and other similar models. (1 hour)

Lecture 3.8 (1.5 hours) Quo vadimus:

We shall discuss current research directions, such as the HPLS languages (Fortress, X10, Chapel), ZPL, etc.; and speculate about the landscape of parallel programming in coming years. (1.5 hours)

Hands-on session 3 - Programming models and environments for petascale computing

[Assistant Marc Tajchman (CEA/DEN Saclay), ]

• 1ère série : autour de MPI et OpenMP
• Séance 1
• Présentation du cas de simulation envisage (problème à N corps)
• Présentation du programme des TP
• Étude d'un code séquentiel (fourni)
• Parallélisation mémoire partagée (OpenMP)
• Tests de différentes options OpenMP
• Séance 2
• Second code séquentiel (fourni) avec notion de cellules
• Parallélisation type échange de messages (MPI)
• Test de différentes options MPI
• Séance 3
• Suite et fin du TP précédent autour de MPI
• Programmation hybride C - C++ - fortran – java (tests simples)
• Séance 4
• Parallélisation hybride (MPI + OpenMP) (Tests simples à partir du code parallélisé MPI, ajout de directives OpenMP)
• Séance 5
• Finalisation des TP de la première série
• Comparaison des performances
• 2ème série : outils et modèles de programmation autour de l'adressage global
• Séance 6
• Présentation des différents outils basés sur l'adressage global (Tests simples avec UPC)
• Adaptation du code séquentiel en utilisant UPC
• Tests en utilisant plusieurs architectures de communication sous-jacentes
• Séance 7
• Présentation d'une version java du code séquentiel
• Adaptation du code séquentiel en utilisant le modèle de programmation Titanium
• Contrôle d'une couche de calcul en C + interface en java : tests simples (en fonction du temps restant)
• 3ème série : plateformes objets
• Séance 8
• Utilisation de la bibliothèque C++ Intel TBB
• Modification du premier code séquentiel
• Comparaison avec le programme utilisant OpenMP
• Séance 9
• Utilisation de la programmation objet pour exprimer le parallélisme
• Exemple avec les classes A++/P++ (séquentiel/parallèle)
• Séance 10
• Utilisation de la plateforme Cactus
• Bilan sur les TP du cours 3 et questions éventuelles
  
  
  

We gratefully acknowledge the assistance of SMAI in promoting these schools.