CIS5930 Parallel and Distributed Systems

Syllabus

Lecture 1

• Syllabus, Introduction
• Reading:
  • From a few cores to Many: A Tera-scale Computing Research Overview , An Intel white paper about tera-scale (parallel) computing, 2006
  • Software and the Concurrency Revolution, by Herb Sutter and James Larus (Microsoft), ACM Queue Special Issue on Multiprocessor 3(7) Sept. 2005.
• Homework: Write a 300-word essay titled "Key hardware and software features in my workstation five years from now.". Due: next Monday before the class. Hints: You can probably borrow some ideas from the two papers.

Lecture 2

• Basics of Parallelism
• Reading: Culler's book, Chapter 1

Lecture 3

• Basics of Parallelism (2)

• Parallel machine architecture overview

• Shared Memory Architectures 1: Cache Coherence Protocols,
• Reading:
  • Culler's book chapters 6 and 8.

• Shared Memory Architectures 2: Memory Consistency Models,
• Reading:
  • Shared Memory Consistency Models: A Tutorial.

• Simultaneous Multithreading and Chip Multiprocessor,
• Reading:
  • Simultaneous Multithreading: Maximizing on-Chip Parallelism.,
  • The case for a Single-Chip Multiprocessor

• The JAVA Memory Model, Matthew Small's slides
• Reading:
  • The JAVA Memory Model, by Jeremy Manson, William Pugh, and Sarita V Adve, POPL'05

• The Cell architecture
• Reading:
  • Introduction to the Cell Multiprocessor, J.A. Kahle, et. al, IBM Journal of Research and Development, 49(4/5):589-604, Sept. 2005
  • The Cell Project at IBM Research

Potential term project topics, Please for a group and decide a topic (can be your own topic) by next Wednesday.

Lecture 10

• Scalable Multiprocessors
• Reading: Culler's book, Chapter 8

• Interconnects
• Reading: Culler's book, Chapter 9

• Software Overhead in Message Layers: Where Does the Time Go

• InfiniBand Architecture

• IBM Blue Gene/L Architecture
• Reading: Overview of the Blue Gene/L system architecture, by A. Gara, et. al., IBM Journal of Research and Development, 49(2/3):195-212, March/May 2005.

• Layered routing in irregular networks (presented by Wickus Nienaber).
• Reading: Lysne, O.; Skeie, T.; Reinemo, S.-A.; Theiss, I., "Layered routing in irregular networks," Parallel and Distributed Systems, IEEE Transactions on , vol.17, no.1, pp. 51-65, Jan. 2006 (you can get it from IEEE xplorer).

• Introduction to parallel programming models and pthreads

• Pthreads (2)

• Due March 24: "The Landscape of Parallel Computing Research: A View from Berkeley," Technical Report NO. UCB/EECS-2006-183.
• Due March 31: "Thread Clustering: Sharing Aware Scheduling on SMP-CMP-SMT Multiprocessors," ACM Eurosys'2007.
• Due April 7: "Comparing Memory Systems for Chip Multiprocessors." ACM ISCA'2007.
• Due April 14: "Rotary Router: An Efficient Architecture for CMP Interconnection Networks." ACM ISCA 2007.
• Due April 21: "Oblivious Routing for Fat-Tree Based System Area Networks with Uncertain Traffic Demands." ACM Sigmetrics, 2007.

• OpenMP

• MPI

• Loop optimizations, parallelizing compilers, and dependence analysis

• Compiling for distributed memory machines
• Reading: "The Rise and Fall of High Performance Fortran: An Historical Object Lesson", by K Kennedy, C. Koelbel, and H. Zima, ACM SIGPLAN on History of Programming Languages, 2007.

• Automatic Empiricial techniques: ATLAS
• Reading: Automated Empirical Optimization of Software and the ATLAS Project by R. Clint Whaley, A. Petitet, and J. Dongarra, Parallel Computing, 27(1-2)3-35, 2001.

• Using Automatic Empiricial techniques in MPI implementations