Lecture 19

Learning objectives

After this class, you should be able to:

1. Given data for the following operations: (i) reduction, (ii) broadcast, (iii) gather, and (iv) scatter, show the messages sent at each step and the state of each processor at the end of each step, for the algorithms discussed in class for these operation.
2. Derive the time complexity for each of the above operations.
3. Explain the idea behind pipelining, which is used to develop efficient parallel algorithms.
4. Show the steps involved in the iterative parallel prefix algorithm discussed in class, and derive its time complexity.
5. Show how a linear recurrence can be formulated as a parallel prefix problem, and derive the time complexity of the parallel algorithm based on this formulation.

Reading assignment

1. Handout on Parallel algorithms.

Exercises and review questions

• Questions on current lecture's material
  1. Given eight processors with data 1, 3, 2, 4, 5, 3, 1, 8 respectively, show the messages sent at each step of the all-gather operation, and the data present in each processor at the end of each step.
  2. Give an efficient algorithm for the scatter operation, and derive its time complexity.
  3. Show a pipelined scheme for performing n > > P broadcasts on P processors, with the time complexity mentioned in the Parallel algorithms handout.
  4. If a linear recurrence had three terms instead of two, then show how we can formulate it as a parallel prefix problem, and derive the time complexity.

• Questions on next lecture's material
  1. (Post your answer on the discussion board) Show any 3 x 3 matrix, any vector with three components, and the result of multiplying the matrix by the vector.