Classroom 152DSL Bb Prism Phoenix Mendel Tempest Selected books

High-Performance Computing

Introduction

This course is an introductory course on high-performance computing. High-performance computing refers to the use of (parallel) supercomputers and computer clusters, and everything from software to hardware to speed up computations. In this course you will learn how to write faster code that is highly optimized for modern multi-core processors and clusters, using modern software development tools and performance profilers, specialized algorithms, parallelization strategies, and advanced parallel programming constructs in OpenMP and MPI.

Overview

Topics in this course include:

For this course we made a selection of material from the best textbooks available:

A list of hardware platforms you will use for projects in this course:

Prism cluster (7 nodes, 24 cores):

Phoenix cluster (16 nodes, 32 cores, see Phoenix system status):

Tempest cluster (10 nodes, 24 cores):

Software tools you will use in this course:

Sun Studio Performance Analyzer Demo Sun Studio Performance Analyzer Demo Jumpshot-4 (MPI) Intel Codecov

(Select image)

Syllabus

Synopsis
ISC5935-01/CIS5930-13, Fall 2007, 3 hrs, letter grade, TR 2:00-3:15PM, 152DSL
Course Web Site
http://www.cs.fsu.edu/~engelen/courses/HPC
Instructor
Professor Robert van Engelen
engelen at scs dot fsu dot edu
475DSL (School of Computational Science at 4th floor in Dirac)
645-0309.
When and Where?
152 DSL on Tuesday and Thursday from 2:00 PM to 3:15 PM.
Office Hours
Mondays 12:30 PM to 1:30 PM and upon request.
Intended Audience
Graduate students and senior undergraduate students.
Prerequisites
Some experience programming in Java, or C, or C++, or Fortran is required.
Material (note: these textbooks do not need to be purchased)
[HPC] "Software Optimization for High Performance Computing: Creating Faster Applications" by K.R. Wadleigh and I.L. Crawford, Hewlett-Packard professional books, Prentice Hall.
[OPT] "The Software Optimization Cookbook" (2nd ed.) by R. Gerber, A. Bik, K. Smith, and X. Tian, Intel Press.
[PP2] "Parallel Programming: Techniques and Applications using Networked Workstations and Parallel Computers" (2nd ed.) by B. Wilkinson and M. Allen, Prentice Hall.
[PSC] "Parallel Scientific Computing in C++ and MPI" by G. Karniadakis and R. Kirby II, Cambridge University Press.
[SPC] "Scientific Parallel Computing" by L.R. Scott, T. Clark, and B. Bagheri, Princeton University Press.
[SRC] "Sourcebook of Parallel Programming" by J. Dongara, I. Foster, G. Fox, W. Gropp, K. Kennedy, L. Torczon, and A. White (eds), Morgan Kaufmann.
Other Material
Class handouts.
Teaching Assistant
-
Exams and Grading
There will be two exams (one midterm exam and one final exam), a set of laboratory projects, and homework exercises. More details on the grading policy, grade breakdown, and exam material can be found on the exam page. Exam dates are scheduled in advance. A grade of zero will be recorded for missed exams unless prior arrangements have been made.
Assignments
Homework and programming projects will be assigned on a regular basis. All assignments are mandatory and part of the final grade. The assignments, when turned in before or at the due date before midnight will be graded. When turned in late, 5% will be deducted from the homework grade per day until the homework has been received, with a maximum extension of five days.
Attendance
Roll is not taken, but you are responsible for all material delivered in class.
Cheating
All exams and assignments must be completed individually, unless stated otherwise. Copying solutions is considered cheating. Submitted source code listings will be compared. Keep a copy of the listings to provide evidence of creative work. Students are expected to uphold the Honor Code. Any student involved in cheating is in violation of the Honor code. Consult the "Student Handbook" for more details on the Honor code.

Computer Accounts

You will need an account to log on to the SCS machines. If you don’t have one then please contact the instructor to request access.

You will also need an ACNS account (i.e., @garnet.fsu.edu) for receiving class emails. If you want, you can forward your garnet email to other accounts (visit https://cars.acns.fsu.edu for more details).

Schedule, Lecture Notes, and Assignments

The schedule with lecture notes and project assignments is shown below (refresh this page if necessary). You are encouraged to download and print the PDF notes as handouts to take to class. Links will be active when new notes have been added. The schedule will be updated when needed and significant changes will be announced on Blackboard.

Project and homework assignments can be downloaded by selecting the links in the "Assignments due" column.

Date Lectures Reading material Other resources Assignments due Date
8/28 Introduction [PDF] [PP2] pages 3-12
[SPC] pages 1-15, 37-45, 48-52
Grand Challenge problems
Colatz conjecture
Colatz on the Cell BE
- 8/28
8/30 [SRC] pages 8-13 Moore's law - 8/30
9/4 Architecture and Compilers

Part I: uniprocessors [PDF]
Part II: multiprocessors [PDF]
[HPC] pages 7-56, 81-124
- 9/4
9/6 [OPT] pages 51-65, 69-78, 89-91, 96-105, 107-121 [OPT] pages 143-188 - 9/6
9/11 [PP2] pages 13-26
Homework1 9/11
9/13 [SPC] pages 71-95
- 9/13
9/18 [HPC] pages 25-28 [SRC] pages 15-42 - 9/18
9/20 Performance Analysis [PDF] [HPC] pages 125-142 [OPT] pages 19-50 Homework2 9/20
9/25 [PP2] pages 64-75 [SRC] pages 443-468 - 9/25
9/27 [SPC] pages 56-64
- 9/27
10/2 Programming Models [PDF]

- 10/2
10/4

- 10/4
10/9 Programming with Shared Memory

Part I: basics [PDF]
Part II: OpenMP [PDF]
Part III: synchronous examples [PDF]
[PP2] pages 230-271 [HPC] pages 191-218 Homework3 10/9
10/11
Semaphores Project1 10/11
10/16
OpenMP - 10/16
10/18
More examples - 10/18
10/23 Midterm exam [PDF]

- 10/23
10/25 Programming with Message Passing

Part I: basics [PDF]
Part II: MPI [PDF]
Part III: message passing examples [PDF]
[PP2] pages 42-61 MPI - 10/25
10/30

- 10/30
11/1

- 11/1
11/6

- 11/6
11/8 [PP2] pages 340-365
- 11/8
11/13 Algorithms

Part I: embarrassingly parallel [PDF]
Part II: partitioning, divide & conquer [PDF]
[PP2] pages 79-99
- 11/13
11/15

- 11/15
11/20 [PP2] pages 106-131
Project2 11/20
11/22 Thanksgiving: no class

n/a 11/22
11/27 Algorithms [PSC] pages 321-337
- 11/27
11/29 High-Performance Libraries, Programming Languages, and Tools [PDF]
Part I: Libs&Tools [PDF]
Part II: Languages [PDF]


- 11/29
12/4 [SRC] pages 621-647
- 12/4
12/6

- 12/6
12/14 Final exam 3:00PM-5:00PM 152DSL

Project3 12/14

Assignment Submissions

Submit your assignments to the instructor by email. Please send only one email message that includes all sources of your programming assignment, the input and output files (when applicable), and a Makefile. Use the zip utility to compress and archive the material for inclusion as an attachment in the email message. Use the email subject line "HOMEWORK #", where # is the homework/assignment number. Also, please include your name and the assignment number in the program source code

The assignments must be turned in before or at the due date before midnight. When turned in late, 5% will be deducted from the homework grade per day until the homework has been received, with a maximum extension of five days.

Academic Honor Policy

The Florida State University Academic Honor Policy outlines the University’s expectations for the integrity of students’ academic work, the procedures for resolving alleged violations of those expectations, and the rights and responsibilities of students and faculty members throughout the process.  Students are responsible for reading the Academic Honor Policy and for living up to their pledge to “. . . be honest and truthful and . . . [to] strive for personal and institutional integrity at Florida State University.”  (Florida State University Academic Honor Policy, found at http://dof.fsu.edu/honorpolicy.htm.)

ADA

Students with disabilities needing academic accommodation should:
(1) register with and provide documentation to the Student Disability Resource Center; and
(2) bring a letter to the instructor indicating the need for accommodation and what type. This should be done during the first week of class.

This syllabus and other class materials are available in alternative format upon request.

For more information about services available to FSU students with disabilities, contact the:

Student Disability Resource Center
97 Woodward Avenue, South
108 Student Services Building
Florida State University
Tallahassee, FL 32306-4167
(850) 644-9566 (voice)
(850) 644-8504 (TDD)
sdrc@admin.fsu.edu
http://www.disabilitycenter.fsu.edu/


Copyright: Robert van Engelen, Florida State University, 2007.
Last modified: Thursday, December 6, 2007