Bb FSU Research Computing Center HPC books

High-Performance Computing

and
Scientific Computing

Introduction

This course is an introductory course on high-performance computing. High-performance computing refers to a specialized use and programming of (parallel) supercomputers, computer clusters, and everything from software to hardware to speed up computations. The CPU clock speed of desktop and commodity processors has reached a maximum range, due to physical limitations. As a result, more advanced (and often creative) use of software and parallel hardware is required to further speed up processing. In this course you will learn how to write faster code that is highly optimized for modern multi-core processors and clusters, using modern software development tools, performance profilers, specialized algorithms, parallelization strategies, and advanced parallel programming constructs in OpenMP and MPI.

Overview

Topics in this course include (but are not limited to):

For this course we made a selection of material from the best textbooks available:

Software tools you will use in this course:

Sun Studio Performance Analyzer Demo Sun Studio Performance Analyzer Demo Jumpshot-4 (MPI) Intel Codecov

(Select image to enlarge)

Syllabus

Course Web Site
http://www.cs.fsu.edu/~engelen/courses/HPC
Instructor
Professor Robert van Engelen
engelen at cs dot fsu dot edu
168LOV
Office Hours
Tuesdays and Thursdays 1:30 PM to 2:30 PM and upon request
Location
301 LOV on Tuesdays and Thursdays from 9:30AM to 10:45AM
Prerequisites
COP4020 (or equivalent) and some experience programming in Java, C, C++, or Fortran
Material (note: only parts of each of these textbooks will be used)
[HPC] "Software Optimization for High Performance Computing: Creating Faster Applications" by K.R. Wadleigh and I.L. Crawford, Hewlett-Packard professional books, Prentice Hall.
[OPT] "The Software Optimization Cookbook" (2nd ed.) by R. Gerber, A. Bik, K. Smith, and X. Tian, Intel Press.
[PP2] "Parallel Programming: Techniques and Applications using Networked Workstations and Parallel Computers" (2nd ed.) by B. Wilkinson and M. Allen, Prentice Hall.
[PSC] "Parallel Scientific Computing in C++ and MPI" by G. Karniadakis and R. Kirby II, Cambridge University Press.
[SPC] "Scientific Parallel Computing" by L.R. Scott, T. Clark, and B. Bagheri, Princeton University Press.
[SRC] "Sourcebook of Parallel Programming" by J. Dongara, I. Foster, G. Fox, W. Gropp, K. Kennedy, L. Torczon, and A. White (eds), Morgan Kaufmann.
Other Material
Class handouts
Teaching Assistant
TBD
Exams and Grading
There will be two exams (one midterm exam and one final exam), laboratory projects, and homework exercises. Exam dates are scheduled in advance. A grade of zero will be recorded for missed exams unless prior arrangements have been made.
Assignments
Homework and programming projects will be assigned on a regular basis. All assignments are mandatory and part of the final grade. The assignments, when turned in before or at the due date before midnight will be graded. When turned in late, 5% will be deducted from the homework grade per day until the homework has been received, with a maximum extension of five days.
Honor Code Violations
All exams and assignments must be completed individually, unless stated otherwise. Copying solutions is considered cheating. Submitted source code listings will be compared. Keep a copy of the listings to provide evidence of creative work. Students are expected to uphold the Honor Code, see also below. Any student involved in cheating is in violation of the Honor code. Consult the "Student Handbook" for more details on the Honor code.

Computer Accounts

You will need an account to access FSU RCC. If you don’t have an account then request a sponsored account as follows:

 

Schedule, Lecture Notes, and Assignments

The tentative schedule with PDF lecture notes is shown below (refresh this page to view the lates changes). You are encouraged to download and print the PDF notes and take them as handouts to class. The schedule will be updated when needed and significant changes will be announced on Blackboard.

You can view assignments by selecting the links under the "Assignments due" column.

Date Lectures Reading material Other resources Assignments due Date
1/10 Introduction [PP2] pages 3-12
[SPC] pages 1-15, 37-45, 48-52, 110-112
Grand Challenge problems
Colatz conjecture
- 1/10
1/12 [SRC] pages 3-13 Moore's law - 1/12
1/17 Uniprocessors [HPC] pages 7-56 [OPT] pages 51-141 - 1/17
1/19     - 1/19
1/24 Compiler optimizations for uniprocessors [HPC] pages 81-124 GCC optimizations - 1/24
1/26     - 1/26
1/31 Multiprocessors [PP2] pages 13-26   HW1 1/31
2/2 [SPC] pages 71-95   - 2/2
2/7 Parallel programming models     - 2/7
2/9 Performance analysis [PP2] pages 62-75
[HPC] pages 125-142
[SRC] pages 443-468 - 2/9
2/14 [SPC] pages 56-65 - HW2 2/14
2/16 Programming with Shared Memory:
Part I: basics
Part II: OpenMP
Part III: synchronous parallel
[HPC] pages 191-218 [HPC] pages 219-240 - 2/16
2/21 [PP2] pages 230-271 Semaphores - 2/21
2/23
OpenMP
OpenMP best practices
- 2/23
2/28   More OpenMP examples - 2/28
3/2 Review

- 3/2
3/7 Midterm exam     - 3/7
3/9 Programming with Message Passing:
Part I: basics
Part II: MPI
Part III: message passing parallel
[PP2] pages 42-61 MPI PR1 3/9
3/21 [PP2] pages 340-365   - 3/21
3/23     - 3/23
3/28 Algorithms:
Part I: embarrassingly parallel
Part II: partitioning, divide & conquer
[PP2] pages 79-99   - 3/28
3/30 [PP2] pages 201-210   HW3 3/30
4/4     - 4/4
4/6 [PP2] pages 106-131   - 4/6
4/11 [PSC] pages 309-337   PR2 4/11
4/13 Parallel Programming Languages   UPC - 4/13
4/18     HW4 4/18
4/20 OpenCL and CUDA programming   CUDA Part A and Part B - 4/20
4/25 Libs & Tools [SRC] pages 621-647
BLAS and LAPACK - 4/25
4/27   PETSc - 4/27
5/2 Final exam 10:00AM-12:00PM   TAU profiler PR3 (due 5/5) 5/2

 

Grade Breakdown


Test Percentage Example
Midterm exam 25% 2012 midterm PDF
Final exam 25% 2012 final PDF
Projects 40% n/a
Homework 10% n/a

 

Letter Grades


94-100% A 87-89% B+ 77-79% C+ 67-69% D+ 0-59% F
90-93% A- 83-86% B 73-76% C 63-66% D

80-82% B- 70-72% C- 60-62% D-

Assignment Submissions

All assignments are mandatory and part of the final grade. Assignments and homework should be turned in before or at the due date before midnight. When turned in late, 5% will be deducted from the project grade per day until the submission has been received, with a maximum extension of five days.

Submit your assignments via Blackboard. Submitted projects should includes all source code, the input and output files (when applicable), and a Makefile. Include your name and the assignment number in the program source code file(s).

Attendance

First class attendance is mandatory. Attendance during exams is also mandatory. You are responsible for obtaining all material delivered in class to study.

Excused absences include documented illness, deaths in the family and other documented crises, call to active military duty or jury duty, religious holy days, and official University activities. These absences will be accommodated in a way that does not arbitrarily penalize students who have a valid excuse. Consideration will also be given to students whose dependent children experience serious illness.

Academic Honor Policy

The Florida State University Academic Honor Policy outlines the University’s expectations for the integrity of students’ academic work, the procedures for resolving alleged violations of those expectations, and the rights and responsibilities of students and faculty members throughout the process.  Students are responsible for reading the Academic Honor Policy and for living up to their pledge to “. . . be honest and truthful and . . . [to] strive for personal and institutional integrity at Florida State University.”  (Florida State University Academic Honor Policy, found at http://fda.fsu.edu/Academic-Resources/Academic-Honor-Policy.)

ADA

Students with disabilities needing academic accommodation should:
(1) register with and provide documentation to the Student Disability Resource Center; and
(2) bring a letter to the instructor indicating the need for accommodation and what type. This should be done during the first week of class.

This syllabus and other class materials are available in alternative format upon request.

For more information about services available to FSU students with disabilities, contact the:

Student Disability Resource Center
97 Woodward Avenue, South
108 Student Services Building
Florida State University
Tallahassee, FL 32306-4167
(850) 644-9566 (voice)
(850) 644-8504 (TDD)
sdrc@admin.fsu.edu
http://dos.fsu.edu/sdrc

Syllabus Change Policy

"Except for changes that substantially affect the evaluation (grading), this syllabus is a guide for the course and is subject to change with advance notice.”


Copyright: Robert van Engelen, Florida State University, 2017.
Last modified: December 29, 2016