Bb	HPC @ FSU	HPC books

High-Performance Computing

and
Scientific Computing

This course is an introductory course on high-performance computing. High-performance computing refers to a specialized use and programming of (parallel) supercomputers, computer clusters, and everything from software to hardware to speed up computations. The CPU clock speed of desktop and commodity processors has reached a maximum range, due to physical limitations. As a result, more advanced (and often creative) use of software and parallel hardware is required to further speed up processing. In this course you will learn how to write faster code that is highly optimized for modern multi-core processors and clusters, using modern software development tools, performance profilers, specialized algorithms, parallelization strategies, and advanced parallel programming constructs in OpenMP and MPI.

Topics in this course include (but are not limited to):

• Programming languages and programming-language extentions for HPC
• Compiler options and optimizations for modern single-core and multi-core processors
• Execution profiling, timing techniques, and benchmarking for modern single-core and multi-core processors
• Hardware architecture
• Parallelization strategies, task parallelism, data parallelism, and work sharing techniques
• Parallel programming with OpenMP and (Posix) threads
• Message passing with MPI
• HPC numerical libraries and auto-tuning libraries
• Advanced parallel algorithms
• Other parallel programming tools and languages (UPC, CAF, ...)

For this course we made a selection of material from the best textbooks available:

• "Software Optimization for High Performance Computing: Creating Faster Applications" by K.R. Wadleigh and I.L. Crawford, Hewlett-Packard professional books, Prentice Hall.
• "The Software Optimization Cookbook" (2nd ed.) by R. Gerber, A. Bik, K. Smith, and X. Tian, Intel Press.
• "Parallel Programming: Techniques and Applications using Networked Workstations and Parallel Computers" (2nd ed.) by B. Wilkinson and M. Allen, Prentice Hall.
• "Parallel Scientific Computing in C++ and MPI" by G. Karniadakis and R. Kirby II, Cambridge University Press.
• "Scientific Parallel Computing" by L.R. Scott, T. Clark, and B. Bagheri, Princeton University Press.
• "Sourcebook of Parallel Programming" by J. Dongara, I. Foster, G. Fox, W. Gropp, K. Kennedy, L. Torczon, and A. White (eds), Morgan Kaufmann.

Software tools you will use in this course:

• Compilers for C, C++, and Fortran: GCC, ICC (Intel compiler)
• Profiles and performance analyzers: gprof, codecov, Sun Studio Express performance analyzer and/or Intel Vtune
• MPI (OpenMPI) programming with MPE and Jumpshot-4 for performance visualization
• SGE job submission scripts
• OpenMP programming with ICC and Sun Studio Express 12 (development and performance analysis)
• Autoparallelization with ICC and Sun Studio Express 12
• Timers: RDTSC (Intel hw counter), times(), gettimeofday(), getrusage()
• Timing techniques: measuring wall clock or CPU user/system time, timer resolution and rollover, spin loops
• POSIX threads programming, tips and tricks
• Cache-conscious programming techniques: data structures and blocking techniques
• High-performance libraries: BLAS, LAPACK, FFTW

Sun Studio Performance Analyzer Demo	Sun Studio Performance Analyzer Demo	Jumpshot-4 (MPI)	Intel Codecov

(Select image to enlarge)

Synopsis

ISC5318/CIS5930-3, Fall 2012, 3 credit hours, letter grade, MW 3:35 - 4:50PM, 301LOV

Course Web Site

http://www.cs.fsu.edu/~engelen/courses/HPC

Instructor

Professor Robert van Engelen
engelen at cs dot fsu dot edu
259LOV
645-0309.

Office Hours

Tuesdays and Thursdays 1:30 PM to 2:30 PM and upon request.

Intended Audience

Graduate students.

Prerequisites

Some experience programming in Java, or C, or C++, or Fortran is required.

Material (note: only parts of each of these textbooks will be used)

[HPC] "Software Optimization for High Performance Computing: Creating Faster Applications" by K.R. Wadleigh and I.L. Crawford, Hewlett-Packard professional books, Prentice Hall.
[OPT] "The Software Optimization Cookbook" (2nd ed.) by R. Gerber, A. Bik, K. Smith, and X. Tian, Intel Press.
[PP2] "Parallel Programming: Techniques and Applications using Networked Workstations and Parallel Computers" (2nd ed.) by B. Wilkinson and M. Allen, Prentice Hall.
[PSC] "Parallel Scientific Computing in C++ and MPI" by G. Karniadakis and R. Kirby II, Cambridge University Press.
[SPC] "Scientific Parallel Computing" by L.R. Scott, T. Clark, and B. Bagheri, Princeton University Press.
[SRC] "Sourcebook of Parallel Programming" by J. Dongara, I. Foster, G. Fox, W. Gropp, K. Kennedy, L. Torczon, and A. White (eds), Morgan Kaufmann.

Other Material

Class handouts.

Teaching Assistant

-

Exams and Grading

There will be two exams (one midterm exam and one final exam), a set of laboratory projects, and homework exercises. More details on the grading policy, grade breakdown, and exam material can be found on the exam page. Exam dates are scheduled in advance. A grade of zero will be recorded for missed exams unless prior arrangements have been made.

Assignments

Homework and programming projects will be assigned on a regular basis. All assignments are mandatory and part of the final grade. The assignments, when turned in before or at the due date before midnight will be graded. When turned in late, 5% will be deducted from the homework grade per day until the homework has been received, with a maximum extension of five days.

Attendance

Roll is not taken, but you are responsible for all material delivered in class.

Cheating

All exams and assignments must be completed individually, unless stated otherwise. Copying solutions is considered cheating. Submitted source code listings will be compared. Keep a copy of the listings to provide evidence of creative work. Students are expected to uphold the Honor Code. Any student involved in cheating is in violation of the Honor code. Consult the "Student Handbook" for more details on the Honor code.

You will need an account on the FSU HPC. If you don’t have one then request access as follows:

If you are a student in the department of Scientific Computing and you don't have an account on the HPC yet, then:

• Use your favorite web browser to go to www.hpc.fsu.edu
• Select "Apply for an account" in the left menu, then click the "Apply For An Account" button
• Select "Scientific Computing" from the drop-down list
• Select "Non-Faculty Account" from the drop-down list
• Enter you FSUID and the rest of the information
• Carefully read and accept the HPC user policies and select "Request Account"
  

If you are a CS student (or other department), then:

• Use your favorite web browser to go to www.hpc.fsu.edu
• Select "Apply for an account" in the left menu, then click the "Apply For An Account" button
• Select "General Access" from the drop-down list
• Select "Sponsored Account" from the drop-down list
• Select "van Engelen" from the Sponsor drop-down list
• Enter you FSUID and the rest of the information (memorize your password)
• Carefully read and accept the HPC user policies and select "Request Account"

Submit your assignments to the instructor using the Blackboard drop box. When uploading to the dropbox, please make sure to include all source code of your programming assignment, the input and output files (when applicable), and a Makefile. Use the zip utility to compress and archive the material. Make sure to mark the Blackboard file upload with "HOMEWORK #" and "PROJECT #", where # is the homework/assignment number. Also, please include your name and the assignment number in the program source code

The assignments must be turned in before or at the due date before midnight. When turned in late, 5% will be deducted from the homework grade per day until the homework has been received, with a maximum extension of five days.

Roll is not taken for this course. Attendance is required the first week of classes and during exams.

Excused absences include documented illness, deaths in the family and other documented crises, call to active military duty or jury duty, religious holy days, and official University activities. These absences will be accommodated in a way that does not arbitrarily penalize students who have a valid excuse. Consideration will also be given to students whose dependent children experience serious illness.

The Florida State University Academic Honor Policy outlines the University’s expectations for the integrity of students’ academic work, the procedures for resolving alleged violations of those expectations, and the rights and responsibilities of students and faculty members throughout the process.  Students are responsible for reading the Academic Honor Policy and for living up to their pledge to “. . . be honest and truthful and . . . [to] strive for personal and institutional integrity at Florida State University.”  (Florida State University Academic Honor Policy, found at http://dof.fsu.edu/honorpolicy.htm.)

Students with disabilities needing academic accommodation should:
(1) register with and provide documentation to the Student Disability Resource Center; and
(2) bring a letter to the instructor indicating the need for accommodation and what type. This should be done during the first week of class.

This syllabus and other class materials are available in alternative format upon request.

For more information about services available to FSU students with disabilities, contact the:

Student Disability Resource Center
97 Woodward Avenue, South
108 Student Services Building
Florida State University
Tallahassee, FL 32306-4167
(850) 644-9566 (voice)
(850) 644-8504 (TDD)
sdrc@admin.fsu.edu
http://www.disabilitycenter.fsu.edu/

"Except for changes that substantially affect implementation of the evaluation (grading) statement, this syllabus is a guide for the course and is subject to change with advance notice.”

Copyright: Robert van Engelen, Florida State University, 2007-2012.
Last modified: Wednesday, October 3, 2012