Fall 2020
Tue, Thu
3:00-04:15 PM
RWST 306 or Web Broadcast
Wed 4.30-5.30PM, Fri 2.00-3.00PM on Zoom (Canvas)
This course is an introduction to high-performance computing. High-performance computing (HPC) is the practice of leveraging parallel computing power, from computer clusters and supercomputers, to solve complex computational problems faster. In this course students will learn the main programming models, such as openMP and MPI, to write highly optimized code for modern multi-core processors, clusters, and modern computing architecture (e.g., GPUs). Furthermore, this course will explore the concepts and challenges of performance scalability, portability, and energy efficiency of HPC applications and systems.
Lecturer: Dr. Steve Petruzza TA: Kohl Kennedy
Topics include: multithread programming, openMP, distributed programming, MPI, performance profiling, parallel and distributed algorithms, parallel I/O, in situ analytics, GPU programming.
Prerequisites: a good C/C++ programming background. Most of the assignment will be in C/C++. In particular, a student needs to have knowledge of data structures, arrays, pointers and memory allocation in C/C++. If you are not too familiar with C/C++ here is a tutorial that could be useful, the “C++ basic” section is a good place to start. If you are not familiar with the Unix terminal:
Day | Topic | References | Homework |
---|---|---|---|
Sep 1 | Introduction | IPP Ch. 1 | |
Sep 3 | Computing architecture and memory | IPP Ch. 2 | |
Sep 8 | Parallel hardware and interconnects | IPP Ch. 2 | |
Sep 10 | Parallel software and scalability | IPP Ch. 2 | HW1 |
Sep 15 | Shared memory programming: PThreads | IPP Ch. 4 | |
Sep 17 | Shared memory programming: PThreads (2) | IPP Ch. 4 | |
Sep 22 | Shared memory programming: PThreads (3) | IPP Ch. 4 | |
Sep 24 | C++11 Threads and running on an HPC system | CHPC | HW2 |
Sep 29 | OpenMP (1) | IPP Ch. 5 | |
Oct 1 | OpenMP (2) | IPP Ch. 5 | |
Oct 6 | Distributed programming: basics | IPP Ch. 3 | |
Oct 8 | Distributed programming: MPI | IPP Ch. 3 | |
Oct 13 | Mid-term exam | Mid-term | |
Oct 15 | Reviews, more MPI (2) | HW3 | |
Oct 20 | Applications (1) | ||
Oct 22 | Applications (2) | ||
Oct 27 | Projects, Parallel I/O | HW4 | |
Oct 29 | Introduction to GPU Programming and CUDA | GPU Ch.1,2,3 | |
Nov 3 | GPU: execution model | GPU Ch.4 | |
Nov 5 | GPU: memory model | GPU Ch.5 | |
Nov 10 | GPU: tiling and matrix multiplication | GPU Ch.6,7 | HW5 |
Nov 12 | GPU: convolution | GPU Ch.8 | |
Nov 17 | GPU: convolution (2) | GPU Ch.9 | |
Nov 19 | GPU: convolution analysis | GPU Ch.9 | |
Nov 24 | GPU: deep learning | ||
Nov 26 | Thanksgiving | ||
Dec 1 | Python for HPC | ||
Dec 3 | Python for HPC (2) | ||
Dec 8 | Parallel programming languages | ||
Dec 10 | Review and Project presentations | ||
Dec 15 | No class. | ||
Dec 17 | Final exam | Final |
The schedule above is subject to change
Selection/proposal deadline: Nov 15, 2020
Submission deadline: Dec 6, 2020
A [93%, 100%] A- [87%, 93%) B+ [81%, 87%) B [75%, 81%) B- [70%, 75%) C+ [65%, 70%) C [60%, 65%) C- [58%, 60%) D+ [55%, 58%) D [53%, 55%) F Below 53%
If a student does not attend a class during the first week of the term or by the second class meeting, whichever comes first, the instructor may submit a request to have the student dropped from the course. (This does not remove responsibility from the student to drop courses which they do not plan to attend.) Students who are dropped from courses will be notified by the Registrar’s Office through their preferred e-mail account. Students may drop courses without notation on the permanent record through the first 20 percent of the class. If a student drops a course following the first 20 percent of the class, a W will be permanently affixed to the student’s record (check General Catalog for exact dates). Students with extenuating circumstances should refer to the policy regarding Complete Withdrawal from the University and the Incomplete (I) Grade policy in the General Catalog.
In order to continue to provide various forms of face-to-face instruction at USU, and to limit the spread of COVID-19 during the pandemic, students are asked to follow certain classroom protocols during the fall 2020 semester. These protocols are based on CDC, state, and local health department guidelines and requirements are in place not only for your safety but also the safety of the entire campus community.
Academic freedom is the right to teach, study, discuss, investigate, discover, create, and publish freely. Academic freedom protects the rights of faculty members in teaching and of students in learning. Freedom in research is fundamental to the advancement of truth. Faculty members are entitled to full freedom in teaching, research, and creative activities, subject to the limitations imposed by professional responsibility. USU Policy 403 further defines academic freedom and professional responsibilities.
The University expects that students and faculty alike maintain the highest standards of academic honesty. The Code of Policies and Procedures for Students at Utah State University (Student Conduct) addresses academic integrity and honesty and notes the following:
Students have a responsibility to promote academic integrity at the University by not participating in or facilitating others’ participation in any act of academic dishonesty and by reporting all violations or suspected violations of the Academic Integrity Standard to their instructors.
To enhance the learning environment at Utah State University and to develop student academic integrity, each student agrees to the following Honor Pledge: “I pledge, on my honor, to conduct myself with the foremost level of academic integrity”. Violations of the Academic Integrity Standard (academic violations) include, but are not limited to cheating, falsification, and plagiarism.
Plagiarism includes knowingly “representing by paraphrase or direct quotation, the published or unpublished work of another person as one’s own in any academic exercise or activity without full and clear acknowledgment. It also includes the unacknowledged use of materials prepared by another person or agency engaged in the selling of term papers or other academic materials.” The penalties for plagiarism are severe. They include warning or reprimand, grade adjustment, probation, suspension, expulsion, withholding of transcripts, denial or revocation of degrees, and referral to psychological counseling.
This course adheres to the cheating policy for courses in the Department of Computer Science.
USU welcomes students with disabilities. If you have, or suspect you may have, a physical, mental health, or learning disability that may require accommodations in this course, please contact the Disability Resource Center (DRC) as early in the semester as possible (University Inn # 101, 435797‐2444, drc@usu.edu). All disability related accommodations must be approved by the DRC. Once approved, the DRC will coordinate with faculty to provide accommodations.
Utah State University is committed to creating and maintaining an environment free from acts of sexual misconduct and discrimination and to fostering respect and dignity for all members of the USU community. Title IX and USU Policy 339 address sexual harassment in the workplace and academic setting.
The university responds promptly upon learning of any form of possible discrimination or sexual misconduct. Any individual may contact USU’s Affirmative Action/Equal Opportunity (AA/EO) Office for available options and resources or clarification. The university has established a complaint procedure to handle all types of discrimination complaints, including sexual harassment (USU Policy 305), and has designated the AA/EO Director/Title IX Coordinator as the official responsible for receiving and investigating complaints of sexual harassment.
Students who feel they have been unfairly treated [in matters other than discipline, admission, residency, employment, traffic, and parking - which are addressed by procedures separate and independent from the Student Code] may file a grievance through the channels and procedures described in the Student Code: Article VII Grievances.
All classes, programs, and extracurricular activities within the University involve some risk, and some involve travel. The University provides opportunities to participate in these programs on a voluntary basis. Therefore, students should not participate in them if they do not care to assume the risks. Students can ask the respective program leaders/sponsors about the possible risks a program may generate, and if students are not willing to assume the risks, they should not select that program. By voluntarily participating in classes, programs, and extracurricular activities, students do so at their own risk. General information about University Risk Management policies, insurance coverage, vehicle use policies, and risk management forms can be found at http://www.usu.edu/riskmgt/.
Mental health is critically important for the success of USU students. As a student, you may experience a range of issues that can cause barriers to learning, such as strained relationships, increased anxiety, alcohol/drug problems, feeling down, difficulty concentrating and/or lack of motivation. These mental health concerns or stressful events may lead to diminished academic performance or reduce your ability to participate in daily activities. Utah State University provides free services for students to assist them with addressing these and other concerns. You can learn more about the broad range of confidential mental health services available on campus at Counseling and Psychological Services (CAPS).
Students are also encouraged to download the “SafeUT App” to their smartphones. The SafeUT application is a 24/7 statewide crisis text and tip service that provides real-time crisis intervention to students through texting and a confidential tip program that can help anyone with emotional crises, bullying, relationship problems, mental health, or suicide related issues.