Course details

Parallel System Architecture and Programming

ARC Acad. year 2006/2007 Summer semester 5 credits

Current academic year

The course covers architecture and programming of parallel systems with functional- and data-parallelism. First the parallel system theory and program parallelization are discussed. The description of the most proliferated multi-core and multi-processor symmetrical multiprocessors (SMP) follows and their programming in OpenMP environment is dealt with. The course goes on  with the treatment of interconnection networks, a basic structure for popular networks of workstations and other message-passing systems. Their programming in standardized interfaces MPI and PVM is illustrated with case studies of parallel applications. In conclusion the advanced DSM NUMA systems are described. 

Guarantor

Dvořák Václav, prof. Ing., DrSc.

Language of instruction

Czech

Completion

Examination

Time span

  • 39 hrs lectures
  • 13 hrs pc labs
  • 13 hrs projects

Department

Department of Computer Systems (UPSY)

Subject specific learning outcomes and competences

Overview of principles of parallel system design and of interconnection networks, communication techniques and algorithms. Survey of parallelization techniques of fundamental scientific problems, knowledge of parallel programming in MPI and OpenMP.

Knowledge of capabilities and limitations of parallel processing, ability to estimate performance of parallel applications. Language means for process/thread communication and synchronization. Competence in hardware-software platforms for high-performance computing and simulations.

Learning objectives

To orientate oneself in parallel systems on the market, be able to assess communication and computing possibilities of a particular architecture and to predict the performance of parallel applications. To get acquainted with the most important parallel programming tools (MPI, OpenMP ), to learn their practical use and solving problems in parallel.

Prerequisite knowledge and skills

Von-Neumann computer architecture, computer memory hierarchy, cache memories and their organization, programming in assembly and in C/C++

Study literature

Fundamental literature

  • Hennessy, J.L., Patterson, D.A.: Computer Architecture - A Quantitative Approach. 5. vydání, Morgan Kaufman Publishers, Inc., 2012, 912 s., ISBN: 9780123747501 
  • Pacecho, P.: Introduction to Parallel Programming. Morgan Kaufman Publishers, 2011, 392 s., ISBN: 9780123742605 


Syllabus of lectures

  • Function- and data-parallelism, performance measures, overhead, speedup-limiting laws.
  • Bus-based shared memory multiprocessors. Bus saturation, memory organization.
  • Cache coherence, MSI and MESI protocols, event synchronization in multiprocessors, locks and barriers.
  • Programming with threads, shared memory communication, loop parallelization, OpenMP. Program paradigms: pool, D&C, BSP.
  • A case study of a parallel application (graph SSP search).
  • Interconnection and switching networks. Models of communication, routing techniques, routing algorithms.
  • Message-passing programming: MPI, PVM, computing-communication overlap.
  • Program paradigms: pipeline, farm. Prototyping parallel programs, performance prediction.
  • Cluster computing with group communication, examples (parallel neural networks).
  • Distributed (shared) memory architectures and message passing, communication support in hardware.
  • Data parallelism, vector processors and instructions.
  • Data-parallel programming, matrix and vector operations, application examples, HPF.
  • SIMD machines and SIMD-like processing. Systolic structures.

Progress assessment

Study evaluation is based on marks obtained for specified items. Minimimum number of marks to pass is 50.

To complete successfuly session work and be able to write examination, one has to get at leat 20 points out of maximum 40.

Controlled instruction

Three small projects in duration of 5, 4 a 4 hours ; midterm examination.

Back to top