Introduction to Scheduling Arnaud Legrand, CNRS, University of Grenoble LIG laboratory, [email protected] November 16, 2015 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling 1 / 89 Outline 1 Modeling Applications, General Notions Introducing Fundamental Notions Through the Matrix Product Example Adaptive Parallel Programs Task Graphs and Parallel Tasks From Outer Space 2 Defining a Scheduling Problem Rules of the Game Criteria: How Do You Win the Game? Analysis Method Graham Notation 3 Batch Scheduling Principles Theoretical results Basic idea: FCFS + Backfilling EASY How Good is the Schedule? 4 Gang Scheduling as an Alternative Principles A. Legrand (CNRS-LIG)Drawbacks INRIA-MESCAL Introduction to Scheduling 2 / 89 Batch Scheduling it is then Batch Scheduling and Grids? Outline 1 Modeling Applications, General Notions Introducing Fundamental Notions Through the Matrix Product Example Adaptive Parallel Programs Task Graphs and Parallel Tasks From Outer Space 2 Defining a Scheduling Problem Rules of the Game Criteria: How Do You Win the Game? Analysis Method Graham Notation 3 Batch Scheduling Principles Theoretical results Basic idea: FCFS + Backfilling EASY How Good is the Schedule? 4 Gang Scheduling as an Alternative Principles A. Legrand (CNRS-LIG)Drawbacks INRIA-MESCAL Introduction to Scheduling Applications 3 / 89 Batch Scheduling it is then Batch Scheduling and Grids? Outline 1 Modeling Applications, General Notions Introducing Fundamental Notions Through the Matrix Product Example Adaptive Parallel Programs Task Graphs and Parallel Tasks From Outer Space 2 Defining a Scheduling Problem Rules of the Game Criteria: How Do You Win the Game? Analysis Method Graham Notation 3 Batch Scheduling Principles Theoretical results Basic idea: FCFS + Backfilling EASY How Good is the Schedule? 4 Gang Scheduling as an Alternative Principles A. Legrand (CNRS-LIG)Drawbacks INRIA-MESCAL Introduction to Scheduling Applications 4 / 89 Batch Scheduling it is then Batch Scheduling and Grids? Matrix Product: Sequential Version 1 f To compute C C + A × B g; 2 for i = 1 to n do 3 for j = 1 to n do 4 for k = 1 to n do 5 Ci;j Ci;j + Ai;k × Bk;j B1;1 B1;2 B2;1 B2;2 A1;1 A1;2 C1;1 C1;2 A2;1 A2;2 C2;1 C2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 5 / 89 1 Load C1;1, A1;1, B1;1; 3 Unload A1;1, B1;1. Load A1;2, B2;1; 5 Unload C1;1, A1;2, B2;1. Load C1;2, A1;1, B1;2; 7 Unload A1;1, B1;2; Matrix Product: Sequential Version 2 C1;1 C1;1 + A1;1 × B1;1; 4 C1;1 C1;1 + A1;2 × B2;1; 6 C1;2 C1;2 + A1;1 × B1;2; B1;1 B1;2 8 C1;2 C1;2 + A1;2 × B2;2; 9 ... B2;1 B2;2 A A C C 2648CPU 1;1 1;2 1;1 1;2 A2;1 A2;2 C2;1 C2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 5 / 89 Matrix Product: Sequential Version 1 Load C1;1, A1;1, B1;1; 2 C1;1 C1;1 + A1;1 × B1;1; 3 Unload A1;1, B1;1. Load A1;2, B2;1; 4 C1;1 C1;1 + A1;2 × B2;1; 5 Unload C1;1, A1;2, B2;1. Load C1;2, A1;1, B1;2; 6 C1;2 C1;2 + A1;1 × B1;2; 7 Unload A1;1, B1;2; B1;1 B1;2 8 C1;2 C1;2 + A1;2 × B2;2; 9 ... B2;1 B2;2 A A C C CPU 2 46 8 1;1 1;2 1;1 1;2 I/O 15 3 7 A2;1 A2;2 C2;1 C2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 5 / 89 Matrix Product: Sequential Version 1 Load C1;1, A1;1, B1;1; 2 C1;1 C1;1 + A1;1 × B1;1; 3 Unload A1;1, B1;1. Load A1;2, B2;1; 4 C1;1 C1;1 + A1;2 × B2;1; 5 Unload C1;1, A1;2, B2;1. Load C1;2, A1;1, B1;2; 6 C1;2 C1;2 + A1;1 × B1;2; 7 Unload A1;1, B1;2; B1;1 B1;2 8 C1;2 C1;2 + A1;2 × B2;2; 9 ... B2;1 B2;2 A A C C SequentialCPU Programs2 46 8 1;1 1;2 1;1 1;2 SequentialI/O 15 programs3 are generally7 a succession of CPU burst and A A C C I/O burst. 2;1 2;2 2;1 2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 5 / 89 Matrix Product: Parallel Version (1/2) Setting P1;1 P1;2 Server P2;1 P2;2 I A, B, and C are initially located on the server. I We will distribute A, B, and C on P1;1, P1;2, P2;1, P2;2. I We will make use of all four processors to compute C C + A × B. I Such a parallel program could be written using for example MPI. We want a SPMD algorithm. A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 6 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B2;1 1;2 B1;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;2 C1;1 A1;1 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B1;1 2;2 B2;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;1 C2;1 A2;2 C2;2 7 Unload Ci;j to the server; A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 7 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B1;1 1;2 B2;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;1 C1;1 A1;2 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B2;1 2;2 B1;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;2 C2;1 A2;1 C2;2 7 Unload Ci;j to the server; A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 7 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B2;1 1;2 B1;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;2 C1;1 A1;1 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B1;1 2;2 B2;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;1 C2;1 A2;2 C2;2 7 Unload Ci;j to the server; Server P1;1 P1;2 P2;1 P2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 7 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B2;1 1;2 B1;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;2 C1;1 A1;1 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B1;1 2;2 B2;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;1 C2;1 A2;2 C2;2 7 Unload Ci;j to the server; Server P1;1 P1;2 P2;1 P2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 7 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B2;1 1;2 B1;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;2 C1;1 A1;1 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B1;1 2;2 B2;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;1 C2;1 A2;2 C2;2 7 Unload Ci;j to the server; Server P1;1 P1;2 P2;1 P2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 7 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B1;1 1;2 B2;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;1 C1;1 A1;2 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B2;1 2;2 B1;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;2 C2;1 A2;1 C2;2 7 Unload Ci;j to the server; Server P1;1 P1;2 P2;1 P2;2 A. Legrand (CNRS-LIG) INRIA-MESCAL Introduction to Scheduling Applications 7 / 89 Matrix Product: Parallel Version (2/2) Algorithm 1 f Pi;j is responsible for computing Ci;j. g; P P 1;1 B1;1 1;2 B2;2 2 Load Ci;j, Ai;(i+j)%2, B(i+j)%2;j from the server; A1;1 C1;1 A1;2 C1;2 3 Clocal Clocal + Alocal × Blocal; 4 Exchange A with horizontal neighbor; local P P 2;1 B2;1 2;2 B1;2 5 Exchange Blocal with vertical neighbor; 6 Clocal Clocal + Alocal × Blocal; A2;2 C2;1 A2;1 C2;2 7 Unload Ci;j to the server; Server P1;1 P1;2 P2;1 P2;2 A.