® Adding Parallelism with Intel Parallel Studio: No Parallelism Experience Required Mark W. Davis Developer Products Division April 22, 2010 Software & Services Group Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 2 Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 3 Overview – adding parallelism • What is parallelism? – Overlapping portions of a serial algorithm, so it runs faster • Eg: Search both sides of a tree at the same time • Start with a serial algorithm • Modify it to be a parallel-ready serial algorithm • Add code to – Perform statements in parallel – Synchronize the sharing of common resources Analysis, preparation and testing of the serial code is key to simplifying adding parallelism Software & Services Group 4 Overview – the tools • Decide where to add the parallelism – Analyze the serial program – Prepare it for parallelism – Test the preparations • Add the parallelism – Cilk, Intel® Threading Building Blocks, OpenMP*, o/s threads • Find logic problems – Only fails sometimes – Place of failure changes • Find performance problems Adding and using parallelism requires enhanced tools Software & Services Group 5 Overview – Intel Parallel Advisor original serial C/C++ program Software & Services Group 6 Overview – Intel Parallel Advisor Run Survey tool to measure original serial where your application is C/C++ Advisor Survey spending time program tool Software & Services Group 7 Overview – Intel Parallel Advisor Run Survey tool to measure original serial where your application is C/C++ Advisor Survey spending time program tool Add annotations to describe how annotated you might divide the work of the serial hot-spot into parallel tasks C/C++ program Software & Services Group 8 Overview – Intel Parallel Advisor The annotated program still has serial semantics. Working on it is annotated just normal serial programming. C/C++ program Software & Services Group 9 Overview – Intel Parallel Advisor Use the performance modeling (suitability) and race detection (Correctness) tools to identify potential The annotated issues program still has serial semantics. Advisor Working on it is annotated Suitability tool just normal serial programming. C/C++ program Advisor Correctness tool Software & Services Group 10 Overview – Intel Parallel Advisor Use the performance modeling (suitability) and race detection (Correctness) tools to identify potential The annotated issues program still has serial semantics. Advisor Working on it is annotated Suitability tool just normal serial programming. C/C++ program Advisor Correctness tool Adjust the annotations or the program code itself to resolve performance and correctness issues Software & Services Group 11 Overview – Intel Parallel Advisor The final version of the annotated program is projected to have good annotated serial performance and no C/C++ races. program Software & Services Group 12 Overview – Intel Parallel Advisor annotated serial C/C++ program Now, it’s time to change the annotations into real parallel code – TBB or Cilk or your favorite model. This straightforward translation is done by the programmer. parallel C/C++ program Software & Services Group 13 Overview – Intel Parallel Advisor Use the other tools in Intel Parallel Studio to work with your parallel application (compile, debug, tune, etc) other parallel C/C++ Intel Parallel Studio program tools Software & Services Group 14 Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 15 Finding where to add parallelism • Find steps which are using lots of time • Measure the number of steps needed to make a good size task • Find how the steps interact Analyze, don’t guess. Computers are good at analysis. Software & Services Group 16 Tachyon Ray-tracer: Serial Software & Services Group 17 Finding where to add parallelism • Find steps which are using lots of time – Profile the serial algorithm : production build+ data – Find outer calls/loops that take lots of time, and that the architect thinks could be done in parallel Software & Services Group 18 Survey Report – Profile: Hot Call Chain(s) Software & Services Group 19 Finding where to add parallelism • Annotate the chosen sites and tasks – Sites contain where the tasks start, won’t exit until the contained tasks end – Tasks are modeled as if they run in parallel Macro use expands to a call to a stub function that can be hooked Software & Services Group 20 Finding where to add parallelism • Measure the site and task times – Profile the annotated algorithm : release program + data – Annotations in bad places may slow execution • How the measurement is done – Runtime binary instrumentation (Pin) • Intercept the calls to the stub functions – Measure the times between the calls • Uses the QPC() function and _rdtsc instruction – Compress and write to disk • Same as the other Intel Parallel Studio tools – Analyze • To understand the counts and times of the site, task, and lock execution Software & Services Group 21 Suitability Report – Model Parallel Performance Software & Services Group 22 Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 23 Finding code preventing parallelism • Intel® Parallel Advisor runs the annotated serial program – It is serial, so violations of data dependencies don’t crash the run • Binary instrumentation (Pin) watches memory traffic and annotations – Use small data sets • Slow-down can be substantial • Analyzes the data to determine what could go wrong when the tasks are done in parallel – Use debug builds, so problems can be mapped to sources • Only memory sharing issues and locking problems are found Software & Services Group 24 Correctness Report – Model Parallel Behavior Software & Services Group 25 Source for Data Communication Error (Race) Software & Services Group 26 Resolving roadblocks to parallelism • Change the serial code to remove these – Debug these changes like you do any other change • Learn about and use existing parallelized libraries; or thread-safe libraries • Use per-task resources rather than shared resources – local variables, own files, other resources • Use reductions – Supported directly by Cilk, Intel® TBB, OpenMP™ • Use locks only if none of the above are suitable Fix the issues in the serial code Software & Services Group 27 Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 28 Adding parallelism • Choose your parallel framework – Will depend on your programming language, portability needs, future growth needs • Cilk, Intel® Threading Building Blocks (Intel® TBB), OpenMP*, o/s threads • Intel® Parallel Composer supports these • Modify your build and deployment environments – Compiler switches – Libraries may need to be installed on target machines • Insert parallel framework statements for each annotation, implementing its “parallel” semantics Choose the best framework for you Software & Services Group 29 Tachyon – Parallel (Intel TBB), 4 cores Appears divided into quarters Thread stealing to finish Software & Services Group 30 Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 31 Correctness and Performance • Parallel programs are sensitive to timing issues – A failure may only happen only a few times in a million runs of the program, or only on some hardware – Intel® Parallel Advisor correctness analysis depends on the annotations matching what you implemented – Intel® Parallel Inspector detects problems that may happen, even if they did not happen this time • Parallel program timings are sensitive to task sizes, thread interactions, memory caches – Intel® Parallel Amplifier measures where the time is spent Measure, don’t guess Software & Services Group 32 Agenda • Overview • Find where parallelism can be usefully added to a program • Fix the parts of the program that prevent parallelism • Add parallelism • Test the revised program for correctness and performance • Don’t panic Software & Services Group 33 Don’t Panic • Parallel programming has been around for a long time, and is perceived as very difficult • But any programmer can do it and get worthwhile benefits IF – They go step by step – They use modern frameworks and tools – They avoid the more complex techniques • Intel® Parallel Advisor documentation introduces the programmer to this world Intel® Parallel Studio helps with each step Software & Services Group 34 Summary • Analysis, preparation and testing of the serial code simplifies adding parallelism • Adding and using parallelism