Large-Scale Program Behavior Analysis for Adaptation and Parallelization by Xipeng Shen Submitted in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy Supervised by Professor Chen Ding Department of Computer Science The College Arts and Sciences University of Rochester Rochester, New York 2006 To my dear wife, Xiaoyan, for giving me the greatest love and unconditional support. iii Curriculum Vitae Xipeng Shen was born in Shandong, China on April 20th, 1977. He attended the North China University of Technology from 1994 to 1998 receiving a Bachelor of En- gineering in Industry Automation in 1998 and joined the Chinese Academy of Sciences after that. In 2001, he received his Master of Science in Pattern Recognition and Intel- ligent Systems. He came to the University of Rochester in the Fall of 2001 and began graduate studies in Computer Science. He pursued his research in programming sys- tems under the direction of Professor Chen Ding. He received a Master of Science degree in Computer Science from the University of Rochester in 2003. iv Acknowledgments I have been very fortunate to have an advisor, Professor Chen Ding, who has pro- vided me the independence to pursue my research interests and guided me through the course of this dissertation. From him, I learned not only the splendid attractions of science, a rigorous attitude toward research, and intelligent ways to solve problems, but also persistency in pursuing excellence, optimism when facing difficulties, and a profound understanding of life. Without his unconditional support and consistent in- spirations, this thesis would not be possible. I owe a particular debt to my committee. Sandhya Dwarkadas and Michael Scott have always promptly replied to my requests and put the greatest effort to provide me the excellent advice on my research, presentations and paper writings. Dana Ballard gave me invaluable encouragement when I was facing the difficulty of choosing my research area. Michael Huang offered insightful technical discussions and suggestions. Their supports helped to shape many aspects of this thesis. In addition to my committee, Professor Kai Shen and Mitsunori Ogihara helped me with intelligent discussions on research and generous mentoring on my career search. I also thank Dr. Kevin Stoodley, Yaoqing Gao, and Roch Archambault for precious help and collaborations in IBM Toronto Lab. All the people in the Computer Science department made my graduate life more enjoyable and helped me in various ways. In particular, I thank my collaborators, Yutao Zhong, Chengliang Zhang, Ruke Huang, Jonathan Shaw, Kirk Kelsey and Tongxin Bai for pleasant discussions. v I would express my special acknowledgment to my family. Mom and Dad gave me their greatest love, hope and belief. I would, above all, thank most my wife Xiaoyan for all she has brought me, the support, the love, and the best gift in the world—my dear son Daniel. This material is based upon work supported by the National Science Foundation (Grants nos. CNS-0509270, CCR-0238176, CCR-0219848 and EIA-0080124) and the Department of Energy (Contract No. DE-FG02-02ER25525). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the au- thor(s) and do not necessarily reflect the views of the funding organizations. vi Abstract Motivated by the relentless quest for program performance and energy savings, pro- gram execution environments (e.g. computer architecture and operating systems) are becoming reconfigurable and adaptive. But most programs are not: despite dramatic differences in inputs, machine configurations, and the workload of the underlying op- erating systems, most programs always have the same code running with the same data structure. The resulting mismatch between program and environment often leads to ex- ecution slowdown and resource under-utilization. The problem is exacerbated as chip multi-processors are becoming commonplace and most user programs are still sequen- tial, increasingly composed with library code and running with interpreters and virtual machines. The ultimate goal of my research is an intelligent programming system, which injects into a program the ability to automatically adapt and evolve its code and data and configure its running environment in order to achieve a better match between the (improved) program, its input, and the environment. Program adaptation is not possible without accurately forecasting a program’s be- havior. However, traditional modular program design and analysis are ill-fitted for finding large-scale composite patterns in increasingly complicated code, dynamically allocated data, and multi-layered execution environments (e.g. interpreters, virtual ma- chines, operating systems and computer architecture.) My research views a program as a composition of large-scale behavior patterns, each of which may span a large num- ber of loops and procedures statically and billions of instructions dynamically. I apply statistical technology to automatically recognize the patterns, build models of program vii behavior, and exploit them in offline program transformation (e.g. array regrouping based on locality and reference affinity research) and online program adaptation (e.g. behavior-oriented parallelization based on behavior phases) to improve program per- formance and reliability. viii Table of Contents Curriculum Vitae iii Acknowledgments iv Abstract vi List of Tables xiv List of Figures xvi 1 Introduction 1 1.1 ProgramBehaviorModel. 2 1.2 New Challenges to Programming Systems . 3 1.3 Behavior-BasedProgramAnalysis . 5 1.4 Dissertation Organization . 10 2 Data Locality 11 2.1 Introduction................................ 11 2.1.1 BasicPredictionMethod . 13 2.1.2 Factors Affecting Prediction Accuracy . 14 2.2 Terminology................................ 16 ix 2.3 Single-Model Multi-Input Prediction . 17 2.4 Multi-ModelPrediction. 19 2.5 Evaluation................................. 22 2.5.1 ExperimentalSetup. 22 2.5.2 ResultsonLargeInputs . 23 2.5.3 ResultsonSmallInputs . 26 2.5.4 Comparison............................ 27 2.6 Uses in Cache Performance Prediction . 29 2.7 RelatedWork ............................... 30 2.7.1 OtherResearchonReuseDistance. 30 2.7.2 Comparison with Program Analysis Techniques . 32 2.8 FutureDirections ............................. 33 2.9 Summary ................................. 34 3 Reference Affinity 36 3.1 Introduction................................ 37 3.2 Distance-based Affinity Analysis . 39 3.2.1 Distance-Based Reference Affinity Model . 39 3.2.2 k-Distance Affinity Analysis . 40 3.3 Lightweight Frequency-Based Affinity Analysis . ..... 41 3.3.1 Frequency-Based Affinity Model . 42 3.3.2 Lightweight Affinity Analysis Techniques . 43 3.3.2.1 Unit of Program Analysis . 43 3.3.2.2 Static Estimate of the Execution Frequency . 44 3.3.2.3 Profiling-Based Frequency Analysis . 46 x 3.3.2.4 Context-Sensitive Lightweight Affinity Analysis . 46 3.3.2.5 Implementation . 48 3.4 Affinity-Oriented Data Reorganization . 50 3.4.1 Structure Splitting . 50 3.4.2 ArrayRegrouping ........................ 52 3.5 Evaluation................................. 54 3.5.1 AffinityGroups.......................... 55 3.5.2 Comparison with Distance-Based Affinity Analysis . 56 3.5.3 Comparison with Lightweight Profiling . 57 3.5.4 Performance Improvement from Array Regrouping . 57 3.5.5 Choice of Affinity Threshold . 58 3.6 RelatedWork ............................... 60 3.6.1 AffinityAnalysis . 60 3.6.2 DataTransformations. 61 3.7 FutureDirections ............................. 63 3.8 Summary ................................. 64 4 Locality Phase Analysis through Wavelet Transform 65 4.1 Introduction................................ 66 4.2 HierarchicalPhaseAnalysis . 69 4.2.1 Locality Analysis Using Reuse Distance . 69 4.2.2 Off-line Phase Detection . 71 4.2.2.1 Variable-Distance Sampling . 71 4.2.2.2 Wavelet Filtering . 72 4.2.2.3 Optimal Phase Partitioning . 75 xi 4.2.3 PhaseMarkerSelection. 77 4.2.4 MarkingPhaseHierarchy . 78 4.3 Evaluation................................. 80 4.3.1 PhasePrediction ... .... .... .... .... .... .. 81 4.3.1.1 Comparison of Prediction Accuracy . 83 4.3.1.2 GccandVortex . 89 4.3.2 AdaptiveCacheResizing. 90 4.3.3 Phase-BasedMemoryRemapping . 93 4.3.4 Comparison with Manual Phase Marking . 97 4.4 RelatedWork ............................... 99 4.5 Summary .................................101 5 Behavior Phase Analysis through Active Profiling 103 5.1 Introduction................................103 5.2 Active Profiling and Phase Detection . 107 5.2.1 Terminology ...........................107 5.2.2 Constructing Regular Inputs . 108 5.2.3 SelectingPhaseMarkers . .109 5.3 Evaluation.................................112 5.3.1 GCC ...............................114 5.3.2 Perl ................................117 5.3.3 Comparison with Procedure and Interval Phase Analysis . .118 5.4 UsesofBehaviorPhases . .123 5.5 RelatedWork ...............................125 5.6 FutureDirections .............................126 5.7 Summary .................................127 xii 6 Behavior-Oriented Parallelization 129 6.1 Introduction................................129 6.2 SpeculativeCo-processing . .133 6.2.1 Possibly Parallel Regions . 133 6.2.2 Correctness ............................135 6.2.2.1 Three Types of Data Protection . 135 6.2.2.2 Correctness of Speculative Co-processing . 140 6.2.2.3 NovelFeatures. .144 6.2.3 Performance ...........................145 6.2.3.1 ParallelEnsemble . .145 6.2.3.2 Understudy Process . 147 6.2.3.3