Intel® Technology Journal | Volume 14, Issue 3, 2010 Publisher Managing Editor Content Architect Richard Bowles Andrew Binstock Muntaquim Chowdhury Program Manager Technical Editor Technical Illustrators Stuart Douglas Marian Lacey InfoPros Technical and Strategic Reviewers Per Hammarlund Dave L. Hill Ronak Singhal Muntaquim Chowdhury Intel® Technology Journal | 1 Intel® Technology Journal | Volume 14, Issue 3, 2010 Intel Technology Journal Copyright © 2011 Intel Corporation. All rights reserved. ISBN 978-1-934053-33-1, ISSN 1535-864X Intel Technology Journal Volume 14, Issue 3 No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744. Requests to the Publisher for permission should be addressed to the Publisher, Intel Press, Intel Corporation, 2111 NE 25th Avenue, JF3-330, Hillsboro, OR 97124-5961. E-mail: [email protected]. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in professional services. If professional advice or other expert assistance is required, the services of a competent professional person should be sought. Intel Corporation may have patents or pending patent applications, trademarks, copyrights, or other intellectual property rights that relate to the presented subject matter. The furnishing of documents and other materials and information does not provide any license, express or implied, by estoppel or otherwise, to any such patents, trademarks, copyrights, or other intellectual property rights. Intel may make changes to specifications, product descriptions, and plans at any time, without notice. Third-party vendors, devices, and/or software are listed by Intel as a convenience to Intel’s general customer base, but Intel does not make any representations or warranties whatsoever regarding quality, reliability, functionality, or compatibility of these devices. This list and/or these devices may be subject to change without notice. Fictitious names of companies, products, people, characters, and/or data mentioned herein are not intended to represent any real individual, company, product, or event. Intel products are not intended for use in medical, life saving, life sustaining, critical control or safety systems, or in nuclear facility applications. Intel, the Intel logo, Celeron, Intel Centrino, Intel Core Duo, Intel NetBurst, Intel Xeon, Itanium, Pentium, Pentium D, MMX, and VTune are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. †Other names and brands may be claimed as the property of others. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more complete information about performance and benchmark results, visit HYPERLINK “http://www.intel.com/benchmarks”www.intel.com/ benchmarks This book is printed on acid-free paper. Publisher: Richard Bowles Managing Editor: Andrew Binstock Library of Congress Cataloging in Publication Data: Printed in United States of America 10 9 8 7 6 5 4 3 2 1 First printing: July, 2011 2 | Foreword Intel® Technology Journal | Volume 14, Issue 3, 2010 INTEL® TECHNOLOGY JOURNAL THE TICk TOCk Beat OF MICROPROCESSOR DEVELOPMENT AT INTEL Articles Foreword ................................................................................................................................................................5 Preface ...................................................................................................................................................................7 The Next-generation Intel® Core™ Microarchitecture ...........................................................................................8 The Uncore: A Modular Approach to Feeding the High-performance Cores ....................................................... 30 Energy-efficient Computing: Power Management System on the Nehalem Family of Processors ..................... 50 The Feeding of High-performance Processor Cores—QuickPath Interconnects and the New I/O Hubs ........... 66 Architected for Performance—Virtualization Support on Nehalem and Westmere Processors .......................... 84 Circuit and Process Innovations to Enable High-performance, and Power and Area Efficiency on the Nehalem and Westmere Family of Intel Processors ............................................................................... 104 The Road to Production—Debugging and Testing the Nehalem Family of Processors ..................................... 128 The Toolbox for High-performance CPU Design ................................................................................................148 Table of Contents | 3 Intel® Technology Journal | Volume 14, Issue 3, 2010 4 | Foreword Intel® Technology Journal | Volume 14, Issue 3, 2010 FOREWORD by Reynold D’Sa General Manager Converged Core Development Organization Intel Corporation It is a singular honor to introduce this special edition of the Intel Technology Journal (ITJ) dedicated to the recent spate of products designed and developed by the Converged Core Development Organization (CCDO) at Intel. In the last few years, CCDO, in conjunction with our partners in the Architecture and other silicon-development organizations, has delivered a wide array of products that have been well received by our customers and the technical press: these comprise our flagship products. From a distance, it would seem that these products are delivered effortlessly very much in cadence with the tick-tock drumbeat of the Intel® product roadmap. However, behind each product stands teams of engineers that have spent years working on the product from inception to tape-in. I would like to talk about how we conceive, execute, and deliver our products and briefly explain the role of the many and varied technologists that are involved with every step of our design process. Each of our products goes through three distinct phases of the product development cycle: product definition, execution, and post-Si debug. During product definition, we evaluate a highly-complex matrix of technology trends and innovations to determine what our partners and customers will need from us and what they will want four years down the road. At this initial stage, our technologists (architects, process engineers, and design engineers) work together to create a portfolio of technical innovations. These innovations span the spectrum of technical possibilities, from new architecture and circuit technologies to break-through platform capabilities. In parallel, we collaborate with our business-group partners and their planning teams to understand our customers’ needs and the development trajectory of their products. The product that we plan to build stems from the confluence of these two streams—Intel innovation and customer requests. This is our recipe for defining leadership products that are constantly pushing the technological boundaries of our industry. The execution phase requires a monomaniacal focus on CPU design and tape- out. The entire design team, along with our partners in Process Technology and Architecture, goes into full gear to transform the abstract product specification into the fabric of interconnected transistors—often billions in numbers—that actually implement the product. In the course of execution, we go through various levels of abstractions, starting with a register transfer level (RTL) model that is eventually transformed into the layout, which in turn defines the actual silicon implementation. In addition to “building” the product, we also validate the correctness of the implementation across many dimensions ranging from Foreword | 5 Intel® Technology Journal | Volume 14, Issue 3, 2010 logical correctness, adherence of the design to stringent process requirements, fidelity of the end-product to projected performance, and so on. This large and complex array of activities has to be executed flawlessly. The execution phase culminates in the tape-in of the actual die. This step also starts the final phase of our product development cycle, which includes various post-silicon activities. The articles in this issue of the ITJ do not cover this phase of the development cycle. Another challenging aspect of our design cycle is that at any given time, we have multiple products under development. In the last two years, we have had five major products in concurrent development. We have achieved this break- through level of execution efficiency and productivity as a result of the “one- team” execution model that was first introduced by CCDO. In this execution