SQL Server In-Memory OLTP Internals for SQL Server 2016 Technical White Paper Writer: Kalen Delaney Technical Reviewers: Sunil Agarwal and Jos de Bruijn Published: June 2016 Applies to: SQL Server 2016 RTM Summary: In-memory OLTP, frequently referred to by its codename “Hekaton”, was introduced in SQL Server 2014. This powerful technology allows you to take advantage of large amounts of memory and many dozens of cores to increase performance for OLTP operations by up to 30 to 40 times! SQL Server 2016 is continuing the investment in In-memory OLTP by removing many of the limitations found in SQL Server 2014, and enhancing internal processing algorithms so that In-memory OLTP can provide even greater improvements. This paper describes the implementation of SQL Server 2016’s In-memory OLTP technology as of SQL Server 2016 RTM. Using In-memory OLTP, tables can be declared as ‘memory optimized’ to enable In-Memory OLTP’s capabilities. Memory-optimized tables are fully transactional and can be accessed using Transact-SQL. Transact-SQL stored procedures, triggers and scalar UDFs can be compiled to machine code for further performance improvements on memory-optimized tables. The engine is designed for high concurrency with no blocking. Copyright The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication. This white paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, AS TO THE INFORMATION IN THIS DOCUMENT. Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be reproduced, stored in, or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or otherwise), or for any purpose, without the express written permission of Microsoft Corporation. Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as expressly provided in any written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks, copyrights, or other intellectual property. © 2016 Microsoft Corporation. All rights reserved. Microsoft, Microsoft Azure, SQL Server, Visual Studio, Windows, and Windows PowerShell are trademarks of the Microsoft group of companies. All other trademarks are property of their respective owners. 2 1. Contents 1. Contents ................................................................................................................... 3 Introduction ..................................................................................................................... 6 Design Considerations and Purpose ............................................................................... 6 Terminology .................................................................................................................... 7 What’s Special About In-Memory OLTP? ........................................................................ 7 Memory-optimized tables .......................................................................................................................... 8 Indexes on memory-optimized tables ....................................................................................................... 9 Concurrency improvements ...................................................................................................................... 9 Natively Compiled Modules..................................................................................................................... 10 Performance ............................................................................................................................................ 10 Using In-Memory OLTP ................................................................................................. 10 Creating Databases ................................................................................................................................ 11 Azure SQL Database .............................................................................................................................. 12 Additional Database Requirements and Recommendations .................................................................. 12 Creating Tables ....................................................................................................................................... 13 Altering Tables and Indexes .................................................................................................................................................. 15 Row and Index Storage ................................................................................................. 17 Memory Consumers ................................................................................................................................ 17 Rows ....................................................................................................................................................... 19 Row header ................................................................................................................................................................................. 19 Payload area .............................................................................................................................................................................. 20 Indexes On Memory-Optimized Tables .................................................................................................. 20 Hash Indexes ............................................................................................................................................................................... 21 Range Indexes ........................................................................................................................................................................... 23 Columnstore Indexes ............................................................................................................................................................... 31 Index Metadata ..........................................................................................................................................................................37 Data Operations ...................................................................................................................................... 38 Isolation Levels Allowed with Memory-Optimized Tables ......................................................................................... 39 Deleting ....................................................................................................................................................................................... 40 Updating and Inserting .......................................................................................................................................................... 40 Reading ......................................................................................................................................................................................... 41 T-SQL Support........................................................................................................................................................................... 42 Garbage Collection of Rows in Memory .......................................................................................................................... 43 Transaction Processing ................................................................................................. 45 3 Isolation Levels ....................................................................................................................................... 46 Validation and Post-processing .............................................................................................................. 50 Validation of Foreign Keys .................................................................................................................................................... 52 Commit Dependencies ........................................................................................................................................................... 52 Commit processing .................................................................................................................................................................. 53 Post-processing ........................................................................................................................................................................ 53 Concurrency ............................................................................................................................................ 54 Locks ............................................................................................................................................................................................. 54 Latches ........................................................................................................................................................................................