NVM Programming Model (NPM) Version 1.2 Abstract: This SNIA document defines recommended behavior for software supporting Non- Volatile Memory (NVM). This document has been released and approved by the SNIA. The SNIA believes that the ideas, methodologies and technologies described in this document accurately represent the SNIA goals and are appropriate for widespread distribution. Suggestion for revision should be directed to http://www.snia.org/feedback/. SNIA Technical Position June 19, 2017 USAGE The SNIA hereby grants permission for individuals to use this document for personal use only, and for corporations and other business entities to use this document for internal use only (including internal copying, distribution, and display) provided that: 1. Any text, diagram, chart, table or definition reproduced shall be reproduced in its entirety with no alteration, and, 2. Any document, printed or electronic, in which material from this document (or any portion hereof) is reproduced shall acknowledge the SNIA copyright on that material, and shall credit the SNIA for granting permission for its reuse. Other than as explicitly provided above, you may not make any commercial use of this document, sell any or this entire document, or distribute this document to third parties. All rights not explicitly granted are expressly reserved to SNIA. Permission to use this document for purposes other than those enumerated above may be requested by e-mailing [email protected]. Please include the identity of the requesting individual and/or company and a brief description of the purpose, nature, and scope of the requested use. All code fragments, scripts, data tables, and sample code in this SNIA document are made available under the following license: BSD 3-Clause Software License Copyright (c) 2017, The Storage Networking Industry Association. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. * Neither the name of The Storage Networking Industry Association (SNIA) nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE NVM Programming Model (NPM) SNIA Technical Position 2 Version 1.12 DISCLAIMER The information contained in this publication is subject to change without notice. The SNIA makes no warranty of any kind with regard to this specification, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The SNIA shall not be liable for errors contained herein or for incidental or consequential damages in connection with the furnishing, performance, or use of this specification. Suggestions for revisions should be directed to http://www.snia.org/feedback/. Copyright © 2017 SNIA. All rights reserved. All other trademarks or registered trademarks are the property of their respective owners. NVM Programming Model (NPM) SNIA Technical Position 3 Version 1.12 Revision History Changes since version 1: • The former informative Consistency annex is reworded and moved to two places in the specification body: o New section 6.10 Aligned operations on fundamental data types o New section 10.1.1 Applications and PM Consistency in NVM.PM.FILE • A number of editorial fixes to make spelling, terminology, and spacing more consistent Changes from version 1.1 to version 1.2: • The former informative PM error handling annex is elaborated, improved and moved to the following places in the specification body: o New section 10.1.2 PM Error Handling. o New action 10.2.10 NVM.PM.FILE.CHECK_ERROR o New action 10.2.11 NVM.PM.FILE.CLEAR_ERROR o New attribute 10.3.9 NVM.PM.FILE.ERROR_EVENT_MINIMAL_CAPABILITY o New attribute 10.3.10 NVM.PM.FILE.ERROR_EVENT_PRECISE_CAPABILITY o New attribute 10.3.11 NVM.PM.FILE.ERROR_EVENT_ERROR_UNIT_CAPABILITY o New attribute 10.3.12 NVM.PM.FILE.ERROR_EVENT_MAPPED_SUPPORT_CAPABILITY o New attribute 10.3.12 NVM.PM.FILE.ERROR_EVENT_LIVE_SUPPORT_CAPABILITY • The wording in section 10.2.7 NVM.PM.FILE.OPTIMIZED_FLUSH_AND_VERIFY is corrected to make clear that the action does not require verification of the data in the persistence domain, but merely requires reporting of any errors diagnosed during the process of writing the data to the persistence domain. • New section 10.2.8 NVM.PM.FILE.OPTIMIZED_FLUSH_ALLOWED introduces a new action that indicates on a per-file basis whether the application may invoke the OPTIMIZED_FLUSH action or instead is required to call fsync or msync (or the Windows analogs). o Some DAX-capable file systems may require that the application call msync or the Windows equivalent and do not permit the application to call OPTIMIZED_FLUSH in its place, for some subset of the files in the file system. o This situation arises when the file system requires the msync system call in order to force updated file data or metadata to the persistence domain. For example, when a new page is allocated to a sparse file due to a page fault, some DAX filesystems do not eagerly force the allocation metadata to the persistence domain, but instead require an fsync or msync call to guarantee that the metadata is persistent. Similarly, if the filesystem is performing compression or encryption, it will require an fsync or msync to persist the data. • New section 10.2.9 NVM.PM.FILE.DEEP_FLUSH introduces a new action that provides improved reliability when persisting data but at a potentially higher latency cost. The intent of this new action is to enable DAX file systems and applications to limit the loss of data when normal persistence fails. NVM Programming Model (NPM) SNIA Technical Position 4 Version 1.12 o ADR persistence is only probabilistic; thermal conditions or other unforeseen conditions may increase the time needed to flush data in the power-protected domain to the persistent media beyond the hold-up time of the power supply. In such an event, we would like to limit the scope of the damage to less than all the data on the persistent memory devices. o For example, if there are multiple file systems on the persistent memory devices, and some of them are not mounted when ADR fails, then the data in those file systems is not corrupted and can be preserved across the persistence failure. Similarly if a file is not open when ADR fails, and all of the data and file system metadata needed to access the file has been persisted, then the file is not corrupted and can be preserved across the persistence failure. The DEEP_FLUSH action provides the tool needed by file systems to force data and metadata to a more reliable persistence domain, so that upon recovery the file system can detect whether it had been mounted, and, if mounted, whether the it's metadata is intact. Applications can then use DEEP_FLUSH to preserve data that upon recovery after a persistence failure would allow the application to determine whether the file had been open, and thus potentially corrupted, or closed, and thus can be preserved. o Attribute 10.3.8 NVM.PM.FILE.DEEP_FLUSH_CAPABLE enables an application to determine if the DEEP_FLUSH action is supported. • A number of editorial fixes to make spelling, terminology, and spacing more consistent NVM Programming Model (NPM) SNIA Technical Position 5 Version 1.12 Table of Contents FOREWORD ........................................................................................................................... 10 1 SCOPE .............................................................................................................................. 11 2 REFERENCES .................................................................................................................. 12 3 DEFINITIONS, ABBREVIATIONS, AND CONVENTIONS ............................................... 13 3.1 DEFINITIONS .................................................................................................................. 13 3.2 KEYWORDS .................................................................................................................... 14 3.3 ABBREVIATIONS ............................................................................................................. 14 3.4 CONVENTIONS ............................................................................................................... 15 4 OVERVIEW OF THE NVM PROGRAMMING MODEL (INFORMATIVE) ......................... 16 4.1 HOW TO READ AND USE THIS SPECIFICATION ....................................................................