Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality

Total Page:16

File Type:pdf, Size:1020Kb

Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality by Patrick Judd A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto © Copyright by Patrick Judd 2014 Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality Patrick Judd Masters of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto 2014 Abstract Row buffer locality is a consequence of programs ’ inherent spatial locality that the memory system can easily exploit for significant performance gains and power savings. However, as the number of cores on a chip increases, request streams become interleaved more frequently and row buffer locality is lost. Prefetching can help mitigate this effect, but more spatial locality remains to be recovered. In this thesis we propose Prefetch Bundling, a scheme which tags spatially correlated prefetches with information to allow the memory controller to prevent prefetches from becoming interleaved. We evaluate this scheme with a simple scheduling policy and show that it improves the row hit rate by 11%. Unfortunately, the simplicity of the scheduling policy does not translate these row hits to improved performance or power savings. However, future work may build on this framework and develop more balanced policies that leverage the row locality of Prefetch Bundling. ii Acknowledgments There are many people to thank as I reflect back on my Master’s degree. First and foremost I must thank my supervisor, Andreas Moshovos, for his support, confidence in my abilities, and most of all for his ability to keep me grounded during stressful and trying times. I would also like to thank the members of the AENAO research group for their help, support and genuine camaraderie. Special thanks go to Jason Zebchuk for his help and guidance in learning and wrestling with Flexus. I would also like to thank Myrto Papadopoulou for her technical and moral support, and for tolerating my uncanny ability to kill cluster machines. I would also like to thank Andreas Moshovos, the University of Toronto and the Natural Sciences and Engineering Research Council of Canada for their generous financial support. iii Table of Contents 1 Introduction.......................................................................................................................................1 1.1 Contributions.............................................................................................................................4 1.2 Thesis Structure ........................................................................................................................5 2 Background .......................................................................................................................................6 2.1 Spatial Memory Streaming.......................................................................................................6 2.1.1 Overview .......................................................................................................................6 2.1.2 Hardware Design ..........................................................................................................7 2.2 Dynamic Random Access Memory .........................................................................................9 2.2.1 Memory Geometry .....................................................................................................10 2.2.2 DRAM Commands.....................................................................................................12 2.2.3 Row Buffer Management Policies.............................................................................12 2.2.4 Memory Scheduling ...................................................................................................13 2.2.5 Address Mapping........................................................................................................14 3 Motivation.......................................................................................................................................15 3.1 Trace Based Simulation..........................................................................................................15 3.2 Timing Simulation ..................................................................................................................17 4 Design..............................................................................................................................................19 4.1 Single Packet Bundling vs. Tagged Bundling ......................................................................20 4.1.1 Single Packet Bundle .................................................................................................20 4.1.2 Tagged Bundle............................................................................................................21 4.2 Prefetch Bundling Design ......................................................................................................24 4.3 Prefetch Bundling Implementation ........................................................................................25 4.3.1 Prefetch Controller .....................................................................................................25 4.3.2 Caches and Interconnect ............................................................................................26 iv 4.3.3 Bundle Queue .............................................................................................................27 4.4 Bundling Parameters...............................................................................................................29 4.5 Scheduling Policy ...................................................................................................................29 4.6 Tagging Optimization.............................................................................................................30 4.7 Deadlock Avoidance...............................................................................................................31 5 Methodology ...................................................................................................................................33 5.1 System Overview ....................................................................................................................33 5.2 Workloads ...............................................................................................................................36 5.3 Trace Based Simulation..........................................................................................................37 5.4 Timing Simulation ..................................................................................................................38 5.5 Area and Power Models .........................................................................................................38 6 Evaluation .......................................................................................................................................39 6.1 Workload Characterization ....................................................................................................39 6.1.1 Bundle Size Distribution ............................................................................................39 6.1.2 Prefetch Fill Levels ....................................................................................................41 6.1.3 Memory request breakdown ......................................................................................43 6.1.4 Memory Traffic ..........................................................................................................44 6.2 Prefetch Bundling ...................................................................................................................45 6.2.1 Row Hit Rate ..............................................................................................................45 6.2.2 Memory Latency.........................................................................................................48 6.2.3 Memory Bandwidth....................................................................................................51 6.2.4 Memory Power Consumption....................................................................................52 6.2.5 System Performance...................................................................................................54 6.2.6 SMS Performance.......................................................................................................56 7 Related Work ..................................................................................................................................57 7.1 Micro-pages.............................................................................................................................57 v 7.2 Adaptive Stream Detection ....................................................................................................58 7.3 Scheduled Region Prefetching ...............................................................................................58 7.4 Prefetch-Aware DRAM Controllers ......................................................................................59 7.5 Footprint Cache.......................................................................................................................59 7.6 Complexity
Recommended publications
  • PC Hardware Contents
    PC Hardware Contents 1 Computer hardware 1 1.1 Von Neumann architecture ...................................... 1 1.2 Sales .................................................. 1 1.3 Different systems ........................................... 2 1.3.1 Personal computer ...................................... 2 1.3.2 Mainframe computer ..................................... 3 1.3.3 Departmental computing ................................... 4 1.3.4 Supercomputer ........................................ 4 1.4 See also ................................................ 4 1.5 References ............................................... 4 1.6 External links ............................................. 4 2 Central processing unit 5 2.1 History ................................................. 5 2.1.1 Transistor and integrated circuit CPUs ............................ 6 2.1.2 Microprocessors ....................................... 7 2.2 Operation ............................................... 8 2.2.1 Fetch ............................................. 8 2.2.2 Decode ............................................ 8 2.2.3 Execute ............................................ 9 2.3 Design and implementation ...................................... 9 2.3.1 Control unit .......................................... 9 2.3.2 Arithmetic logic unit ..................................... 9 2.3.3 Integer range ......................................... 10 2.3.4 Clock rate ........................................... 10 2.3.5 Parallelism .........................................
    [Show full text]
  • Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality
    Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality by Patrick Judd A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto © Copyright by Patrick Judd 2014 Bundling Spatially Correlated Prefetches for Improved Main Memory Row Buffer Locality Patrick Judd Masters of Applied Science Graduate Department of Electrical and Computer Engineering University of Toronto 2014 Abstract Row buffer locality is a consequence of programs ’ inherent spatial locality that the memory system can easily exploit for significant performance gains and power savings. However, as the number of cores on a chip increases, request streams become interleaved more frequently and row buffer locality is lost. Prefetching can help mitigate this effect, but more spatial locality remains to be recovered. In this thesis we propose Prefetch Bundling, a scheme which tags spatially correlated prefetches with information to allow the memory controller to prevent prefetches from becoming interleaved. We evaluate this scheme with a simple scheduling policy and show that it improves the row hit rate by 11%. Unfortunately, the simplicity of the scheduling policy does not translate these row hits to improved performance or power savings. However, future work may build on this framework and develop more balanced policies that leverage the row locality of Prefetch Bundling. ii Acknowledgments There are many people to thank as I reflect back on my Master’s degree. First and foremost I must thank my supervisor, Andreas Moshovos, for his support, confidence in my abilities, and most of all for his ability to keep me grounded during stressful and trying times.
    [Show full text]
  • Are We Susceptible to Rowhammer? an End-To-End Methodology for Cloud Providers
    Are We Susceptible to Rowhammer? An End-to-End Methodology for Cloud Providers Lucian Cojocar, Jeremie Kimxy, Minesh Patelx, Lillian Tsaiz, Stefan Saroiu, Alec Wolman, and Onur Mutluxy Microsoft Research, xETH Zurich,¨ yCMU, zMIT Abstract—Cloud providers are concerned that Rowhammer poses vendors claim that their memory is safe against Rowhammer a potentially critical threat to their servers, yet today they lack a attacks; these claims are delivered to cloud providers with each systematic way to test whether the DRAM used in their servers is new DRAM feature: DDR4 [77], ECC-equipped DRAM [33], vulnerable to Rowhammer attacks. This paper presents an end- to-end methodology to determine if cloud servers are susceptible [31], and TRR-equipped DRAM [81], [33]. There is a large to these attacks. With our methodology, a cloud provider can gap between a proof-of-concept exploit carried out in a re- construct worst-case testing conditions for DRAM. search lab and an actual attack in the wild. In fact, no evidence We apply our methodology to three classes of servers from a indicates that Rowhammer attacks have been carried out in major cloud provider. Our findings show that none of the CPU practice. In the absence of attacks in the wild, one could easily instruction sequences used in prior work to mount Rowhammer attacks create worst-case DRAM testing conditions. To address dismiss Rowhammer as a credible threat. this limitation, we develop an instruction sequence that lever- This confusion is further fed by the lack of a system- ages microarchitectural side-effects to “hammer” DRAM at a atic methodology to test for Rowhammer.
    [Show full text]
  • Defeating Software Mitigations Against Rowhammer Tatar, Andrei; Giuffrida, Cristiano; Bos, Herbert; Razavi, Kaveh
    VU Research Portal Defeating software mitigations against rowhammer Tatar, Andrei; Giuffrida, Cristiano; Bos, Herbert; Razavi, Kaveh published in Research in Attacks, Intrusions, and Defenses 2018 DOI (link to publisher) 10.1007/978-3-030-00470-5_3 document version Publisher's PDF, also known as Version of record document license Article 25fa Dutch Copyright Act Link to publication in VU Research Portal citation for published version (APA) Tatar, A., Giuffrida, C., Bos, H., & Razavi, K. (2018). Defeating software mitigations against rowhammer: A surgical precision hammer. In M. Bailey, S. Ioannidis, M. Stamatogiannakis, & T. Holz (Eds.), Research in Attacks, Intrusions, and Defenses: 21st International Symposium, RAID 2018, Proceedings (pp. 47-66). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11050). Springer/Verlag. https://doi.org/10.1007/978-3-030-00470-5_3 General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
    [Show full text]
  • Defeating Software Mitigations Against Rowhammer Tatar, Andrei; Giuffrida, Cristiano; Bos, Herbert; Razavi, Kaveh
    VU Research Portal Defeating software mitigations against rowhammer Tatar, Andrei; Giuffrida, Cristiano; Bos, Herbert; Razavi, Kaveh published in Research in Attacks, Intrusions, and Defenses 2018 DOI (link to publisher) 10.1007/978-3-030-00470-5_3 document version Publisher's PDF, also known as Version of record document license Article 25fa Dutch Copyright Act Link to publication in VU Research Portal citation for published version (APA) Tatar, A., Giuffrida, C., Bos, H., & Razavi, K. (2018). Defeating software mitigations against rowhammer: A surgical precision hammer. In M. Bailey, S. Ioannidis, M. Stamatogiannakis, & T. Holz (Eds.), Research in Attacks, Intrusions, and Defenses: 21st International Symposium, RAID 2018, Proceedings (pp. 47-66). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11050). Springer/Verlag. https://doi.org/10.1007/978-3-030-00470-5_3 General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policy If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.
    [Show full text]
  • On the Viability of Memory Forensics in Compromised Environments
    On the Viability of Memory Forensics in Compromised Environments Zur Praktikabilit¨atvon Hauptspeicherforensik in kompromittierten Umgebungen Der Technischen Fakult¨atder Friedrich-Alexander-Universit¨at Erlangen-N¨urnberg zur Erlangung des Grades DOKTOR-INGENIEUR vorgelegt von Johannes Stuettgen aus Herdecke Als Dissertation genehmigt von der Technischen Fakult¨atder Friedrich-Alexander-Universit¨at Erlangen-N¨urnberg Tag der m¨undlichen Pr¨ufung: 28.05.2015 Vorsitzende des Promotionsorgans: Prof. Dr.-Ing. habil. Marion Merklein Gutachter: Prof. Dr.-Ing. Felix Freiling Prof. Dr. Michael Meier Abstract Memory forensics has become a powerful tool for the detection and analysis of ma- licious software. It provides investigators with an impartial view of a system, expos- ing hidden processes, threads, and network connections, by acquiring and analyzing physical memory. Because malicious software must be at least partially resident in memory in order to execute, it cannot remove all its traces from RAM. However, the memory acquisition process is vulnerable to subversion in compromised envi- ronments. Malicious software can employ anti-forensic techniques to intercept the acquisition and filter memory contents while they are copied. In this thesis, we analyze 12 popular memory acquisition tools for Windows, Linux, and Mac OS X, and study their implementation in regard to how they enumerate and map memory. We find that all of the analyzed programs use the operating system to perform these tasks, and further illustrate this by implementing an open source memory acquisition framework for Mac OS X. In a survey of kernel rootkit techniques, that prevent or filter physical memory access, we show that all 12 tested programs are vulnerable to anti-forensics, because they rely on the operating system for critical functions.
    [Show full text]
  • (12) United States Patent (10) Patent No.: US 8,171,377 B2 Dell Et Al
    US008171377B2 (12) United States Patent (10) Patent No.: US 8,171,377 B2 Dell et al. (45) Date of Patent: May 1, 2012 (54) SYSTEM TO IMPROVE MEMORY 6.732,289 B1 5/2004 Talagala et al. ................... T14?6 RELIABILITY AND ASSOCATED METHODS 6.738,947 B1* 5/2004 Maeda ......... T14f785 6.851,086 B2 2/2005 Szymanski ........ T14f781 7,017,099 B2 3/2006 Micheloni et al. ............ 714/752 (75) Inventors: Timothy J. Dell, Colchester, VT (US); 7,047.478 B2 5/2006 Gregorietal. ... 714,773 Luis A. Lastras-Montano, Cortlandt 7,100,096 B2 8/2006 Webb, Jr. et al. ............... 714.52 Manor, NY (US); Barry M. Trager, 7,117,421 B1 10/2006 Danilak .. T14f763 Yorktown Heights, NY (US); Shmuel 7.356,753 B2 * 4/2008 Chen ............................. 714/755 Winograd, Scarsdale, NY (US) (Continued) (73) Assignee: International Business Machines FOREIGN PATENT DOCUMENTS Corporation, Armonk, NY (US) JP 100911394 4f1998 WO WO2005062478 A1 7/2005 (*) Notice: Subject to any disclaimer, the term of this patent is extended or adjusted under 35 OTHER PUBLICATIONS U.S.C. 154(b) by 1127 days. IEEE (Signal Processing Systems Design and Implementation, SIPS 06), Wei Liu et al., “Low-Power High-Throughput BCH Error Cor (21) Appl. No.: 12/023,374 rection VLSI Design for Multi-Level Cell NAND Flash Memories”, pp. 303-308, Oct. 2006. (22) Filed: Jan. 31, 2008 (Continued) (65) Prior Publication Data Primary Examiner — Fritz Alphonse US 2010/0287.445A1 Nov. 11, 2010 (74) Attorney, Agent, or Firm — Ido Tuchman (51) Int. Cl. GI IC 29/00 (2006.01) (57) ABSTRACT (52) U.S.
    [Show full text]
  • Oracle® Spatial and Graph Map Visualization Developer's Guide
    Oracle® Spatial and Graph Map Visualization Developer's Guide 18c E87774-01 February 2018 Oracle Spatial and Graph Map Visualization Developer's Guide, 18c E87774-01 Copyright © 2001, 2018, Oracle and/or its affiliates. All rights reserved. Primary Author: Chuck Murray Contributors: Joao Paiva, L.J. Qian, Jayant Sharma, Honglei Zhu This software and related documentation are provided under a license agreement containing restrictions on use and disclosure and are protected by intellectual property laws. Except as expressly permitted in your license agreement or allowed by law, you may not use, copy, reproduce, translate, broadcast, modify, license, transmit, distribute, exhibit, perform, publish, or display any part, in any form, or by any means. Reverse engineering, disassembly, or decompilation of this software, unless required by law for interoperability, is prohibited. The information contained herein is subject to change without notice and is not warranted to be error-free. If you find any errors, please report them to us in writing. If this is software or related documentation that is delivered to the U.S. Government or anyone licensing it on behalf of the U.S. Government, then the following notice is applicable: U.S. GOVERNMENT END USERS: Oracle programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, delivered to U.S. Government end users are "commercial computer software" pursuant to the applicable Federal Acquisition Regulation and agency- specific supplemental regulations. As such, use, duplication, disclosure, modification, and adaptation of the programs, including any operating system, integrated software, any programs installed on the hardware, and/or documentation, shall be subject to license terms and license restrictions applicable to the programs.
    [Show full text]
  • Exploiting the DRAM Rowhammer Bug to Gain Kernel Privileges
    Exploiting the DRAM rowhammer bug to gain kernel privileges Mark Seaborn, sandbox builder and breaker with contributions by Thomas Dullien, reverse engineer 9th March 2015 This article was originally published on the Google Project Zero blog. Contents Overview Introduction to the rowhammer problem Refining the selection of addresses to hammer Using the physical address mapping Random address selection Selecting addresses using timing Double­sided hammering Exploiting rowhammer bit flips NaCl sandbox escape Kernel privilege escalation Break it down: exploit steps Exploiting write access to page tables Routes for causing row hammering Experimental results Testing your own machine Mitigations Targeted refreshes of adjacent rows BIOS updates and increasing refresh rates Monitoring for row hammering using perf counters On disclosures On exploitability of bugs On vulnerability of machines Conclusion Credits Overview “Rowhammer” is a problem with some recent DRAM devices in which repeatedly accessing a row of memory can cause bit flips in adjacent rows. We tested a selection of laptops and found that a subset of them exhibited the problem. We built two working privilege escalation exploits that use this effect. One exploit uses rowhammer­induced bit flips to gain kernel privileges on x86­64 Linux when run as an unprivileged userland process. When run on a machine vulnerable to the rowhammer problem, the process was able to induce bit flips in page table entries (PTEs). It was able to use this to gain write access to its own page table, and hence gain read­write access to all of physical memory. We don’t know for sure how many machines are vulnerable to this attack, or how many existing vulnerable machines are fixable.
    [Show full text]
  • ABSTRACT Exploiting Unused Storage Resources to Enhance
    ABSTRACT Exploiting Unused Storage Resources to Enhance Systems’ Energy Efficiency, Performance, and Fault-Tolerance by Hai Huang Chair: Kang G. Shin The invention of better fabrication materials and processes in solid-state devices has led to unprecedented technological breakthroughs in computer hardware. Today’s system software, however, often cannot take full advantage of the hardware’s rapidly improving capabilities, thus resulting in idling resources, e.g., unoccupied memory and disk space. To make hardware operate more efficiently and to reduce the amount of idle resources, this thesis proposes several techniques that can harness such resources to the benefit of users. Although there are many different types of hardware resources, this thesis focuses on the reclamation of idle resources in the storage hierarchy. First, we implemented a pure software technique to reduce the power dissipation of main memory. By aggregating un- mapped and unused memory pages and powering down unused memory ranks, a signifi- cant amount of energy can be saved with little or no performance degradation. Next, we explored several architectural-level solutions that can more aggressively reduce energy, but at the expense of performance. We also developed techniques to exploit unused disk capacity to improve disks’ perfor- mance and energy-efficiency. These techniques are realized in our implementation of the Free Space File System (FS2). Unlike traditional file systems where extra disk space is not used, FS2 actively makes use of it to hold replicas of temporally-related data blocks that were poorly placed by the underlying file system. Using contiguous regions of free disk space to place related data blocks closer to one another enables disk heads to work more efficiently.
    [Show full text]
  • (12) Patent Application Publication (10) Pub. No.: US 2010/0293438 A1 Lastras-Montano Et Al
    US 2010O293438A1 (19) United States (12) Patent Application Publication (10) Pub. No.: US 2010/0293438 A1 Lastras-Montano et al. (43) Pub. Date: Nov. 18, 2010 (54) SYSTEM TO IMPROVE ERROR (73) Assignee: INTERNATIONAL BUSINESS CORRECTION USING VARIABLE LATENCY MACHINES CORPORATION, AND ASSOCATED METHODS Armonk, NY (US) (21) Appl. No.: 12/023.546 (75) Inventors: Luis A. Lastras-Montano, 1-1. Cortlandt Manor, NY (US); Piyush (22) Filed: Jan. 31, 2008 C. Patel, Cary, NC (US); Eric E. Publication Classification Retter, Austin, TX (US); Barry M. (51) Int. Cl. Trager, Yorktown Heights, NY H03M, 3/05 (2006.01) S. Mits; Snmuel WAEaWinograd, Cary, G06F II/It (2006.01) Scarsdale, NY (US); Kenneth L. (52) U.S. Cl. .................. 714/763; 714/E11.034; 714/785 Wright, Austin, TX (US) (57) ABSTRACT A system to improve error correction may include a fast Correspondence Address: decoder to process data packets until the fast decoder finds an IBM CORPORATION uncorrectable error in a data packet at which point a request ROCHESTER PLAW DEPT.917 for at least two data packets is generated. The system may also 3605 HIGHWAY 52 NORTH include a slow decoder to possibly correct the uncorrectable ROCHESTER, MN 55901-7829 (US) error in a data packet based upon the at least two data packets. 812 CHANNEL BUFFER FAST DECODER MARKING INFO SCORE 802- NEW ERROR CE, NCSE MARKING MARKING INFO LOCATION AND ENSUEFA INFO PREPROCESSING 804 MAGNITUDE CALCULATION MODIFIED COMPUTATION so 806 SYNDROME Eir ERROR 72B SYNDROME COMPUTATION MAGNITUDE CEON GENERATION COMPUTATION OPTIONAL FOR KNOWN SYNDROME LOCATIONS 811 BYPASS 807 SYNDROME BYPASS SELECT Patent Application Publication Nov.
    [Show full text]
  • Comptia® A+ Core 1 (220-1001) and Core 2 (220-1002) Cert Guide Fifth Edition
    CompTIA® A+ Core 1 (220-1001) and Core 2 (220-1002) Cert Guide Fifth Edition Rick McDonald CompTIA® A+ Core 1 (220-1001) and Core 2 (220-1002) Cert Guide Editor-in-Chief Fifth Edition Mark Taub Rick McDonald Product Line Manager Copyright © 2020 Pearson Education, Inc. Brett Bartow Published by: Pearson Education Acquisitions Editor 221 River Street Paul Carlstroem Hoboken, NJ 07030 Managing Editor All rights reserved. No part of this book may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any Sandra Schroeder information storage and retrieval system, without written permission from the publisher, except for the inclusion of brief quotations in a review. Development Editor Christopher Cleveland ISBN-13: 978-0-7897-6051-7 ISBN-10: 0-7897-6051-7 Project Editor Library of Congress Control Number: 2019908201 Mandie Frank ScoutAutomatedPrintCode Copy Editor Kitty Wilson Warning and Disclaimer This book is designed to provide information about the CompTIA Core 1 (220-1001) and Technical Editors Core 2 (220-1002) A+ exams. Every effort has been made to make this book as complete and Chris Crayton accurate as possible, but no warranty or fi tness is implied. The information provided is on an “as is” basis. The author and the publisher shall have neither liability nor responsibility to any Editorial Assistant person or entity with respect to any loss or damages arising from the information contained in Cindy Teeters this book or from the use of the supplemental online content or programs accompanying it. Microsoft and/or its respective suppliers make no representations about the suitability Designer of the information contained in the documents and related graphics published as part of Chuti Prasertsith the services for any purpose all such documents and related graphics are provided “as is” without warranty of any kind.
    [Show full text]