Lock Or No-Lock?

Lock Or No-Lock? Abstract guaranteeing atomicity are necessary when these operations must be atomic. Many scientific applications require high perfor- With the advent of parallel I/O libraries data can be mance concurrent IO accesses to a file by multiple pro- accessed in various complex patterns. Locking mecha- cesses. Those applications rely indirectly on atomic IO nisms are used to ensure that shared data is not being capabilities inorder to perform updates to structured violated. Adapted from the POSIX semantics, parallel datasets, such as those stored in HDF5 format files. file system like GPFS [9] and Lustre [10] provide byte Current support for atomic mode operations such as range locking mechanism. Byte range locks provide these in MPI-IO is performed by locking around the an option for guaranteeing atomicity of non-contiguous operations, imposing lock overhead in all situations, operations. By locking the entire region, changes can even though in many cases these operations are non- be made using a read-modify-write sequence. How- overlapping in the file. We propose to isolate non- ever, this approach does not consider the actual non- overlapping accesses from overlapping ones in collec- contiguous access pattern that may occur in a byte tive I/O cases, allowing the non-overlapping ones to range and introduces false sharing. This approach also proceed without imposing lock overhead. To enable this limits the benefits of parallel I/O that can be gained, we have implemented an efficient conflict detection al- by unnecessarily serializing the accesses. To address gorithm in MPI-IO using MPI file views and datatypes. these particular cases, [4, 6] propose to lock the exact We show that this conflict detection scheme imposes non-contiguous regions within a byte range and maxi- minimal overhead on I/O operations, making it an ef- mize the concurrent I/O access. fective mechanism for avoiding locks when they are not needed. The atomic mode is set in the application program, and relies on the locks supported by the file system to guarantee sequential consistency semantics. An obser- 1 Introduction vation is that the locks are requested even if there are no overlaps, which is not needed e.g. file locks, byte- range locks, and list locks. This leads us to a very im- Some of the scientific applications are I/O intensive portant question that is it possible to isolate the cases and demand high performance I/O access and band- where atomic mode is required and where it is not, and width to store and retrieve their simulation results. optimize the atomic mode. Of course it is possible, if we A relatively slow procedure to store and retrieve the filter out the overlapping and non-overlapping regions scientific application results to and from the storage and if there are no data overlaps during a concurrent system limits the overall performance despite of the access, atomic mode is unnecessary. ever increasing compute power. Parallel file systems and high level parallel I/O libraries are provided as We acquire this approach and propose a scheme a solution to mitigate this lag. Parallel file systems to identify the conflicts at I/O application level, by provide the semantics of local file system but face ad- identifying the concurrent access patterns and overlaps ditional challenges regarding consistency and atomic- within an application. Our algorithm, implemented ity. Supporting strict sequential consistency seman- at MPI-IO level for collective I/O operations uses file tics complicates the parallel file system design. Some views and datatype decoding to extract the exact off- parallel file system like PVFS [7] and PVFS2 [8] does set/length pair and determine the overlaps. We claim not support POSIX [16] semantics for atomicity and that when the file view for a process does not overlap consistency. These semantics are guaranteed only if with the file views of other processes, locking is not re- there is no data sharing among concurrent processes quired; there will be no conflicts. This approach will and are inadequate to describe complex requests in sci- also reduce the lock overhead at minimum and han- entific computing applications that are non-contiguous dle the overlapped region using the lock approach or in files. Scientific applications do perform these types adopting a logging scheme. of operations [18, 19], so alternative mechanisms for In the next section we briefly overview the atomic- 1 ity and consistency semantics for PFS and Parallel I/O MPI File sync. libraries. Section 3 discusses the design and algorithm, using MPI-IO concepts like datatype and file views, 2.2 UPC-IO section 4 describes the implementation with ROMIO, and section 5 presents the performance results of con- UPC-IO [17], by default, supports weak consistency flict detection. Finally, the last two sections consist of and atomicity semantics but the user can select the related work, conclusion and future work. stronger semantics by setting certain flags. No writes are visible unless sync is called in case of weak con- 2 Atomicity and Consistency Seman- sistency. If UPC STRICT flag is set, then user has tics selected the strong atomicity and consistency semantics and no call to the sync is required. In strict case atomicity is guaranteed at the UPC-IO function level. The consistency semantics specify the results of mul- Since, there is no guarantee of consistency and atom- tiple accesses to a single file whereas the atomicity se- icity by default unless atomic mode is set as described mantics define the outcome of multiple concurrent I/O earlier, we propose a self-checking atomic mode, that accesses, atleast one of which is a write to overlapping will extract the overlapping regions of a shared file and regions. Parallel I/O libraries like UPC-IO [17] and decide whether or not an atomic mode is needed. MPI-IO [1] provide atomic modes pertaining to their programming models. In parallel applications, the file access pattern can be contiguous or non-contiguous and 3 Design overlapping or non-overlapping. In the following sub- sections, we discuss the atomicity and consistency (AC) We propose a self-checking atomic mode, that will semantics provided by MPI-IO and UPC-IO for each be able to determine whether or not an atomic mode is of these access patterns. needed. Although we have designed our approach with MPI-IO, but the idea can be leveraged to UPC-IO or 2.1 MPI-IO any other parallel I/O libraries. The main idea is to uti- lize the file views created before a read or a write oper- MPI-IO allows the programmer to define complex ation by an application developer. In our algorithm, af- datatypes, hence creating complex file views and ac- ter deciding for the master node, the first step is to get cess patterns. Since ROMIO [1] implementation uses the file view per process that will return the displace- I/O APIs supported by the file system, the guaran- ment and the datatype. Next, the datatype is decoded tee of atomic mode indirectly relies on the file sys- by each process to get the offset/length pairs and com- tem. Some parallel file system supports POSIX se- municated to the master process. Finally, these values mantics that work for the contiguous access patterns are compared to get the conflicting regions. The cur- but have no control over the non-contiguous access rent design targets the collective I/O operations that patterns. ROMIO uses fcntl locks for non-contiguous are guaranteed to be performed at the same time by access, and locks the range of the bytes encompass- all calling processes. ing the non-contiguous access patterns. This approach serializes the accesses, if there is no true overlapping 3.1 File Views of patterns. To deal with this case, list locking and datatype locking is used that provides lock to the ex- File views allow the programmers to access the act non-contiguous regions with in a range. But if there data from the files by specifying varying offsets is no overlapping at all even in non-contiguous access, and access patterns. These access patterns are de- locking can be eliminated. Parallel file systems like fined as datatype. The datatypes can be predefined PVFS [7] and PVFS2 [8] do not provide atomicity and i.e.NAMED e.g. MPI Byte, MPI Int, etc or de- rely on the I/O library layer to provide these guaran- rived e.g. MPI Type Vector, MPI Type Indexed, etc. tees. In short, the default mode provided by MPI-IO For example if different processes are accessing dif- is non-atomic, in which the results of such concurrent ferent regions of a file in an interleaved manner, an accesses are undefined. The second mode is atomic, MPI Type Vector datatype can be created by spec- in which case the overlapping region will contain data ifying the constant stride and the block length as shown from any one process only if locks are supported by the in the figure 1. In the file view each process will specify parallel file system. its displacement from the start of the file. The con- Either MPI File set atomicity or MPI File sync in stant stride for each process is 3, and the block length MPI-IO can be used to enforce sequential consis- is 1 unit. The displacement is 0, 1, 2, and 3 for each tency. In later case, the data becomes visible to an- of the process respectively. The access pattern led by other process only after both writer and reader call file views of different processes may or may not have 2 P1 3.3 Conflict Detection View L P2 Each monotonically increasing list of offset/length View pair is received by master process, and each satisfies the following condition.

Load more