Louvre: Lightweight Ordering Using Versioning for Release Consistency

Pranith Kumar, Prasun Gera, Hyojong Kim, Hyesoon Kim Georgia Institute of Technology School of , College of Computing Atlanta, GA, USA {pranith, prasun.gera, hyojong.kim, hyesoon.kim}@gatech.edu

Abstract—Fence instructions are fundamental primitives that an ordering point in the instruction stream, and when it is ensure consistency in a weakly consistent multi- encountered by the processor, certain classes of optimizations core processor. The execution cost of these instructions is sig- on the following memory accesses are disabled to ensure con- nificant and adds a non-trivial overhead to parallel programs. In a na´ıve architecture implementation, we track the ordering sistency [4]–[6]. Optimizations such as speculative execution, constraints imposed by a fence by its entry in the reorder prefetch, and buffering of store instructions are implemented buffer and its execution overhead entails stalling the processor’s by modern processors to reduce the cost of memory accesses pipeline until the store buffer is drained and also conservatively [7]–[9]. The fence instructions restrict optimizations which invalidating speculative loads. These actions create a cascading exploit instruction-level parallelism, disable speculative exe- effect of increased overhead on the execution of the following instructions in the program. We find these actions to be overly cution of post-fence loads and stores, and introduce stalls to restrictive and that they can be further relaxed thereby allowing drain the store buffer. aggressive optimizations. Researchers have proposed techniques that allow a proces- The current work proposes a lightweight mechanism in which sor to employ hardware optimizations when ordering instruc- we assign ordering tags, called versions, to load and store in- tions are scheduled, thereby reducing their execution over- structions when they reside in the load/store queues and the write buffer. The version assigned to a memory access allows us to fully head [10], [11]. One such technique utilizes check-pointing exploit the relaxation allowed by the weak consistency model and along with aggressive speculative execution of post-fence restricts its execution in such a way that the ordering constraints loads and stores. The check-pointed state is tracked and by the model are satisfied. We utilize the information captured updated so that it enables roll back in case of ordering vi- through the assigned versions to reduce stalls caused by waiting olations [12]–[14]. Another promising approach is to increase for the store buffer to drain and to avoid unnecessary squashing of speculative loads, thereby minimizing the re-execution penalty. the granularity of enforcing consistency, thereby amortizing This method is particularly effective for the release consistency the overhead of ordering instructions. This technique has been model that employs uni-directional fence instructions. We show applied both in hardware [15] and at an algorithmic level [16], that this mechanism reduces the ordering instruction latency by [17]. Further, to increase the scope for speculative execution 39.6% and improves program performance by 11% on average based on the observation that ordering violations rarely occur over the baseline implementation. at runtime [12], studies have proposed techniques that identify scenarios in which violations are likely to occur and handle I.INTRODUCTION them in special ways. These techniques use annotations that As core counts have continued to increase over the last tag ordering instructions [18], hardware extensions [13], or decade, the shared memory programming model has become run-time information [19]. However, all these techniques were an attractive choice for high performance, scientific, and proposed in the context of stronger memory consistency mod- arXiv:1710.10746v3 [cs.AR] 5 Dec 2017 general purpose parallel applications. The use of this model els such as that of and little work has explored reducing is facilitated by high-level libraries and frameworks such as the overhead of ordering instructions, specifically in weak [1] and OpenMP [2]. The shared memory programming memory model architectures. model necessitates a consistent view of the memory for pro- Release consistency (RCsc) [20] is one of the most widely grammers to reason about the accuracy of parallel programs. implemented weak memory model in recent processors. Vari- This consistent view is dictated by the memory consistency ants of this model have been adopted in architectures such as model of the processor. , ARM64, and PowerPC and it is also being considered Higher level languages standardize on a consistency model for adoption in RISC-V processors [21]. In this memory to maintain portability across architectures. Languages such as model memory accesses are classified as ordinary, acquire, C and C++ have standardized on data-race-free memory mod- or release operations. This classification is done using special els [3] that guarantee that accesses to synchronization objects instructions in the ISA. A load-acquire operation and a store- (or atomics) are sequentially consistent. Sequential consistency release operation place ordering constraints either on the for such accesses is ensured by the , which generates memory accesses following it or preceding it respectively. This appropriate ordering instructions. These instructions indicate is in contrast to a full fence which places ordering constraints on both the memory accesses. These acquire and release two-way one-way 100% operations allow more leeway for possible re-ordering of the 80% memory accesses, thereby allowing a processor implementa- 60% tion to reduce the overhead of ensuring memory consistency. 40% We propose LOUVRE, a low-overhead hardware extension 20% that ensures memory ordering using versioning and helps in 0% reducing the overhead of fence instructions in a processor implementing the release consistency model. Our proposed mechanism adds ordering tag fields to the load/store queues and the write buffer. In the proposed technique, we assign Fig. 1. Distribution of fences in ARM64 versions to memory instructions when they are issued. We update the version being assigned upon issuing an ordering instruction. Instead of tracking the ordering constraints by the entry of the fence instruction in the reorder buffer (ROB), II.BACKGROUNDAND MOTIVATION we track them in a separate FIFO queue. This allows us to retire the ordering instruction from the ROB while preserving A. Fence instructions the constraint that needs to be enforced. Using the version as- signed to memory accesses, we identify and rectify any consis- Ordering instructions can be classified as either uni- tency violations at run-time without the need for maintaining directional (also known as one-way fences) or bi-directional any global state or inter-core communication. Compared to (also known as full fences). Instructions mfence and dmb are versioning, traditional implementation of ordering instructions bi-directional in that they do not allow any re-ordering across stall the pipeline to drain the store buffer to ensure order them. The restrictions placed by such fence instructions on among stores in the buffer and memory accesses following the memory accesses conflict with the optimization techniques ordering instruction. In the proposed versioning mechanism, used to hide the latency of long memory accesses in modern we issue and execute instructions that follow the ordering processors. Efforts to reduce this overhead have led to the instruction without draining the store buffer. Since we version design of the release consistency model [20]. This model pre- the stores in the store buffer, we enforce the correct ordering scribes acquire and release semantics, which are achieved by among them utilizing these versions. Furthermore, the ver- uni-directional fence instructions. These instructions allow re- sioning mechanism is efficient for one-way fence instructions, ordering of memory accesses across them in only one direction since it allows the processor to speculatively issue and execute and have either acquire or release semantics. Slight variations instructions following a store-release instruction before the of this conventional consistency model are implemented in store-release instruction itself has finished execution. architectures such as Itanium and ARM64. On ARM64, a Although we did not find any hardware optimizations in load-acquire instruction (also called as a synchronizing load, the literature specifically targeting one-way fence instructions, ldar) enforces an order between its load and all the following the work which is closest in spirit to our current work is memory accesses. A store-release instruction (also called as zFence [22]. As in the current work, the authors of the zFence a synchronizing store, stlr) enforces an order between its paper focus on reducing the stalls caused by waiting for the store and all the preceding memory accesses. A store-release store buffer to drain. They achieve this by introducing a new also ensures that the store is multi-copy atomic; a store coherence state which indicates exclusive permission to a is made visible to all other processors simultaneously [23]. cache line without the cache line being available. When all Additionally, a fence instruction is sequentially consistent with the stores waiting in the store buffer acquire this exclusive respect to all other fence instructions. permission, you do not need to drain the store buffer since no Since one-way fences remove constraints on possible mem- other processor can modify those cache lines, thereby reducing ory access re-ordering, generating them is more efficient than the fence overhead. In contrast, the current proposal requires generating full fences whenever possible. Figure 1 shows only minor hardware changes to avoid the store buffer drain the frequency distribution of one-way and full fences in stall. benchmarks from the Parsec [24] benchmark suite compiled The main contributions of our work are as follows: for ARM64 processors. We observe that one-way fences, on • We describe a detailed micro-architectural description for average, are twice as frequent as full fences in most of the implementing RCSC memory model semantics. benchmarks. Exploiting reduced constraints enforced by a one- • We propose a hardware mechanism based on versioning way fence necessitates an efficient micro-architectural imple- loads and stores that reduces the cost of ordering instruc- mentation. However, optimizations specific to one-way fences tions in weak memory model architectures. are limited. As a result, a uni-directional fence, when treated • We propose a low overhead mechanism that optimizes as a full fence limits the potential performance improvement. the execution of uni-directional ordering instructions by To the best of our knowledge, our proposal is the first to taking advantage of re-orderings allowed in a weak present a detailed optimal micro-architectural implementation memory model architecture. of one-way fences. B. Definitions RC1 and RC2 specify that memory accesses that are po- In the rest of the paper, we use the following terminology. before a full fence are ordered before the po-later memory An instruction is fetched (FE), decoded, and then issued (IS). accesses. Once it is issued (possibly out-of-order), it can be executed RC3 specifies that the load of a load-acquire fence should (EX) when all its dependencies are met. Once an instruction be ordered before any po-later memory accesses. reaches the head of the reorder buffer (ROB) and finishes RC4 specifies that the store of a store-release fence should execution, it can commit or retire (RE). Upon retiring, a store be ordered after all the po-previous memory accesses. instruction moves from the ROB to the store buffer (SB), at RC5 specifies that all the fence instructions should be which point it is said to be a post-commit store. A store will ordered in sequential order. complete (CO) once it updates the value in the cache and Finally, RC6 specifies that the value of a load is satisfied drains from the store buffer. A load is said to be satisfied from the latest po-before store to the same address pending in (SA) after it loads the value into a register. While a load the store buffer. If there is no such store, the load is satisfied can retire only after it is satisfied, a store can retire before from the latest global store to the same address. it completes. An instruction remains in a speculative state as We later show in Section IV-A that LOUVRE satisfies the long as it resides in the ROB. The life cycle of a memory RC semantics listed here. access instruction is shown in Figure 2. Please note that a SC satisfied load in a speculative state can be squashed (SQ) and re-executed along with all of its dependent instructions. D. Conventional Microarchitecture

SQUASH In this section, we describe three actions that a conventional SA processor performs to avoid ordering violations and to imple- LOAD ment the semantics of RC model listed in the previous FE IS EX RE CO SC STORE STORE section: retirement of a fence, draining the store buffer, and squashing a speculative load on cache line invalidation. The Fig. 2. Life-cycle of a memory access instruction. processor enforces RC1 for a full fence by draining the store buffer before retiring the full fence instruction. This ensures Program order (po) is the order in which instructions that post-fence stores do not complete before pre-fence stores are fetched and decoded by the same core. We use po- from the unordered store buffer. The processor enforces RC2 before/previous to refer to instructions that are earlier in by squashing any speculative post-fence load upon receiving program order and po-after/later/following to refer to instruc- an invalidation request to the cache line until the full fence tions that are later in program order relative to the current is retired. The one-way fences act as two individual halves instruction. of the full fence. The processor performs the same actions separately to enforce rules RC3, RC4, and RC5. To enforce C. RC Semantic Rules SC RC3 for a synchronizing load, the processor needs to squash The following semantic rules describe the constraints im- any speculative po-later loads upon invalidation as long as posed by RCSC [20]. Here X and Y are two instructions, the synchronizing load is in-flight. To enforce RC4 for a

RC1. X

2 Unnecessary Squash

Fig. 3. Inefficient load speculation in a weak consistency architecture.

III.MOTIVATION These scenarios arise in the presence of all three kinds of In this section we discuss the ordering constraints imposed fences. We detail one such scenario with a synchronizing store by different fence instructions and identify two major sources next. An example with a synchronizing load is illustrated in of overhead caused by these constraints. Section V-F. In Figure 3, instruction I1 is a store-release fence. Instruc- A. Constraints of ordering instructions tions I2 and I3 are independent loads that access addresses A full fence and a load-acquire fence enforce the constraint A1 and A2 respectively. Instructions I4 and I5 are dependent that the processor squash all speculative loads satisfied from a on the value loaded by instruction I3. The illustration shows a cache line when that cache line is invalidated. This constraint situation in which I2 has a cache miss but I3 has a cache hit. prevents speculative loads from using stale values, which Because of the cache hit, the speculative load I3 is satisfied would otherwise not be used if the loads were executed non- earlier than the speculative load I2. The synchronizing store speculatively. A full fence and a store-release fence enforce I1 is waiting for the store buffer to drain to prevent possible the constraint that the processor drain the store buffer of all reordering with any pending stores residing in the store buffer. po-previous stores before retiring the fence. In the case of At this point, an invalidation request for cache line containing a full fence, this constraint prevents a po-later store from A2 arrives (say from core 1) at 1 . Note that this is after completing ahead of a po-previous store. In the case of a the speculative load I3 is satisfied. However, since I3 is store-release fence, this constraint prevents the synchronizing still speculative, the processor squashes it and its dependent store from completing ahead of any po-previous stores. The instructions, I4 and I5, and re-executes them. However, since inefficiencies caused by these constraints are discussed in the there is no fence enforcing order on I3, the value loaded by following sections. it does not violate any ordering constraints. So squashing and re-execution of instructions I3, I4, and I5 is not necessary. B. Inefficient load speculation We identify such situations in LOUVRE and avoid the A load-store queue (LSQ) keeps track of all in-flight load squashing and re-execution penalty. This is explained later in and store instructions in program order. To detect ordering Section V-E. violations, the LSQ snoops the addresses of the incoming cache invalidations. If the address of a cache invalidation C. Inefficient store retirement matches that of an in-flight satisfied load, then the load might have read a stale value causing an ordering violation. In such Typically, a store instruction that misses in the cache waits cases, the processor squashes the load and all later instructions, in the ROB until the cache line is ready for an update. When and re-executes them. this store reaches the head of the ROB, instead of holding The first inefficiency we identify is the unnecessary invali- up the following instructions waiting for the cache line, the dation of speculative loads to prevent ordering violations in processor retires it and buffers it in the store buffer [7], [25]. A a processor implementing release consistency. While loads store retires from the ROB and moves to the store buffer from require squashing in stricter memory model processors, it where it drains after completion [26], [27]. In architectures is not always necessary in weaker model processors. In implementing the TSO memory model, the store buffer is particular, the processor needs to squash a speculative load ordered; the stores drain out from the buffer in FIFO order. only in the presence of an in-flight fence enforcing order on In weaker memory model architectures the store buffer can be the load. Since ordering instructions retire after they enforce unordered, that is, the stores can drain out in any order. In our the required constraints, a speculative load no longer violates experiments, we assume a weak memory model architecture ordering when all preceding fence instructions are retired. In and an unordered store buffer. Additionally, the store buffer is such cases we do not need to squash it on an invalidation. augmented with a CAM structure that tracks the destination waiting in S1: STR W0, [A0] cache miss FE IS EX RE store buffer CO cache hit (a) S2: STLR W1, [A1] FE IS EX RE CO Fence Delay S3: STR W3, [A2] FE IS EX cache hit RE CO (v) Early Retirement S1: STR W0, [A0] (0) FE IS EX cache miss RE CO (b) S2: STLR W1, [A1] (1) FE IS EX cache hit RE CO S3: STR W3, [A2] (0) FE IS EX cache hit RE CO Early Completion

Fig. 4. Early retirement and completion of stores from an unordered store buffer (a) baseline and (b) optimal with versions (ver) address and relative age of the store in the buffer. This is IV. LOUVRE necessary for store-to-load forwarding. In this section we describe the semantics of LOUVRE and explain how we employ the versioning rules to order loads The second inefficiency we focus on is the stall of a full and stores. fence or synchronizing store at the head of the ROB, while waiting for the stores in the unordered store buffer to drain. A. Semantic Rules This stall prevents ordering violations in which post-fence This section details the versioning semantic rules (VSR) stores are ordered before pre-fence stores. Without this stall, a that describe the global order enforced by versions. Here X processor, after retiring the fence will retire post-fence stores and Y are two instructions; vx and vy are the versions assigned and place them in the store buffer along with pre-fence stores. to those instructions. LDAR(X) is true if X is a load-acquire Since the store buffer is unordered, these post-fence stores can instruction. L(a) and S(a) are load and store to address a drain before pre-fence stores resulting in an ordering violation. respectively. The cost to drain the store buffer is the number of cycles it VSR1. X

Stores: A store reaching the head of ROB will retire im- Fig. 7. Overview of a simplified architecture implementing LOUVRE. mediately. Retired stores move to the store buffer awaiting completion. ORQ (orq): We need to track in-flight synchronizing loads Loads: In conventional implementations, a satisfied load that and full fences to squash speculative post-fence loads upon reaches the head of the ROB can retire. However, in LOUVRE, invalidation. We use an ordering queue to track these fences. we need to consider the stores residing in the store buffer A simplified architecture with the above structures is de- because of the relaxed constraints on fence retirement. In the tailed in Fig. 7. proposed mechanism, a load cannot retire as long as a store exists in the store buffer with a lower version. This is to ensure B. Version Registers Update that any store pending in the store buffer with a lower version We increment the last fence version register, lfvr, after completes before this load and hence is ordered before it. issuing any fence instruction. This is done to ensure that the Consider the case where a storeA; fence; loadB sequence ordering instructions are properly ordered. of instructions execute. We assign version 1 to store A and We update the version register vr to lfvr when we issue a version 2 to load B. load B should not retire until store A full fence instruction. This updated version register ensures completes from the store buffer even if it is at the head of that the version to later memory access is higher and hence ROB. orders them with previous memory accesses. To check for a store versioned lower than the load at the Table II summarizes the assignment of versions and updates head of the ROB without probing the store buffer, we use the to the version register vr after issuing an instruction. Since we vmin,sb. As explained in Section V-A, the version in vmin,sb is continuously increment the version registers, depending on its the minimum version of the stores in the store buffer. Hence, size, it will occasionally overflow. To handle this overflow, we if the version of the load at the head of the ROB is not greater stop issuing any further instructions and wait for the pipeline to than the vmin,sb, then we satisfy all the ordering constraints drain. After draining the pipeline, we reset the version registers and can retire the load at the head of the ROB. and start reissuing instructions. Assuming that we issue 10 Ordering Instructions A synchronizing store, similar to a fence instructions per kilo instructions, with a 10-bit version store, retires once it reaches the head of the ROB and then register we expect to see an overflow approximately once every moves to the store buffer. There is no other constraint on it hundred thousand instructions. since the version assigned to its store ensures the required ordering. A satisfied synchronizing load, similar to a load, TABLE II retires once it reaches the head of the ROB and its version VERSIONASSIGNMENTANDREGISTERUPDATEONISSUINGAN INSTRUCTION. is not greater than the version in the vmin,sb. A full fence instruction retires once it reaches the head of the ROB. There Issued Inst. Version assignment Register update is no delay in retiring a full fence instruction in the proposed load/store version = vr vr is not updated technique, since versioning ensures ordering on the memory load-acquire version = vr vr is not updated store-release version = vr + 1 vr is not updated accesses for which these fences are responsible. fence — vr = lfvr D. Store Completion Stores complete from the store buffer once they update We illustrate an example of version assignment to a se- the cache line. Since the store buffer is unordered, it allows quence of memory operations in Table III. out-of-order completion; the stores can complete in an order according to the versions assigned. For completion, of all C. Retirement the stores that have their cache line available, one of the In this section we detail the conditions required for the lowest versioned stores completes first. This ensures that the retirement of different kinds of instructions in LOUVRE. stores complete in the order enforced by the versions. We also complete the oldest store residing in the store buffer TABLE IV irrespective of its version, since no ordering constraint can SIMULATED ARCHITECTURE PARAMETERS exist on the oldest store. Processor 8-core, 2 GHz with 192 entry ROB E. Speculative Execution and Invalidation Fetch 6 instructions, 2 loads or stores per cycle As discussed in Section III-B, we do not need to squash Issue Out-of-order schedule and re-execute all speculative loads when we invalidate the Store Buffer 16 and 32 entry unordered buffer cache line that a load reads from. We need to squash loads L1 Cache 64 KB I-cache and D-cache per core only in cases where there is an in-flight full fence or a load- 2 cycle access latency acquire instruction enforcing order on it. We ensure this order L2 Cache 512 KB shared L2 cache (4 core) (LLC) by comparing the versions of the loads that need invalidation 10 cycle access latency and the versions vmin,sb and vmin,lsq. We squash and re- L3 Cache 8 MB shared L3 cache (LLC) execute speculative loads only if the version of the load is 25 cycle access latency greater than these versions. Only post-fence loads with an Coherence MESI protocol active fence (detected using orq) have a version that is higher Memory 4 GB, 100 cycle access latency than the version in vmin,sb and vmin,lsq. Squashing such loads ensures that it is not ordered ahead of the pre-fence stores, which may reside in the store buffer. In all the other cases, we the minimum version registers, we avoid squashing the load as do not squash loads upon a cache invalidation. This increases shown in Figure 9 (b). Note that both the executions produce the effectiveness of speculation in the presence of ordering a valid order even though the values read are different. In the instructions. base architecture, core 0 reads (W 0 : 0,W 1 : 1) whereas in We illustrate an optimal execution of the scenario shown in the LOUVRE architecture, core 0 reads (W 0 : 0,W 1 : 0). The Figure 3 when using versioning in Figure 8. In this example, only invalid combination is (W 0 : 1,W 1 : 0). instruction I1 is a store-release fence. Once it retires, the

ordering store moves to the store buffer with a version of G. Satisfying RCSC 1. The version assigned to loads I2 and I3 is 0. When the invalidation message for cache line containing A2 arrives at In this section, we discuss how the versioning rules de- RC 1 , we can determine that no ordering constraint exists by scribed in Section IV-B satisfy the ordering rules of SC listed in Section II-C. We satisfy RC1 and RC2 by versioning comparing the version of the load to the vmin,sb. Since the rule 1 which ensures that the assigned version is incremented version of the load is lower than vmin,sb, it allows us to skip squashing I3 thereby saving the re-execution penalty. when a full fence instruction is issued. We satisfy RC3 by the restriction on retirement of a synchronizing load as described in Section IV-B2. We satisfy RC4 by the versioning rule 3 Core #0 1 A2 Invalidation Core #1 No wait for store which assigns a higher version to the synchronizing store than I1: STLR W0, [A0] EX RE buffer drain the po-before memory accesses. Finally, we satisfy RC5 by I2: LDR W2, [A1] EX cache miss SA RE cache the squashing and retirement semantics of LOUVRE described I3: LDR W3, [A2] EX SA No RE hit in Section V. I4: ADD W4, W3, #1 EX Squash RE (= vmin, sb) I5: STR W4, [A3] EX RE VI.ANALYSIS AND EXPERIMENTS

OUVRE Fig. 8. Efficient load speculation in LOUVRE We implement and analyze L using 10-bit version tags. Each entry in the LSQ and the store buffer is augmented with these tags. For a 16 entry store buffer and a 64-entry F. Versioning Example LSQ the storage cost of these tags is (16 + 64) * 10 = 800 We present a simple two core execution scenario for a bits or 100 bytes. sample instruction sequence that has a synchronizing load in To implement the tracking of lowest versions in the store the base architecture in Figure 9 (a) and execution for the same buffer, we use a hierarchical comparator network [28]. This instruction sequence in LOUVRE in Figure 9 (b). network for a store buffer of 16 entry size uses 15 comparators. In this example, instruction I1 on core 1 generates an We also use two additional comparators to compare versions invalidation request when it writes to address A1. In the base in these version registers. architecture, this invalidation request squashes the satisfied In our experiments we assume a release consistency archi- load I2 on core 0 since it is speculative as shown in Figure 9 tecture loosely modeled on an in-production ARM64 system (a). However, since the synchronized load I1 on core 0 has in Apple’s Cyclone processor [29]. ARM64 follows the RCsc retired by this point, squashing I2 is unnecessary. In LOUVRE, [20], [30] consistency model where the ordering operations we can detect this scenario by comparing the version of the are sequentially consistent. We consider a 192 entry reorder load that the processor has to squash with the minimum ver- buffer which is interfaced to the data cache through the sion registers. Since the version of this load is not greater than load-store queue. We assume an out-of-order execution core Core #0 I1: LDAR W0, [X0] EX cache hit SA RE W0: 0 I2: LDR W1, [X1] EX cache hit SA SQ EX SA RE W1: 1

Core #1 X1 invalidation 1 2 X0 invalidation I1: STR #1, [X1] EX cache miss RE CO Initial: [X0] = 0, [X1] = 0 I2: FULL FENCE EX waiting for store buffer drain RE Core #0 Core #1 I3: STR #1, [X0] EX RE CO I1: LDAR W0, [X0] I1: STR #1, [X1] (a) Core #0 I2: LDR W1, [X1] I2: FULL FENCE No squash I3: STR #1, [X0] I1: LDAR W0, [X0] EX cache hit SA RE saved W0: 0 I2: LDR W1, [X1] EX cache hit SA RE W1: 0 Invalid: W0: 1, W1: 0 Core #1 X1 invalidation 1 2 X0 invalidation I1: STR #1, [X1] EX cache miss RE CO saved I2: FULL FENCE EX RE saved I3: STR #1, [X0] EX RE CO (b)

Fig. 9. A synchronizing load execution in (a) the base architecture (b) LOUVRE which can speculatively execute loads and stores. Other micro- A. Performance Evaluation architectural details of the simulation are listed in Table IV. The main performance benefit of the proposed mechanism We assume an invalidation based coherence protocol and use is being able to retire a fence instructions without waiting for MESI protocol in our experiments. So as to not complicate the store buffer to drain. Also speculatively executing loads our simulation, we assume that all the caches participate in and stores past the ordering instruction without incurring any the same coherence domain, i.e., any updates in one cache delay allows additional benefits. We keep track of the ordering will be propagated to all processors. Please note that this constraint using versions and the version registers when using assumption is stricter than that in the ARM memory model versioning which enables us to retire the fences ahead of time which allows non-multi-copy atomic behavior. We assume that when compared to a naive implementation. This retirement stores are multi-copy atomic in our model. We evaluate our reduces the stalls in the pipeline. Also note that since this implementation on architectures using an unordered 16 entry technique is primarily implemented within the core, based on store buffer. contention for shared lines, the performance should increase We model our simulator after the one implemented in the with increasing number of cores. O3 cpu model of the Gem5 simulator [31] as our baseline Figure 10 shows the performance of our versioning scheme architecture. In this model, speculatively executed loads are when compared to a recently proposed fence implementation squashed immediately upon invalidation and all the ordering zFence [22]. In the zFence mechanism, the stores with LLC instructions disable speculative execution of post-fence loads misses request exclusive permission, which when granted and stores. allows execution past the fence. This mechanism, similar to We evaluate our implementations in a cycle-accurate simu- ours, tries to reduce the delay of fence caused by draining lator implemented using the Qsim [32] framework which in- the store buffer. However, there is still a significant delay cludes SST [33] and MacSim [34]. We ran parallel benchmarks in acknowledging this permission request by the directory from two suites, PARSEC [24] and SPLASH-2 [35]. controller since you need to get the permission for all the stores residing in the store buffer before proceeding with further execution. Whereas in LOUVRE, no such permission request VII.RESULTS needs to be issued. On average we improve performance by 11% with streamcluster gaining the most at 18% IPC In this section we present our evaluation of the proposed over the baseline. When compared to zFence we improve versionining mechanism. We compare our results against the the performance by 4% on average. Although we did not base line version with an equal store buffer capacity. The var- perform any power consumption studies, we believe that the ious characteristics which we focus on to show the efficiency performance gains will translate to energy savings. of the proposed technique are as follows: B. Scheduling Stalls 1) Performance evaluation (IPC). 2) Reduction in stalls caused by an ordering instruction. As we explained in Section III-A, an ordering instruction 3) Reduction in store latency. introduces bubbles in the pipeline causing a stall. One of the 4) The average residency of a fence in ROB. major component of this stall is the wait for the store buffer to 25 zfence louvre 20 15

10 % IPC % 5 0 blackscholes canneal dedup fluidanimate stream swaptions barnes fmm ocean-cp radiosity HM

Fig. 10. Performance of LOUVRE when compared to zFence for a 16-entry store buffer. drain. Using a versioned store buffer eliminates this stall. The reduction in the stalls when compared to a conventional store avg buffer is shown in Figure 11. We see an average reduction of radiosity 19% in these stalls. ocean fmm 25 barnes 16 entry 32 entry

20 swaptions stalls 15 streamcluster

sched fluidanimate 10 dedup 5 canneal 0

blackscholes % reduction % in 0 2 4 6 8 10 12

Fig. 13. Average reduction in stalled cycles for post-fence memory accesses Fig. 11. Reduction in scheduling stalls in LOUVRE. when using a 16-entry versioned store buffer.

C. Store latency this can violate the memory consistency guarantees. Using Scheduling stores without any fence induced delay allows a versioned store buffer allows us to relax this constraint, us to issue a memory request earlier for it. Also, since the thereby, enabling us to speculatively execute post-fence mem- stores with the same version can complete out of order from ory accesses. The reduction in stalled cycles in camparison the store buffer, the store latency is reduced from an 63 cycles to a naive store buffer implementation is shown in Fig. 13. to 46 cycles on average, which is a 26% reduction as shown On average we find that a post-fence store in a versioned in Figure 12. It should be noted that savings in store latency store buffer implementation starts executing 8 cycles early do not directly transfer to increase in performance. This is (contributing to 9% reduction in latency) when compared to because of other architectural optimizations implemented that an implementation with naive store buffer. hide memory access latency like prefetching and speculative execution even in the baseline architecture.

40 E. Fence Residency in ROB 16 entry 32 entry 30 An instruction is allocated an entry in the ROB once it is 20 scheduled. This entry remains until the instruction is retired from the ROB. A fence instruction retires once it enforces all 10 its consistency guarantees. A naive store buffer needs to be 0 drained completely before a fence can be retired and removed % % reductioninstore latency from the ROB. A versioned store buffer, on the other hand allows us to retire the fence without any delays (as explained in Section V-C) from the ROB as the version assigned to Fig. 12. Reduced store latency caused by early retirement the memory access will be responsible for ensuring ordering. This reduces the lifetime of a fence in the ROB. In other words, it reduces the number of cycles a fence resides in the D. Early Execution ROB. This fence residency goes down from an average of 142 Whenever a full fence or an acquire fence is active, we cycles in the base implementation to 82 cycles in the versioned do not speculatively execute post-fence memory accesses as implementation, a reduction of 39.6% as shown in Fig. 14. 60 16 entry 32 entry seen in POWER processors. In such processors, the hardware threads share the store buffer from where they can read the 40 value of an earlier write from another hardware before others. However, in our modeling, to keep our simulation 20 simple, we assume that all the caches participate in the same

0 coherence domain and that there are no shared store buffers.

% reduction in fence reduction fence % in stalls We plan to explore the non-multi-copy atomic behavior in our future work. B. Branch Prediction Fig. 14. Reduced residency of fences in LOUVRE. Since we introduce two version registers vr and lfvr that are updated on instruction issue, we need to consider the scenario when branch mis-prediction occurs. We need to make F. Congested Store Buffer sure that the version registers have correct values when the One side-effect of delaying scheduling of stores past an processor starts fetching instructions from the correct path ordering instruction is not being able to issue a memory after recovering from a mis-prediction. We ensure this by request for them. Pre-fetching can mitigate this issue, though check-pointing registers along with the state on spec- you get it as a side-effect with LOUVRE. As a result of ulation. On mis-predicting, we flush the pipeline and restore not issuing a memory request early, there might be multiple the version registers from the check-pointed state. post-fence stores with pending memory requests. When these reach the head of the ROB, they are retired and placed in the C. Load-Store Queue Design store buffer. This causes congestion and leads to further stalls. We discuss two possible implementation choices for the Figure 15 shows the reduction in such stalls when we employ LSQ: the snoopy and the insulated queue [37]. In a snoopy versioning. In all but two benchmarks, we have a decrease in queue implementation, when a cache line is invalidated, the the number of stalls caused by a congested write buffer. In the LSQ is probed to squash all speculative loads that were two benchmarks ocean and radiosity using a 32-entry store satisfied from the invalidated cache line to prevent ordering buffer increases the number of stalls. This is because these violations. In an insulated queue implementation, this violation benchmarks are store intensive, and we issue lots of memory is prevented at the time of retiring a load. During retirement, requests causing greater cache pressure. the LSQ conservatively squashes all later loads in program order satisfied ahead of the current load that can potentially 160 versioning(entries=16) versioning(entries=32) violate consistency. Whereas processors with a stricter consis- 140 120 tency model usually employ a snoopy queue design, those with 100 80 a weaker consistency model use an insulated queue design. 60 After all, in a strict consistency model, we need to enforce 40

20 ordering on a large number of in-flight loads and stores. % store buffer store % buffer full 0 Since ordering violations in a weak consistency processor are infrequent, an insulated load queue is usually utilized at the cost of conservative performance. However, since a fence instruction enforces consistency, an insulated queue implementation can be used to avoid unnecessary squashes. Fig. 15. Stalls caused by the store buffer being full In such a case, at the time of load retirement, we check for an active fence instruction, and if such a fence is found, we VIII.DISCUSSION identify all the satisfied loads that are later in program order A. Multi-copy Atomicity and squash them. Architectures like ARM and POWER are not multi-copy D. Write Combining atomic [23], [36]. This property allows a store to become A write combining buffer merges multiple pending writes visible to only to some cores and not others. One of the reasons to the same address to reduce the number of memory trans- for this behavior is the sharing of cache by some cores. An actions. However, in LOUVRE, we ensure that the buffer only update from one core can become visible to the cores sharing merges stores with the same version. If it combines stores with cache with it before the update propagates to the memory. different versions, then an ordering violation can occur. Intuitively, the protocol should ensure that a reader sees the latest update in the entire cache hierarchy. IX.RELATED WORK However, if we restrict the coherence domain to the caches of Considerable previous work has been done with a focus on a core cluster, it will give rise to this behavior. Another reason reducing the cost of fences in both software and hardware. for this behavior is when multiple hardware threads share The main impetus for research in this area is that a consid- the same core using simultaneous multi-threading (SMT) as erable fraction of fences are determined to be unnecessary at run-time, or that they are ensuring ordering on memory REFERENCES accesses for which ordering is unnecessary, for example, [1] R. D. Blumofe, C. F. Joerg, B. C. Kuszmaul, C. E. Leiserson, K. H. thread-local memory accesses. Various hardware and software Randall, and Y. Zhou, “Cilk: an efficient multithreaded runtime system,” techniques were proposed to remove the fence overhead. in PPOPP ’95: Proc. of the fifth ACM SIGPLAN Symp. on Principles Software techniques focus mainly on compiler techniques and practice of parallel programming, 1995. [2] A. J. Dorta, C. Rodr´ıguez, F. de Sande, and A. Gonzalez-Escribano, “The which automatically infer fences [38], identify and remove OpenMP Source Code Repository,” Parallel, Distributed, and Network- redundant fences [39] or which reduce the strength of a Based Processing, Euromicro Conf. on, 2005. fence i.e., convert a stronger fence to a weaker fence. Hard- [3] S. V. Adve and M. D. Hill, “Weak ordering — a new definition,” in Proceedings of the 17th Annual International Symposium on Computer ware techniques focused on utilizing the dynamic information Architecture, ser. ISCA ’90. New York, NY, USA: ACM, 1990, pp. available at the time of execution of the fence to reduce or 2–14. [Online]. Available: http://doi.acm.org/10.1145/325164.325100 elide the overhead of ensuring ordering. Ordering needs to be [4]J. Sevˇ cˇ´ık, “Safe optimisations for shared-memory concurrent programs,” in Proceedings of the 32Nd ACM SIGPLAN Conference on Programming ensured only for shared memory locations since sequentially Language Design and Implementation, ser. PLDI ’11. New York, consistent (SC) semantics are guaranteed for local accesses NY, USA: ACM, 2011, pp. 306–316. [Online]. Available: http: i.e., a processor always has a sequentially consistent view of //doi.acm.org/10.1145/1993498.1993534 [5] E. Eide and J. Regehr, “Volatiles are miscompiled, and what its own memory accesses. Taking the help of the compiler to do about it,” in Proceedings of the 8th ACM International to determine such memory accesses and using an un-ordered Conference on Embedded Software, ser. EMSOFT ’08. New store buffer for the identified accesses reduces the overhead York, NY, USA: ACM, 2008, pp. 255–264. [Online]. Available: http://doi.acm.org/10.1145/1450058.1450093 of fence instruction [40]. [6] O. Trachsel, C. von Praun, and T. R. Gross, “On the effectiveness of Lin et al. proposed address-aware fences [41] in which speculative and selective memory fences,” in Proceedings of the 20th the shared memory addresses on which ordering is to be International Conference on Parallel and Distributed Processing, ser. IPDPS’06. Washington, DC, USA: IEEE Computer Society, 2006, pp. ensured are tracked in a watch list. Ordering is only ensured 33–33. [Online]. Available: http://dl.acm.org/citation.cfm?id=1898953. for these memory locations. All other memory accesses are 1898968 unordered. The authors extend the idea to order memory [7] Y. Chou, B. Fahs, and S. Abraham, “Microarchitecture optimizations for exploiting memory-level parallelism,” SIGARCH Comput. Archit. accesses only within the scope of the fence [41]. Singh et al. News, vol. 32, no. 2, pp. 76–, Mar. 2004. [Online]. Available: in [40] proposed an approach to design sequentially consistent http://doi.acm.org/10.1145/1028176.1006708 hardware in which they classify memory accesses as either [8] A. R. Lebeck, J. Koppanalil, T. Li, J. Patwardhan, and E. Rotenberg, “A large, fast instruction window for tolerating cache misses,” in ISCA-29, local or shared and try to maintain order only for the shared 2002, pp. 59–70. accesses. They do this by using an unordered store buffer [9] C.-K. Luk, “Tolerating memory latency through software-Controlled pre- for local memory accesses in addition to a regular FIFO execution in simultaneous multithreading processors,” in ISCA-23. New York, NY, USA: ACM, 2001, pp. 40–51. store buffer for shared memory accesses. The local memory [10] J. F. Mart´ınez and J. Torrellas, “Speculative synchronization: Applying accesses are processed through the unordered store buffer thread-level speculation to explicitly parallel applications,” SIGOPS avoiding the ordering overhead on them. Oper. Syst. Rev., vol. 36, no. 5, pp. 18–29, Oct. 2002. [Online]. Available: http://doi.acm.org/10.1145/635508.605400 [11] P. Ranganathan, V. S. Pai, and S. V. Adve, “Using speculative X.CONCLUSION retirement and larger instruction windows to narrow the performance gap between memory consistency models,” in Proceedings of the Ninth A processor implementing a weak memory model requires Annual ACM Symposium on Parallel Algorithms and Architectures, ser. SPAA ’97. New York, NY, USA: ACM, 1997, pp. 199–210. [Online]. higher number of fence instructions to ensure memory con- Available: http://doi.acm.org/10.1145/258492.258512 sistency for parallel programs than a processor implementing [12] C. Gniady, B. Falsafi, and T. N. Vijaykumar, “Is sc + ilp = rc?” stronger memory models. However, the weaker semantics pro- SIGARCH Comput. Archit. News, vol. 27, no. 2, pp. 162–171, May 1999. [Online]. Available: http://doi.acm.org/10.1145/307338.300993 vide new opportunities for extracting performance by allowing [13] Y. Duan, A. Muzahid, and J. Torrellas, “Weefence: Toward making us more leeway to reorder memory operations. Uni-directional fences free in tso,” SIGARCH Comput. Archit. News, vol. 41, no. 3, pp. fences allow us to only restrict certain possible reordering 213–224, Jun. 2013. [Online]. Available: http://doi.acm.org/10.1145/ 2508148.2485941 ensuring release consistency. In this paper, we propose a light- [14] Y. Duan, X. Feng, L. Wang, C. Zhang, and P.-C. Yew, “Detecting and weight mechanism which assigns versions to in-flight memory eliminating potential violations of sequential consistency for concurrent accesses. We detailed how fences can be retired without c/c++ programs,” in Proceedings of the 7th Annual IEEE/ACM International Symposium on Code Generation and Optimization, ser. causing any significant delays utilizing versions. With minor CGO ’09. Washington, DC, USA: IEEE Computer Society, 2009, pp. modifications to the micro-architecture where we add version 25–34. [Online]. Available: http://dx.doi.org/10.1109/CGO.2009.29 holding structures, we significantly reduce the fence overhead. [15] L. Ceze, J. Tuck, P. Montesinos, and J. Torrellas, “Bulksc: Bulk enforcement of sequential consistency,” SIGARCH Comput. Archit. We also studied the various architectural optimizations that can News, vol. 35, no. 2, pp. 278–289, Jun. 2007. [Online]. Available: be implemented in terms of memory re-orderings and spec- http://doi.acm.org/10.1145/1273440.1250697 ulative execution of memory accesses when uni-directional [16] J. H. Lee, J. Sim, and H. Kim, “Bssync: Processing near memory for machine learning workloads with bounded staleness consistency fence instructions are employed and presented a design for models,” in Proceedings of the 2015 International Conference on such an optimal implementation. We found that implementing Parallel Architecture and Compilation (PACT), ser. PACT ’15. LOUVRE reduces pipeline stalls caused by fences and store Washington, DC, USA: IEEE Computer Society, 2015, pp. 241–252. [Online]. Available: http://dx.doi.org/10.1109/PACT.2015.42 latency thereby increasing performance by 11% on average [17] K. Vora, S. C. Koduru, and R. Gupta, “Aspire: Exploiting asynchronous over the baseline and 4% over zFence. parallelism in iterative algorithms using a relaxed consistency based dsm,” SIGPLAN Not., vol. 49, no. 10, pp. 861–878, Oct. 2014. [Online]. [34] H. Kim, J. Lee, N. B. Lakshminarayana, J. Sim, J. Lim, and T. Pho, Available: http://doi.acm.org/10.1145/2714064.2660227 “Macsim: A cpu-gpu heterogeneous simulation framework user guide,” [18] C. Lin, V. Nagarajan, and R. Gupta, “Address-aware fences,” 2012. in Proceedings of the 27th International ACM Conference on [35] S. C. Woo, M. Ohara, E. Torrie, J. P. Singh, and A. Gupta, “The splash- International Conference on Supercomputing, ser. ICS ’13. New 2 programs: Characterization and methodological considerations,” in York, NY, USA: ACM, 2013, pp. 313–324. [Online]. Available: Proceedings of the 22Nd Annual International Symposium on Computer http://doi.acm.org/10.1145/2464996.2465015 Architecture, ser. ISCA ’95. New York, NY, USA: ACM, 1995, pp. [19] Y. Duan, N. Honarmand, and J. Torrellas, “Asymmetric memory fences: 24–36. [Online]. Available: http://doi.acm.org/10.1145/223982.223990 Optimizing both performance and implementability,” in Proceedings of [36] “Arm architecture reference manual for armv8-a architecture profile,” the Twentieth International Conference on Architectural Support for ARM. [Online]. Available: https://developer.arm.com/docs/ddi0487/a/ Programming Languages and Operating Systems. ACM, 2015, pp. arm-architecture-reference-manual-armv8-for-armv8-a-architecture-profile 531–543. [37] H. W. Cain and M. H. Lipasti, “Memory ordering: a value-based [20] K. Gharachorloo, D. Lenoski, J. Laudon, P. Gibbons, A. Gupta, and approach,” in Computer Architecture, 2004. Proceedings. 31st Annual J. Hennessy, “Memory consistency and event ordering in scalable International Symposium on, June 2004, pp. 90–101. shared-memory multiprocessors,” in Proceedings of the 17th Annual [38] M. Kuperstein, M. Vechev, and E. Yahav, “Automatic inference International Symposium on Computer Architecture, ser. ISCA ’90. of memory fences,” in Proceedings of the 2010 Conference on New York, NY, USA: ACM, 1990, pp. 15–26. [Online]. Available: Formal Methods in Computer-Aided Design, ser. FMCAD ’10. http://doi.acm.org/10.1145/325164.325102 Austin, TX: FMCAD Inc, 2010, pp. 111–120. [Online]. Available: [21] A. Waterman, Y. Lee, D. A. Patterson, and K. Asanovi, “The http://dl.acm.org/citation.cfm?id=1998496.1998518 risc-v instruction set manual, volume i: User-level isa, version [39] V. Vafeiadis and F. Z. Nardelli, “Verifying fence elimination 2.1,” EECS Department, University of California, Berkeley, Tech. optimisations,” in Proceedings of the 18th International Conference Rep. UCB/EECS-2016-118, May 2016. [Online]. Available: http: on Static Analysis, ser. SAS’11. Berlin, Heidelberg: Springer-Verlag, //www2.eecs.berkeley.edu/Pubs/TechRpts/2016/EECS-2016-118.html 2011, pp. 146–162. [Online]. Available: http://dl.acm.org/citation.cfm? [22] S. Aga, A. Singh, and S. Narayanasamy, “zfence: Data-less id=2041552.2041566 coherence for efficient fences,” in Proceedings of the 29th ACM [40] A. Singh, S. Narayanasamy, D. Marino, T. Millstein, and M. Musuvathi, on International Conference on Supercomputing, ser. ICS ’15. New “End-to-end sequential consistency,” in Proceedings of the 39th Annual York, NY, USA: ACM, 2015, pp. 295–305. [Online]. Available: International Symposium on Computer Architecture, ser. ISCA ’12. http://doi.acm.org/10.1145/2751205.2751211 Washington, DC, USA: IEEE Computer Society, 2012, pp. 524–535. [23] S. Flur, K. E. Gray, C. Pulte, S. Sarkar, A. Sezgin, L. Maranget, [Online]. Available: http://dl.acm.org/citation.cfm?id=2337159.2337220 W. Deacon, and P. Sewell, “Modelling the armv8 architecture, [41] C. Lin, V. Nagarajan, and R. Gupta, “Fence scoping,” in Proceedings operationally: Concurrency and isa,” in Proceedings of the 43rd Annual of the International Conference for High Performance Computing, ACM SIGPLAN-SIGACT Symposium on Principles of Programming Networking, Storage and Analysis, ser. SC ’14. Piscataway, NJ, Languages, ser. POPL ’16. New York, NY, USA: ACM, 2016, pp. 608– USA: IEEE Press, 2014, pp. 105–116. [Online]. Available: http: 621. [Online]. Available: http://doi.acm.org/10.1145/2837614.2837615 //dx.doi.org/10.1109/SC.2014.14 [24] C. Bienia, S. Kumar, J. P. Singh, and K. Li, “The PARSEC Bench- mark Suite: Characterization and Architectural Implications,” Princeton University, Tech. Rep. TR-811-08, 2008. [25] Y. Chou, L. Spracklen, and S. G. Abraham, “Store memory- level parallelism optimizations for commercial applications,” in Proceedings of the 38th Annual IEEE/ACM International Symposium on Microarchitecture, ser. MICRO 38. Washington, DC, USA: IEEE Computer Society, 2005, pp. 183–196. [Online]. Available: http://dx.doi.org/10.1109/MICRO.2005.31 [26] T. F. Wenisch, A. Ailamaki, B. Falsafi, and A. Moshovos, “Mechanisms for store-wait-free multiprocessors,” in Proceedings of the 34th Annual International Symposium on Computer Architecture, ser. ISCA ’07. New York, NY, USA: ACM, 2007, pp. 266–277. [Online]. Available: http://doi.acm.org/10.1145/1250662.1250696 [27] R. Bhargava and L. K. John, “Issues in the design of store buffers in dynamically scheduled processors,” in Performance Analysis of Systems and Software, 2000. ISPASS. 2000 IEEE International Symposium on, 2000, pp. 76–87. [28] D. Koch and J. Torresen, “Fpgasort: A high performance sorting architecture exploiting run-time reconfiguration on fpgas for large problem sorting,” in Proceedings of the 19th ACM/SIGDA International Symposium on Field Programmable Gate Arrays, ser. FPGA ’11. New York, NY, USA: ACM, 2011, pp. 45–54. [Online]. Available: http://doi.acm.org/10.1145/1950413.1950427 [29] A. L. Shimpi, “Apple’s cyclone microarchitecture de- tailed,” http://www.anandtech.com/show/7910/apples-cyclone- microarchitecture-detailed, 2014. [30] R. Grisenthwaite, “Armv8 technology preview,” 2011. [31] N. Binkert, B. Beckmann, G. Black, S. K. Reinhardt, A. Saidi, A. Basu, J. Hestness, D. R. Hower, T. Krishna, S. Sardashti, R. Sen, K. Sewell, M. Shoaib, N. Vaish, M. D. Hill, and D. A. Wood, “The gem5 simulator,” SIGARCH Comput. Archit. News, vol. 39, no. 2, pp. 1–7, Aug. 2011. [Online]. Available: http://doi.acm.org/10.1145/2024716.2024718 [32] C. D. Kersey, A. Rodrigues, and S. Yalamanchili, “A universal parallel front-end for execution driven microarchitecture simulation,” in Proceedings of the 2012 Workshop on Rapid Simulation and Performance Evaluation: Methods and Tools, ser. RAPIDO ’12. New York, NY, USA: ACM, 2012, pp. 25–32. [Online]. Available: http://doi.acm.org/10.1145/2162131.2162135 [33] Sandia National Laboratories, “SST,” http://sst.sandia.gov, 2015.