Method and System for Supporting Speculative Execution of Instructions

Europaisches Patentamt J European Patent Office © Publication number: 0 605 872 A1 Office europeen des brevets

EUROPEAN PATENT APPLICATION

@ Date of filing: 27.12.93

® Priority: 08.01.93 US 2445 @ Inventor: Levitan, David S. 9031 Marthas Drive @ Date of publication of application: Austin, Texas 7871 7(US) 13.07.94 Bulletin 94/28

© Designated Contracting States: © Representative: Lettieri, Fabrizio DE FR GB IBM SEMEA S.p.A., © Applicant: INTERNATIONAL BUSINESS Direzione Brevetti, MACHINES CORPORATION Ml SEG 024, Old Orchard Road P.O. Box 137 Armonk, N.Y. 10504(US) I-20090 Segrate (Milano) (IT)

© A data processing system executing speculative update value as the completion version value and instructions includes a memory for storing instruc- means (52) responsive to dispatch of a conditional tions at addresses, count registers (42, 44, 46) for branch instruction. Means (62) responsive to com- storing an update value, a dispatch version value pletion of the branch provide for decrementing con- and a completion version value. A fetcher connected tents of a completion version register. Finally, means to a branch unit fetches instructions from memory (58) responsive to occurrence of an interrupt prior to based upon addresses calculated by the branch unit, completion of the branch provide for replacing the which handles processing of conditional branch dispatch version value with the completion version instructions. Further included are means (60) respon- value to restore the system to a state prior to the sive to completion of initialization for copying the speculative execution of instructions.

MOVE_TO_COUNT STARTS

CM 00 m o CO

TO BRANCH UNIT

Rank Xerox (UK) Business Services (3. 10/3.09/3.3.4) 1 EP 0 605 872 A1 2

The invention relates to data processing sys- and makes fetches of instructions extremely fast. tems and in particular to a method and system for An instruction subset of great interest is that supporting speculative execution of program relating to conditional branches. Conditional branch instructions. Still more particularly, the invention instructions are instructions which dictate the taking relates to preservation of non-conditional state in- 5 of a specified conditional branch within an applica- formation for recovery after speculative execution tion in response to a selected outcome of the fails. processing of one or more other instructions. A Designers of data processing systems are con- practical example is a Fortran do-loop. Conditional tinually attempting to enhance the performance of branch instructions have long been a source of such systems. One technique for enhancing data io difficulty for pipeline computers (including RISC processing system efficiency is the achievement of systems). By the time a conditional branch instruc- short cycle times and a low Cycles-Per-lnstruction tion propagates through a pipeline queue to an (CPI) ratio in the system processor. An example of execution position within the queue, it will have the application of these techniques to data pro- been necessary to load instructions corresponding cessing system is the International Business Ma- 75 to one branch into the queue behind the conditional chines Corporation RISC System/6000(RS/6000)- branch instruction prior to resolving the conditional computer. The RS/6000 system is designed to branch, in order to avoid run-time delays. This perform well in numerically intensive engineering requires a choice be made as to which instruction and scientific applications as well as in multi-user, will follow the conditional branch without knowing commercial environments. The RS/6000 processor 20 the outcome of processing the related instructions. employs a superscalar implementation, which The choice can prove wrong. means that multiple instructions are issued and The execution of instructions prior to the final executed concurrently. possible definition of all conditions effecting execu- Processor architecture relates to the combina- tion is called speculative execution. To wait for the tion of registers, arithmetic units and control logic 25 outcome of conditional branches, or the arrival of to build the computational elements of a computer. all possible interrupts, would make full concurrent An important consideration during building of a processing impossible. Thus, some scheme for processor is the instruction set it will provide. An processor recovery from speculative execution of instruction is a statement which specifies an opera- instructions must be provided if full use of concur- tion and the values or locations of its operands. An 30 rent execution of instructions is to be made. Upon instruction set is the collection of all such valid determination that execution is proceeding down an statements for a particular machine. incorrect branch an interrupt may be generated to As originally conceived, RISC machines would change the course of execution. In responding to execute one instruction per machine cycle. To this an interrupt, the processor is returned to the last end all instructions were of one length and fit a 35 non- speculative execution step. scheme compatible with a pipeline implementation. Experience has demonstrated that use of some Simplicity in the instruction set was the design complex operations in RISC machines can improve objective. This allowed further reduction in the cy- performance. This in part stems from the nature of cle time compared with so called complex instruc- currently preferred technology for implementation tion set computers (CISC). However, some of the 40 of processors, i.e. very large scale integration benefits of RISC were offset by increases in traffic (VLSI). Minimization of area used on a chip is now between the processor and the main memory for a more important than minimizing the number of de- computer. This occurred because a RISC machine vices used to implement the processor. Hence, requires more instruction instances to do a task some complex instructions have begun infiltrating than a CISC machine with its more powerful in- 45 into RISC based designs. The criteria for inclusion struction set. is minimum utilization of space. One instruction in Concurrence in issuance and execution of mul- the RS/6000 instruction set allows execution of a tiple instructions requires independent functional branch on count loop. The branch on count instruc- units that can execute with a high instruction band- tion is a one step instruction replacing what was width. The RS/6000 system achieves this by utiliz- 50 formerly done in three instructions. Substitution of ing separate branch, fixed point and floating point a single instruction for three instructions was en- processing units which are pipelined in nature. The abled by providing a dedicated count register. branch processing unit handles conditional branch However, this arrangement does not in itself sup- instructions. In common with other RISC designs, port speculative execution. Implementation of the complex decoding logic no longer required to de- 55 count register could be done by a mechanism code instructions has been utilized to provide an provided in RS/6000 machines for register rename, instruction cache on the processor chip. This re- but the value for the count register would not be duces traffic between the processor and memory, known during the dispatch cycle resulting in some

2 3 EP 0 605 872 A1 4 loss of machine cycles. Figure 3 is a schematic illustration of a branch Desirable is a hardware implementation of the on count register architecture in accordance with branch on count loop which uses a minimum a preferred embodiment of the invention; and amount of area on a processor chip. Figure 4 is a schematic illustration of a branch It is therefore one object of the invention to 5 on count register architecture in accordance with provide an improved method and system for sup- a second preferred embodiment of the invention. porting speculative execution of program instruc- With reference now to the figures and in par- tions. ticular with reference to Figure 1 , there is depicted It is another object of the invention to provide a high level block diagram of a superscalar com- preservation of conditional state information for re- io puter system 10 which may be utilized to imple- covery after speculative execution fails. ment the method and system of the present inven- The foregoing objects are achieved by the in- tion. As illustrated, computer system to preferably vention as claimed. The invention provides a data includes a memory 18 which is utilized to store processing system for speculatively executing data, instructions and the like. Data or instructions instructions. The data processing system includes is stored within memory 18 are preferably accessed a memory for storing instructions at addresses utilizing cache/memory interface 20 in a method which can be generated by a branch unit in a well known to those having skill in the art. The processor. The processor also has a count register sizing and utilization of cache memory systems is for storing an update value, a dispatch version a well known subspecialty within the data process- value and a completion version value of a branch 20 ing art is not addressed within the present applica- control count. A fetcher connected to the branch tion. However, those skilled in the art will appre- unit fetches instructions from memory based upon ciate that by utilizing modern associative cache addresses calculated by the branch unit. The techniques a large percentage of memory access- branch unit handles processing of conditional es may be achieved utilizing data temporarily branch instructions. To do so, means for initializing 25 stored within cache/memory interface 20. the update value and the dispatch version value for Instructions from cache/memory interface 20 branch control are provided. Further included are are typically loaded into instruction queue 22 which means responsive to completion of initialization for preferably includes a plurality of queue positions. copying the update value as the completion version In a typical embodiment of a superscalar computer value. The system further includes means respon- 30 system the instruction queue may include eight sive to dispatch of a conditional branch instruction queue positions and thus, in a given cycle, be- for examining the dispatch version value to deter- tween zero and eight instructions may be loaded mine if a branch should be taken and then de- into instruction queue 22, depending upon how crementing the dispatch version value. Means re- many valid instructions are passed by sponsive to completion of the branch provide for 35 cache/memory interface 20 and how much space is decrementing contents of a completion version reg- available within instruction queue 22. ister. Finally, means responsive to occurrence of an As is typical in such superscalar computer interrupt prior to completion of the branch provide systems, instruction queue 22 is utilized to dis- for replacing the dispatch version value with the patch instructions to multiple execution units. As completion version value to restore the system to a 40 depicted within Figure 1, computer system 10 in- state prior to the speculative execution of instruc- cludes a floating point processor unit 24, a fixed tions. point processor unit 26, and a branch processor The novel features believed characteristic of unit 28. Thus, instruction queue 22 may dispatch the invention are set forth in the appended claims. between zero and three instructions during a single The invention itself however, as well as a preferred 45 cycle, one to each execution unit. mode of use, further objects and advantages there- In addition to sequential instructions dispatched of, will best be understood by reference to the from instruction queue 22, so-called "conditional following detailed description of an illustrative em- branch instructions" may be loaded into instruction bodiment when read in conjunction with the accom- queue 22 for execution by the branch processor. A panying drawings, wherein: 50 conditional branch instruction is an instruction Figure 1 is a high level block diagram of a which specifies an associated conditional branch to superscalar computer system which may be uti- be taken within the application in response to a lized to implement the method and system of selected outcome of processing one or more se- the present invention; quential instructions. In an effort to minimize run- Figure 2 is a reservation table illustrating the 55 time delay in a pipelined processor system, such manipulation of instruction queue content in a as computer system 10, the presence of a con- prior art data processing system utilizing an ditional branch instruction within the instruction instruction queue; and queue is detected and an outcome of the con-

3 5 EP 0 605 872 A1 6 ditional branch is predicted. As should be apparent in queue 3 of the prior art instruction queue, the to those having skill in the art when a conditional sequential instructions within the queue are loaded branch is predicted as "not taken" the sequential into an alternate instruction queue (not shown). instructions within the instruction queue simply Thereafter, a request for target instructions asso- continue along a current path and no instructions 5 ciated with the conditional branch is initiated at are altered. However, if the prediction as to the cycle 2 and those instructions are loaded into the occurrence of the branch is incorrect, the instruc- instruction queue at cycle 3. These instructions are tion queue must be purged of sequential instruc- based upon the prediction that the conditional tions, which follow the conditional branch instruc- branch associated with the conditional branch in- tions in program order and target instructions must io struction will be "taken." be fetched. Alternately, if the conditional branch is Thereafter, at cycle 4, the compare (cmp) in- predicted as "taken" then the target instructions struction has propagated to the execution position are fetched and utilized to follow the conditional within the instruction queue and the conditional branch, if the prediction is resolved as correct. And branch instruction is "resolved." In the event the of course, if the prediction of "taken" is incorrect is resolution of the conditional branch instruction in- the target instructions must be purged and the dicates that the conditional branch is "not taken" sequential instructions which follow the conditional the sequential instructions previously loaded into branch instruction in program order must be re- the alternate instruction queue are once again load- trieved. ed into the primary instruction queue, as depicted As illustrated, computer system 10 also prefer- 20 at cycle 5. Cycles 6 and 7 within the instruction ably includes a condition registers 32. Condition queue of Figure 2 indicate the subsequent process- registers 32 are utilized to temporarily store the ing of additional sequential instructions. As illus- results of various comparisons which may occur trated, only a single empty cycle is present within utilizing the outcome of sequential instructions the instruction queue following the misprediction of which are processed within computer system 10. 25 the conditional branch instruction. However, as de- Thus, floating point processor unit 24 and fixed scribed above, the implementation of this prior art point processor unit 26 are coupled to condition technique requires the utilization of an alternate registers 32. The status of a particular condition instruction queue. within condition registers 32 may be detected and Referring to Figure 3, a block diagram sche- coupled to branch processor unit 28 in order to 30 matic of a region of registers is depicted, including generate target addresses, which are then utilized general purpose register 40 and three dedicated to fetch target instructions in response to the oc- registers used to implement a branch on count currence of a condition which initiates a branch. instruction with predictive branching, speculative Thereafter, branch processor unit 28 couples execution of instructions and recovery from specu- target addresses to fetcher 30. Fetcher 30 cal- 35 lative execution of instructions along a wrong culates fetch addresses for the target instructions branch. The three dedicated registers include a necessary to follow the conditional branch and cou- dispatch version register 42, an update version ples those fetch addresses to cache/memory inter- register 44 and a completion version register 46. face 20. As will be appreciated by those having Functionally, the dispatch version register 42 skill in the art, if the target instructions associated 40 provides a dispatch stage version of the count to with those fetch addresses are present within be used for address generation. The dispatch stage cache/memory interface 20, those target instruc- version always leads the completion stage version tions are loaded into instruction queue 22. Alter- of the count stored in completion version register nately, the target instructions may be fetched from 46. The completion stage version count is the memory 18 and thereafter loaded into instruction 45 count corresponding to the last speculatively ex- queue 22 from cache/memory interface 20 after a ecuted and confirmed instruction. The contents of delay required to fetch those target instructions. update version register 44 correspond to the count The manipulation of instruction queue content for an instruction speculatively executed, which will in a prior art data processing system utilizing an be copied into register 46 when confirmed. The alternate instruction queue is illustrated in Figure 2 50 purpose of the update version of the count is to within reservation table 36 therein. Figure 2 depicts enable decrementing of the dispatch version before a table illustrating manipulation of instruction queue the MOVE_TO_COUNT instruction completes ex- data content through seven consecutive cycle ecution. The MOVE_TO_COUNT instruction may times. It may be seen that at cycle time 1, the itself be speculatively executed. instruction queue includes a conditional branch in- 55 The movement of data into registers 42, 44 struction (be), a compare instruction (cmp) and four and 46 is controlled by occurrence of certain arithmetic logic unit (alu) instructions. Upon the instructions from the processor instruction set. The detection of the conditional branch instruction with- instructions are generated by a compiler and de-

4 7 EP 0 605 872 A1 8 coded by conventional control logic of the proces- BRANCH ON COUNT instructions, which cannot sor unit. Speculative execution of a branch of a execute until a MOVE_TO_COUNT instruction loop in a program is initiated by fixed point unit 26. completes and the completion version register 46 A store instruction loading a count value into gen- has been loaded. eral purpose register 40 is executed. Subsequently 5 With initiation of a MOVE_TO_COUNT in- execution of a MOVE_TO_COUNT instruction be- struction, a gate signal is applied to gate 50 copy- gins, resulting in application of a gate control signal ing the contents of general purpose register 40 into to gates 48 and 50. As a result the contents of dispatch version register 42. When the general purpose register 40 are copied into dis- MOVE_TO_COUNT completes a signal applied patch version register 42 and update version regis- io to gate 60 results in the contents of the dispatch ter 44. version register 42 being applied to completion Upon dispatch of a BRANCH ON COUNT version register 46. Now a DIS- instruction the contents of register 42 are examined PATCH_ON_COUNT instruction can be execut- to determine if the branch should be taken or if ed. The handling of BRANCH ON COUNT execution of steps of the program should fall is instructions, interrupts and complete through sequentially. A gate control signal is ap- BRANCH ON COUNT is identical to the first em- plied to gates 52 and 56, which results in the bodiment. application of the contents of register 42 to branch unit 28 and copying into register 42 of the prior Claims contents of the register less 1 by route of de- 20 crementer 54. 1. A processor for a data processing system (10) If an interrupt is taken prior to completion of including a branch unit (28) for operating on instructions following a BRANCH ON COUNT in- conditional branch instructions and calculating struction, indicating, for example, that the wrong target addresses for use in fetching instruc- sequence of instruction was followed, a gate con- 25 tions from a memory device (18), said proces- trol signal is applied to gate 58. Gate 58 is used to sor comprising: copy the contents of completion version register 46 a source of an initial version of a count into dispatch version register 42. This returns the value related to a conditional branch; state of dispatch version register 42 to that preced- means (42) for storing a dispatch version ing execution of any speculative instruction not yet 30 of the count value; confirmed. means (46) for storing a completion ver- As previously described, gate 48 controls the sion of a count value corresponding to non- copying of the contents of general purpose register speculatively executed instructions; 40 to update version register 44 with initiation of a means (50) responsive to beginning ex- MOVE_TO_COUNT instruction. The contents of 35 ecution of a move to count instruction for load- update register 44 are copied to completion version ing the initial version of the count value into register 46 by gate 60 with completion of the said means (42) for storing the dispatch ver- MOVE_TO_COUNT instruction. There the initial sion; count is preserved until a branch following a means responsive to completion of a move BRANCH ON COUNT instruction is completed 40 to count instruction for moving initial version of and confirmed. Upon completion logic of the pro- the count value to said means (46) for storing cessor signal removing the tentative markings from the completion version; the results of the branch, a gating signal is applied means (52, 54) responsive to dispatch of a to gate 62 resulting in the contents of register 46 branch on count instruction for decrementing being decremented by decrementer 64 and the 45 the dispatch version; and result being copied back into register 46. means (62, 64) responsive to completion Figure 4 illustrates a simplified but lower per- of a branch for decrementing the contents of formance embodiment of the invention, in which said means (46) for storing the completion gate 46 and update version register 44 have been version. eliminated. The movement of data into dispatch 50 version register 42 and completion version register 2. A processor as set forth in Claim 1 .wherein 46 is controlled by signals to various gates. The said means for moving the initial version of the completion version register 46 now receives its count values includes: data from the dispatch version register 42, rather update means (44) for storing the initial than the update version register. The signals are 55 version of the count; and similar to those discussed with reference to Figure a gate (60) for transferring the content of 3, but occur at somewhat different times. The most said update means (44) to said means (46) for important change in timing is that for storing the completion version.

5 9 EP 0 605 872 A1 10

3. A processor as set forth in Claim 2, and further dispatching a branch on count instruction comprising: without regard to the number of uncompleted means (58) responsive to an interrupt for branch on count instructions already queued. loading the contents of said means (46) for storing the completion version into said means 5 9. A data processing system (10) for specula- (42) for storing the dispatch version. tively executing instructions including: a memory (18) for storing instructions; 4. In a data processing system (10) having a a dispatch version register (42) and a com- branch unit (26) in a processor for processing pletion version register (46); conditional branch instructions, a method of 10 means (30) for fetching instructions from speculatively executing instructions recovered said memory (18) based upon calculated ad- from memory (18) based upon addresses cal- dresses; culated by said branch unit (28), the method a branch unit (28) for processing condi- including the steps of: tional branch instructions; initializing a dispatch version register (42) is means (40, 50) for initializing the contents with a count value for control of the conditional of the dispatch version register (42) for branch branch; control; upon completion of initialization of said means (60) responsive to completion of dispatch version register (42), copying the ini- initialization for copying initialization data into tialization data to a completion version register 20 said completion version register (46); (46); means (52) responsive to dispatch of a responsive to dispatch of a conditional conditional branch instruction for examining the branch instruction, examining contents of said dispatch version value to determine a branch dispatch version register (42) to determine a to take and then decrementing the contents of branch to be taken and then decrementing the 25 said dispatch version register (42); contents of said dispatch version register (42); means (62) responsive to completion of a responsive to completion of the taken branch for decrementing contents of said com- branch, decrementing contents of said comple- pletion version register (46); and tion version register (46); and means (58) responsive to occurrence of an upon occurrence of an interrupt prior to 30 interrupt prior to completion of the branch for completion of the taken branch, copying the replacing the content of said dispatch version contents of said completion version register register (42) with the contents of said comple- (46) to said dispatch version register (42). tion version register (46) to restore the system to a state preceding the speculative execution 5. A method as set forth in Claim 4, and further 35 of instructions comprising executing the steps subsequent to the step of copying the contents of an update 10. A data processing system (10) as set forth in version register (44) into said completion ver- Claim 9, wherein said means (40, 50) for initial- sion register (46) as a loop. izing the contents of the dispatch version value 40 comprise: 6. A method as set forth in Claim 5, wherein the means for loading a value into a general step of initializing comprises: purpose register (40); and loading a value into a general purpose means (50) for executing a move of the register (40) of the processor; and contents of the general purpose register (40) executing a move of the contents of said 45 into said dispatch version registers (42). general purpose register (40) into said dispatch version register (42) and into said update ver- 11. A data processing system as set forth in Claim sion register (44). 10, wherein the dispatch of branch on count instructions may be made without regard to the 7. A method as set forth in Claim 6, wherein the 50 number of uncompleted branch on count step of copying the initialization data into said instructions already queued. completion version register (44) comprises: copying the contents of said update ver- 12. A data processing system as set forth in Claim sion register (44) into said completion version 11, and further comprising: register (46). 55 an update version register (44); means (46) for initializing the contents of 8. A method as set forth in Claim 7, and further the update version (44) register synchronously comprising: with initialization of said dispatch version regis-

6 11 EP 0 605 872 A1 12 ter (42); and the means (60) for copying initialization data into the completion version is connected between said update register (44) and said completion version register (46) for copying 5 the contents of the update version register (44) into said completion version register (46).

7 EP 0 605 872 A1

10 f

MEMORY 18

20-J FETCH ADDRESSES CACHE/MEMORY FETCHER INTERFACE

A D FLOATING POINT BRANCH INSTRUCTIONS INSTRUCTION INSTRUCTIONS QUEUE 22" 24 FIXED POINT I INSTRUCTIONS

FLOATING FIXED POINT 2 8-J BRANCH POINT UNIT UNIT UNIT "M6 1 3 2-lI CONDITION REGISTERS

Fig. /

8 r—- u"> •««■ CM — C3 t/> (/■> oo oo L/*> oo

to M N O l/-> I/O C/1

ro CM »— C3 E l/"> 00 C> «> c_> ^ •c-»

ex. = K> CM — O E — ) — 1 — I — 1 — 0 0

• <~>

0-1 — 0 E — — t 1 H— O O O

cm » " = E t> O O O

>~ 3 3 3 => O E„ — — — — -O «J O O O O

„, t-O CM »_ _ °» t7> 0> 0> Q> "O 3 O =» 3 3 O 0 °° I - — a> a> a> 0 a> ' e/» 3 3 3 a> x 10 O Or o- Q i^J ^ NO

n ratent 3 ^ropean EUROPEAN SEARCH REPORT AppHca,i0,, Numb* Office Ep 93 12 og40

LKJCUMEN IS CONSIDERED TO BE RELEVANT Category Citation of document with indication, where appropriate, Relevant CLASSIFICATION OF THE of relevant passages to claim APPLICATION (Int.CL5) A US-A-5 101 484 (KOHN) 1,4,9 G06F9/38 * the whole document *

A IEEE TRANSACTIONS ON COMPUTERS 1,4,9 vol. 37, no. 5 , May 1988 , NEW YORK US pages 562 - 573 SMITH AND PLESZKUN 'Implementing precise interrupts in pipelined processors' * page 566, section IV; page 568, section VI *

A IBM TECHNICAL DISCLOSURE BULLETIN. 1,4,9 vol. 36, no. 1 , January 1993 , NEW YORK US pages 262 - 264 'Looping in MSIS' * the whole document *

A SUPERCOMPUTING '90 12 November 1990 , NEW 1,4,9 YORK, US pages 200 - 212 technical fields TIRUMALAI. LEE AND SCHLANSKER searched a»ta.s) 'Parallelization of loops with exits on G06F pipelined architectures' * page 201, right column, line 35 - page 202, left column, line 16 *

The present search report has been drawn up for all claims Placeofuarct DMe of coquette! of the ie*ck Exaniaer THE HAGUE 25 April 1994 Weinberg, L CATEGORY OF CITED DOCUMENTS T : theory or principle underlying the invention E : earlier patent document, but published on, or „X : particularly, relevant if taken alone after the riling date Y : particularly relevant if combined with another D : document dted in the application document of the same category L : document dted for other reasons A : technological background O : non-written disclosure A : member of the same patent family, corresponding P : intermediate document document