DESIGNING READ AND WRITE APPLICATION BUFFERS FOR THE R4000 NOTE AN-114 SYSTEM INTERFACE Integrated Device Technology, Inc. By Andrew Ng INTRODUCTION THE R4000 MICROPROCESSOR This article describes the basic concepts behind designing The IDT79R4000 MIPS CPU brings high performance 64/ with the IDT79R4000 System Interface. The System Interface 32-bit computing to a single chip microprocessor and thus connects the R4000 CPU to external memory and peripher- extends the family of R3000 compatible parts from the lower als. Topics include: (1) what the basic read and write memory cost 32-bit R3051 CPU and R3081 CPU/FPA. Bench- transactions look like, (2) the basic architecture for designing marks for R4000 systems show their performance to be from buffers and transceivers into the address and data bus paths, 35-54VUPS (VAX Units of Performance) and from 44-72 and (3) explains the convention of using single level read SPECmarks. Initial R4000 parts are being produced to run buffers and multi-level write buffers. The read and write with an external 50MHz clock frequency and future parts with buffers can obviously be implemented with custom FPGAs or the same external bus interface are planned with larger ASICs. However, read and write buffers can also be easily primary caches and for frequencies over 75MHz. As shown in implemented using off-the-shelf discrete logic FIFOs and the block diagram in Figure 1, the R4000 has high perfor- pipelined registers. Thus to more clearly illustrate a read and mance in large part because of its superpipelined architecture write buffer implementation, brief discrete logic examples are which allows a 100MHz internal clock speed which is double given using the 18-bit IDT Double-Density FCT16823T regis- the external clock speed. The R4000 also has an on-chip ter with clock enable, the 16-bit IDT 73200 multi-level pipeline floating-point accelerator, on-chip write-back primary instruc- register, and the 8-bit IDT73210 2-level/1-level pipelined tion and data caches, an optional writeback secondary cache registered transceiver. interface, and on-chip memory management. The Reduced Clock/Control Initialization JTAG Interface Interface Interface FPU ALU 8KByte 8KByte Instruction Data 96-entry Cache Cache TLB Mult/Divide/ Square Root FP Registers Cache Control General Registers Pipeline/Ctrl ALU/Multiply/Divide MMU Pipeline/Control Integer Execution Unit Cache/MMU Floating Point System Secondary Cache Interrupt Interface Interface Interface drw 01 Figure 1. Block Diagram of the R4000 R3081, R3051, R3052 and CEMOS are trademarks of Integrated Device Technology, Inc. R3000 and R4000 are trademarks of MIPS Computer Systems, Inc. 1996 Integrated Device Technology 123 2898/- 2/96 DESIGNING READ AND WRITE BUFFERS FOR THE R4000 SYSTEM INTERFACE APPLICATION NOTE AN-114 Instruction Set Architecture (RISC) and its development envi- R4000 SYSTEM INTERFACE ronment of optimized operating systems, compilers, and As shown in Figure 3, the R4000 System Interface consists rescheduling assemblers place their emphasis on high perfor- of the signals that connect the CPU to the outside world of mance and speed. The R4000 has 3 variants: (1) the 179-pin peripherals and memory. The System Interface has three R4000PC which comes without a secondary cache interface, major elements: (2) the 447-pin R4000SC which comes with a secondary 1. The 64-bit SysAD bus which carries the address and data. cache interface, and (3) the 447-pin R4000MC which comes 2. The 9-bit SysCmd bus which encodes the type of memory with a secondary cache interface and also supports multi- cycle. processing coherency. 3. The control lines to condition the SysCmd bus and control the issue rates of the commands. R4000 Clock Interface This article will discuss each of the System Interface One outstanding characteristic of the R4000 bus, in con- elements in detail. trast to most microprocessors, is that it uses fully synchronous timing. Thus, every output is generated relative to a clock R4000 SysAD Bus edge, and has the same propagation delay relative to the The SysAD(63:0) Bus is 64-bits wide and has 8 additional clock. Also, every input has the same setup and hold time optional ECC/parity bits called SysADC(7:0). The multiplexed relative to the clock. SysAD bus is shared between address and data phases. The This allows the simplification of worst case timing analysis, addresses will be present during the clock cycles where a valid so that hardware designers can concentrate on functional interface command is present on the SysCmd bus. Data will issues. In conjunction with the fully synchronous timing, the be present for the clock cycles where a valid data identifier is R4000 has a PLL, which allows it to match the input clock, present on the SysCmd bus. During the address phase, only MasterIn to the master (MasterOut), processor (PClock), the least significant 36-bits, SysAD(35:0) are used for a 64 GB system(SClock), and transmit clock (TClock). MasterOut is an physical address space. By convention, the upper 28 physical output clock which the PLL matches up to MasterIn. PClock is address bits, SysAD(63:36) are driven to 0 with appropriate an internal clock which runs at twice the frequency of the ECC/parity by the CPU. MasterIn clock. SClock is also an internal clock which is essentially equivalent to TClock and runs at the same fre- R4000 SysCmd Bus quency as the MasterIn clock. The PLL also allows the The SysCmd(8:0) bus is 9-bits wide and has 1 additional alteration of the slew rate of the outputs relative to the clock optional even parity bit called SysCmdP. The command bus and provides an extra receive clock that leads the system encodes the type of transaction that is present on the system clock by 25%, called RClock as can be seen in Figure 2. The interface. For instance, block reads, block writes, single word SyncIn and SyncOut pins shown in the Clock/Control Inter- reads, single byte writes, etc. are identified by the SysCmd face of Figure 3 automatically compensate the clocks for encoding. The MSB (Most Significant Bit), SysCmd(8), indi- external buffer delays. Finally, options exist which allow the cates whether the cycle is a system interface command or system, transmit, and receive clocks to be slowed down data identifier. Thus SysCmd(8) breaks the encodings into relative to the processor clock, such that the bus interface can two main cases, as listed in Tables 1, 2, and 3. Only the more run at 1/2, 1/3, or 1/4 of the normal speed. These options common encodings are listed here, although a complete list is provide flexibility in producing setup, hold, and access times available in the User’s Manual. Finally, some examples of the appropriate for various interfaces. more typical 9-bit commands and data identifiers are given in Table 4. MasterIn, MasterOut PClock (internal) SClock (internal), TClock RClock output lines TDO drw 02 Figure 2. R4000 Clock Interface Timing (PClock to SClock divisor of 2) 124 DESIGNING READ AND WRITE BUFFERS FOR THE R4000 SYSTEM INTERFACE APPLICATION NOTE AN-114 64 128 SysAD(63:0) SCData(127:0) 8 16 SysADC(7:0) SCDChk(15:0) 9 25 SysCmd(8:0) SCTag(24:0) 7 1 SysCmdP SCTChk(6:0) 17 ValidIn SCAddr(17:1) 4 ValidOut SCAddr0(w,x,y,z) 3 ExtRqst SCAPar(2:0) Release SCOE RdRdy 4 SCWr(w,x,y,z) Secondary Cache Interface WrRdy SCDCS IvdAck 3 R4000 SCTCS IvdErr 3 Logic Symbol 2 TClock(1:0) 2 Int(5:1)2 RClock(1:0) Int0 MasterClock NMI MasterOut Interrupt Interface SyncOut ModeClock SyncIn ModeIn IOOut VccOk IOIn ColdReset GrpRun Reset GrpStall Initialization Interface Fault JTDI Clock/Control InterfaceVccP System Interface JTDO VssP 8 JTMS Status(7:0)1 JTCK VccSense1 JTAG Interface 1 = R4000SC and R4000MC only 1 VssSense 2 = R4000PC only 3 = R4000MC only drw 03 Figure 3. The R4000 Interfaces R4000 System Interface Control Signals system is returning a data identifier on the SysCmd bus and The System Interface Control Signals communicate when read data on the SysAD bus, it will assert ValidIn. Two input the System Interface busses are valid, and if the external signals, RdRdy and WrRdy, are used by the memory system agent, (i.e., the memory), is ready to accept the command. to communicate whether or not it is ready to handle the next Their descriptions are given in Table 5. Two signals, the output read and write. The output signal Release is used by the CPU ValidOut ValidIn and the input are used by the CPU and the or bus master to indicate to the memory system that the memory to indicate when they are driving valid signals onto master is tri-stating the bus on the next clock. After Release the SysCmd and SysAD busses. For example, when the CPU asserts, the memory system can drive the SysAD read data is driving a valid command/address or write data on the and SysCmd data identifier back to the CPU. The input signal ValidOut SysCmd bus, it will assert , and when the memory ExtRqst is used by a DMA controller or interrupt controller to 125 DESIGNING READ AND WRITE BUFFERS FOR THE R4000 SYSTEM INTERFACE APPLICATION NOTE AN-114 gain control of the bus from the CPU. Finally, the inputs InvAck R3000/R4000 terminology, to use the term “buffer” in the and InvErr are used only on the R4000MC version to help software sense, meaning, a register location to store data manage cache coherency. rather than the hardware interpretation of amplifying or isolat- To illustrate the use of the System Interface, the following ing a signal without storing it.