Fully Buffered DIMM (FB-DIMM) Design Considerations
Total Page:16
File Type:pdf, Size:1020Kb
Fully Buffered DIMM (FB-DIMM) Design Considerations Howard David Memory Architect Michael McTague Staff Engineer Intel Corp. Feb 18, 2004 1 Agenda y FB-DIMM Architecture y FB-DIMM Channel Electrical Characteristics y FB-DIMM Physical Design y FB-DIMM System Design Considerations 2 Intel Platform Memory Technology Roadmap DDR3 DDR3 FB-DIMM DDR2 667/800 DDR2 DDR2 400/533 DDR DDR333/400 DDR266 RDRAM* PC1066 PC800 SDRAM PC133 2004 2005 20052006 y DDR2 400/533 support in all main IA segments in 2004 – with DDR flexibility y FB-DIMM new server interconnect in 2005 y Low Power SDRAM moving to low power DDR y RDRAM still used in specific applications *Other names and brands may be claimed as the property of others 3 FB-DIMM Architecture FB-DIMM Block Diagram y DRAM interface is entirely behind the buffer – DDR2 DRAM scales from 533 to 800 MT/s, with up to 8 DIMMs (288 devices) per channel Commodity DRAMDRAM DRAMDRAM DRAMDRAM DRAMs DRAMDRAM Differential Pairs DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM Up to 8 DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM 10 DIMMs Buffer Buffer Buffer ••• Buffer Memory 14 Controller DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM SMBus DRAMDRAM DRAMDRAM DRAMDRAM DRAMDRAM CLK Source 4 FB-DIMM Architecture FB-DIMM Module Block Diagram Pass-through Read Data & Merging Read Data (Primary Northbound) Logic (Secondary Northbound) Serializer AMB is the new Data Bus buffer Interface DRAM Data technology for 9 to 36 DRAM server memory De-serializer DRAM C/A Copy A & Decode Logic DRAM C/A Copy B Reference Clock Clocking DRAM Clocks Pass- Write Data and Command Write Data and Command through (Primary Southbound) (Secondary Southbound) Logic Advanced SPD Memory Buffer SMBus EEPROM (AMB) 5 FB-DIMM Channel Electrical Characteristics Signaling Characteristics y Differential point-to-point, 50 Ω termination y DC coupled, current mode driver y Derived clocking ( mesochronous clocking) y Transmitter de-emphasis to reduce ISI y 3.2 Gb/s, 4.0 Gb/s, and 4.8 Gb/s data rates Tx Rx De- emphasis Pre - Sync driver De-skew 50 Clock 50 Recovery PLL 50 50 PLL Reference CLK 6 FB-DIMM Channel Electrical Characteristics Differential Signaling Parameter Description Equation VDIFFp-p Differential Peak to 2*max[VD+ - VD-] Peak Voltage VCM-AC AC Common Mode (VD+ + VD-)/2 Voltage VCM-DC DC Common Mode AVG[(VD+ + VD-)/2] Voltage D- VD+ VCM-AC VD- D+ 7 FB-DIMM Channel Electrical Characteristics Tx Voltage Spec (Medium drive strength example) Parameter Description Min Max VTX-DIFF P-P-Min Differential Swing, 800 mv De-emphasis off VTX-DIFF P-P-Min, Differential swing, 505 mv 566 mv -3.5 Db De-emphasis on VTX-DIFF P-P-Min, Differential swing, 357 mv 450 mv -6.0 Db De-emphasis on VTX-DIFF P-P-Min, Differential swing, 239 mv 301 mv -9.0 Db De-emphasis on VTX-SE Single Ended Swing 700 mv 8 FB-DIMM Channel Electrical Characteristics Tx Timing Spec UI min spec D ata late Tx Total VTx diff min p-p min Data early Median to max Jitter Median Median Jitter Jitter UImin- Tx min- 2Txm edian to m ax jitter= 0 Transfer UI TTx-Total- TTX-Eye-Median-Max-Jitter Rate Nom min (ps) (Gb/s) (ps) (ps) 3.2 312.0 248.5 32.0 4.0 250.0 191.0 29.5 4.8 208.0 157.0 25.68 9 FB-DIMM Channel Electrical Characteristics Rx Spec D+,D- Crossing point D+,D- Crossing point 0.40 0.35 0.30 0.25 0.20 0.15 VRx-DIFFpp-Min 0.10 0.05 1 _ d2 n_ i _ c e 0.00 _r f f di _ d e e_r y -0.05 e -0.10 -0.15 -0.20 -0.25 TRx-Total-Min -0.30 -0.35 -0.40 0.0 40.0 80.0 120.0 160.0 200.0 240.0 280.0 312.5 time, psec •Measured •Simulated Data UI, Nom TRX-Total- TRX-EYE- VRX-DIFFp-p-MIN (mV) Rate (ps) Min (ps) Median-Max (Gb/s) Jitter (ps) 3.2 312.0 115 98.67 170 4.0 250.0 90 80.0 170 4.8 208.0 75 66.68 170 10 FB-DIMM Channel Electrical Characteristics Clock Spec y HCSL (High-Speed Current Steering Logic) clock y 0.7 v swing y 1/4th of the DDR2 DRAM frequency ( 133, 166, and 200 MHz) y Spread Spectrum Clock (SSC) with up to –0.5% down spread y Reference clock jitter specified to allow standard clock buffers. Z0d = 100 33 Ω Z = 100 Z = 100 < 0.2” 0d 0d < 0.2” 33 Ω < TBD” ~2.5” 0.5” DBxxx 50 Ω Clock 50 Ω AMB Driver FB-DIMM Connector 11 FB-DIMM Channel Electrical Characteristics Interconnect Modeling y Three (3) Topologies supported y Accurately models connection from Tx pin to Rx pin y Non-Interleaved routing with no bit to bit length matching required y Differential signaling design rules Tx Package Southbound Interconnect Network Rx Package (Memory Writes) + _ y Package Breakout +_ y Lossy Transmission Lines y Connectors y Vias y Sockets and test points Not Equivalent due to DIMM Routing and connector location 12 FB-DIMM Channel Electrical Characteristics FB-DIMM Topologies DIMMs on Motherboard FB-DIMM Memory Controller AMB Base Board FB-DIMM Connector 13 FB-DIMM Channel Electrical Characteristics FB-DIMM Topologies DIMMs on a Riser Card Memory Riser Board FB-DIMM Connector Memory Controller FB-DIMM Base Board AMB PCI Express* or pin and socket connector *Other names and brands may be claimed as the property of others. 14 FB-DIMM Channel Electrical Characteristics FB-DIMM Topologies DIMM to DIMM FB-DIMM FB-DIMM AMB AMB Base Board FB- DIMM connectors 15 FB-DIMM Channel Electrical Characteristics Interconnect Components y Package Models – Return Loss (<-10 Db) y PCB Model – Impedance, delay, and loss characteristics of traces – 3 pair, non-interleaved, cross talk model – VIA Models y Connector(s) – 3D Models validated using Measurements MC Vendor Model AMB Vendor Model W Element W Element +_ +_ +_ Package Network Package + Network _ +_ +_ 85 ohm Strip Line 6 x 50 Ohms 6 x 50 Ohms 85 ohm 85 ohm Strip Line FBD DDR2 Microstrip Breakout Vias Connector Model 16 FB-DIMM Channel Electrical Characteristics FB-DIMM Connector y Standard DDR2 Connector w/ FB-DIMM connector spec y Characterized up to 8 GHz y Q3D Spice Model based on FOXCONN* Connector Data • Less than 1 Db insertion loss. • Less than 3.2% Far end Cross talk • Less than 10 Db insertion loss *Other names and brands may be claimed as the property of others. 17 FB-DIMM Channel Electrical Characteristics Channel Design Rules y Channel impedance: 85 ohms , normal distribution y Microstrip trace geometry – w=6m, s= 4x h , tolerance = +/- 20%, etc. y Strip line trace geometry – w= 5 mil, s = 3x h, tolerance = +/-15%, etc y Asymmetric strip line geometry – w = 5 mil, s = 4x h , tolerance = +/-15%, etc, etc y Loss tangent 0.017 to 0.025, ε ~ 3.4 (FR4 materials) ef h2 s d εr2 t εr2 w h1 18 FB-DIMM Channel Electrical Characteristics Channel Validation Tx Package ...00110011... Tx Eye + 1. Tx Stand alone Test _ S pecification 2 x 50 ohms 2. Interconnect meets Rx Eye Spec Tx Package Worse Case Rx Eye ISI Pattern + Interconnect Network _ Specification 2 x 50 ohms 3. Rx Meets BER with Minimum Eye Tx Package Rx Package Worse Case Pattern Check ISI Pattern + Interconnect Network + _ _ or “On-die” Measurement FB-DIMM channel requires new memory design techniques 19 FB-DIMM Physical Design Mechanical Prototype 20 FB-DIMM Physical Design Physically Compatible Approach y FB-DIMM fits into existing system infrastructure 21 FB-DIMM Physical Design DIMM Mechanical Outline AMB on front Retention notches reduced to 2.5 mm Nominal height 8 DRAM on front, 10 DRAM on back New end notches Key moved 22 FB-DIMM Physical Design Advanced Memory Buffer (AMB) y New 0.8 mm ball pitch package – 24.5 mm x 19.5 mm x 2.15 mm – (29 columns x 23 rows) – 12 corner balls = 655 balls 1234567891011121314151617181920212223242526272829 TEST2 TEST4 A GND DQ26 DQ12 VDD DQS10 DQ13 VDD DQS01# DQ10 VDD VDD VDD VDD VDD DQ52 DQS15 VDD DQ49 DQS06# VDD DQ48 DQ38 VDD DDRCAL TEST3 TEST5 DDRCAL B VDD DQS03 DQS03# GND DQ14 DQS10# GND DQ11 DQS01 GND VDD GND VDD GND DQS15# DQ53 GND DQS06 DQ50 GND DQS13# DQS13 GND DDRCAL DDRCAL DDRCAL C GND DQS02 DQ18 GND DQ04 DQS09# GND DQ15 DQ09 GND DQ08 GND DQS17 DQS17# GND DQ54 GND DQ55 DQ51 GND DQS07 DQ56 GND DQ46 DQS14# VDD D DQ19 DQS02# GND DQ16 DQ24 GND DQS09 DQ07 GND DQ03 DQS00 GND DQS8# DQS8 VDD CB6 CB7 GND DQS16 DQ63 GND DQ59 DQS07# GND DQ36 DQ44 GND DQS14 DQ47 DRAM Mechanical E DQ21 GND DQ17 DQ29 GND DQ25 DQ06 GND DQ05 DQ01 GND DQ00 CB1 GND CB2 GND CB5 DQS16# GND DQ61 DQ57 GND DQ58 DQ39 GND DQ33 DQ45 GND DQ41 TEST0 TEST1 TEST6 TEST7 Data bus Balls F GND DQ20 DQ23 GND DQ31 DQ27 GND GND DQS00# DQ02 VDD CB0 CB3 CB4 VDD DQ62 DQ60 GND GND DQ37 DQ35 GND DQS05# DQ43 GND G DQS11# DQS11 mech mech mech GND DQS12 DQS12# mech mech mech TEST RFU3 RFU4 RFU5 TEST RFU6 RFU7 mech mech mech DQS04 DQS04# GND mech mech mech DQS05 DQ40 H DQ22 GND mech mech mech DQ28 DQ30 GND mech mech mech GND VDD GND VDD GND VDD GND mech mech mech GND DQ34 DQ32 mech mech mech GND DQ42 J GND CLK2 mech mech mech BA1A GND CKE1A mech mech mech VDD GND VDD GND VDD GND VDD mech mech mech RAS#B GND RFU1 mech mech mech CLK3# GND DRAM K CLK2# CLK0 mech mech mech GND WE#A RAS#A mech mech mech GND VCC GND VCC GND VCC GND mech mech mech ODT0B CS1#B GND mech mech mech CLK1# CLK3 C/A/Clk L CLK0# GND mech mech mech A0A CKE0A GND mech mech mech VCC GND VCC GND VCC GND VCC mech mech mech GND CAS#B WE#B mech mech mech GND CLK1 DRAM M ODT0A RFU0 mech mech mech CAS#A GND BA2A mech mech mech GND VCC GND VCC GND VCC GND mech mech mech CS0#B GND BA1B mech mech mech CKE0B GND copy “A” N CS1#A CS0#A mech mech mech GND BA0A A10A mech mech mech VCC GND VCC GND VCC GND VCC mech mech mech A0B A2B GND mech mech mech BA0B BA2B C/A/Clk P A6A GND mech mech mech A2A A1A A3A mech mech mech GND VCC GND VCC GND VCC GND mech mech mech GND A4B A1B mech mech mech