Opensparc T1 on Xilinx Fpgas – Updates Thomas Thatcher Paul Hartke [email protected] [email protected] Opensparc Engineering Xilinx University Program
Total Page:16
File Type:pdf, Size:1020Kb
OpenSPARC T1 on Xilinx FPGAs – Updates Thomas Thatcher Paul Hartke [email protected] [email protected] OpenSPARC Engineering Xilinx University Program RAMP Retreat – August 2008, Stanford Agenda • Quick OpenSPARC Overview • Progress timeline • Current Status > OpenSPARC T1 1.6 Release > OpenSPARC Book > OpenSPARC FPGA Board > Multi-core T1 design > T1 core on BEE3 • Roadmap • Q & A 2 www.opensparc.net RAMP Retreat-Aug 2008 What is OpenSPARC? • Open-Sourced versions of Sun's Microprocessor Products > RTL, Verification Env,documentation, system software > Available for download at www.opensparc.net • Two Processors Available > OpenSPARC T1 > 8 cores, 4 hardware threads per core > 1 floating-point unit external to core, shared by all cores > 4 banks of L2 cache > OpenSPARC T2 > 8 cores, 8 hardware threads per core > Floating-point internal to core, one per core > 8 banks of L2 cache 3 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC T1 • SPARC V9 implementation • Eight cores, four threads each – 32 simultaneous threads • All cores connect through a 134.4 GB/s crossbar switch • High BW 12-way associative 3 MB on-chip L2 cache • 4 DDR2 channels (23 GB/s) • 70W power • ~300M transistors 4 www.opensparc.net RAMP Retreat-Aug 2008 Sun/Xilinx Partnership: Big Goals • Proliferation of OpenSPARC technology • Proliferation of Xilinx FPGA technology • Make OpenSPARC FPGA friendly > Create reference design with complete system functionality > Boot Solaris/Linux on the reference design > Open it up > Seed ideas in the community Enable multi-core research 5 www.opensparc.net RAMP Retreat-Aug 2008 Timeline July 06 Jan 07 June 07 Jan 08 Aug 08 OpenSPARC T1 Sun/Xilinx OpenSpARC Stand-alone OpenSolaris on Collaboration T1 on ML411 program under ML411 board Begins board hypervisor First ML505 Support Today 6 www.opensparc.net RAMP Retreat-Aug 2008 New Developments • OpenSPARC T1 1.6 Release • OpenSPARC Book • New OpenSPARC Development Kit > ML505 board with XC5VLX110T FPGA • Multi-core Design • OpenSPARC T1 core running on BEE3 Board 7 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC T1 1.6 Release • Released May, 2008 • Implementation of 4-thread T1 core on Virtex 5 FPGAs > ML505-V5LX110T board > EDK Project files (for EDK 9.2) > Scripts to run complete RTL regression on hardware • Complete setup to boot Solaris > Networking support, including telnet and ftp • Quick start ace files included > Creates an out-of-the-box experience > T1 core boots OpenSolaris in 30 minutes 8 www.opensparc.net RAMP Retreat-Aug 2008 Hardware Block Diagram MultiPort Xilinx Embedded Developer’s FPGA Boundary Memory Controller External DDR2 Dimm (EDK) Design Cache-processor interface (CPX) MCH-OPB MemCon CCX-FSL Microblaze Proc Microblaze Debug UART SPARC T1 Core Interface SPARC T1 UART processor- Fast Simplex 10/100 Ethernet cache interface Links interface (PCX) (FSL) IBM Coreconnect Developed and OPB Bus Working 9 www.opensparc.net RAMP Retreat-Aug 2008 Software Setup • OpenSolaris is booted from a RAM disk Image • Memory Allocation: > 1 MB used by Microblaze firmware > 1 MB used for OpenSPARC Boot PROM image > 80 MB for RAM disk image > Leaving 174 MB for OpenSPARC RAM • Microblaze firmware does address translation to map SPARC addresses to board addresses. 10 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC Development Kit • A kit for OpenSPARC development now available > Board based on the ML505, but with an XC5VLX110T FPGA > Includes USB interface for FPGA programming > Tested with OpenSPARC T1 release 1.6 release design > Eliminates the need to buy a board and then upgrade the FPGA • Shipping now! • Kit Includes: > Board, with power supply and 256 MB DRAM > Platform USB download cable > Host to host SATA crossover cable > Compact flash card with OpenSPARC T1 1.6 ace files 11 www.opensparc.net RAMP Retreat-Aug 2008 Kit Contents 12 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC Kit Donation Program • Sun will donate OpenSPARC Development Kits to qualified universities > Web address below: • See the web page for more details • Also available from directly from Digilent > http://www.digilentinc.com http://www.opensparc.net/edu/university-program.html 13 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC Internals Book • Covers both OpenSPARC T1 and T2 • Includes > Architectural Overview > Development environments for OpenSPARC > Source (RTL) code overview > Configuring, extending, and verifying OpenSPARC > Porting operating systems to OpenSPARC • 350 Pages • Available in both hardcopy (Amazon.com) and PDF format • Sign-up sheet for early release PDF copy (by poster) 14 www.opensparc.net RAMP Retreat-Aug 2008 Implementing a Multi-core design • We have created a multi-core system by interconnecting two boards. > Opens the door to multi-core designs on BEE3 board • Uses Xilinx Aurora link-layer protocol running over RocketIO™ GTP serial tranceivers > Connected through the SATA connectors on the board • Each GTP channel is 16 bits at 75 Mhz > Connected to Microblaze through an FSL FIFO 15 www.opensparc.net RAMP Retreat-Aug 2008 Multi-core System Block Diagram Xilinx Embedded Xilinx CacheLink (XCL) FPGA Boundary Developer’s External DDR2 Dimm (EDK) Design Fast Simplex Links (FSL) MemCon CCX-FSL Microblaze Proc Microblaze Debug UART SPARC T1 Core Interface SPARC T1 UART Aurora over GTP Ethernet Developed and FSL connected Aurora-over-GTP Working module to connect to other board New 16 www.opensparc.net RAMP Retreat-Aug 2008 Dual-core system implementation SATA Cable Master Node 17 www.opensparc.net RAMP Retreat-Aug 2008 Four-core system implementation SATA Cable SMA cables SATA Cable Master Node 18 www.opensparc.net RAMP Retreat-Aug 2008 Initial Configuration • Master FPGA hosts entire OpenSPARC Address space. > However, Each client MicroBlaze will run firmware code out of its own memory • Both boards have the same bit file > Avoids need to develop and implement separate bit files > CPU ID set by DIP switches on the board • However, software will be different for each board > Master software: services all memory requests > Slave software: only routes memory requests to the other board 19 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC on the BEE3 • BEE3 board uses Virtex 5 FPGAs (same family as ML505) • Re-implemented Release 1.6 design on BEE3 > Very easy to re-target design. > Updated design to EDK 10.1 > Re-implemented on both XC5VLX110T and XC5VLX155T > Generated ace files for BEE3 board > Verified OpenSolaris Boot • Seamless and trouble-free porting experience • Validated BEE3 board infrastructure • Stop by to see our demo! 20 www.opensparc.net RAMP Retreat-Aug 2008 OpenSPARC on the Bee3 Details • Single 4-thread OpenSPARC core – 62.5MHz OpenSPARC; 125MHz Microblaze MPMC/MIG – 6-LUTs: 59,350 / 97,280 61% – 36kbit BRAM: 147 / 212 69% – Ethernet in design but not tested Microblaze – 1.5 hour implementation time • EDK project based on Bee3 EDK reference design – Uses EDK MPMC4 and MIG 4-thread OpenSPARC core 21 www.opensparc.net RAMP Retreat-Aug 2008 Bee3 OpenSPARC Four-Core System Diagram (Master-node Configuration) 305.00 105.00 105.00 QSH “cross-over” cable 25.00 25.00 Fujitsu 2x2 15.00 Fujitsu 2x2 30.00 RJ45 CX4 CX4 RJ45 35.00 20.00 40.00 P P P P J T DDR2 DIMM0 DDR2 DIMM0 C C C C A I I I I - - - - G E E E E 65.00 60.00 x x x x p p p p DDR2 DIMM1 DDR2 DIMM1 r r r r e e e e Ring Wiring s s s s s s s s 8 8 8 8 5 DDR2 DIMM2 DDR2 DIMM2 x x x x 0 p 4 4 4 4 i n G G G G M M M M 2 DDR2 DIMM3 DDR2 DIMM3 CCX link m A A A A B B B B m R R R R H D D D D D D D D e D D D D a 1 1 3 6 F 7 7 F 7 7 d R R R R e 6 6 6 6 r V L X T 2 2 2 2 5 5VLXT 100.00 6 6 6 6 - - - - 133 133 133 133 - - - - 6 6 6 6 2 2 2 2 6 6 FF1136 6 6 R R R R 7 7 7 7 D D D D D D D D D D D D R R R R 4 1 - 2 p B B B B V A A A A i CX4 CX4 n M M M M G G G G 21.00 QSH-DP- User1 User2 QSH-DP- 4 4 4 4 8 1 - 40x2 2 72* 40x2 p V i 040 5VLXT 5VLXT 040 n 180.00 29.00 CX4 CX4 PCI-E PCI-E 2 4 2 Ring Wiring 8X 8X 380.00 p . i 5 n V V V V V V V A 0 8 8 0 8 8 T . X 72* 72* 1 1 1 1 1 1 CCX link P W R PCI-E PCI-E Q CX4 8X 8X CX4 0 S 4 H 0 Ring Wiring - - 4 4 4 4 D P P QSH-DP- User3 User4 QSH-DP- D G G G G M M M M - - 0 A A A A H 40x2 72* B B B B 40x2 4 78.00 S 0 R R R R D D D D 040 5VLXT 5VLXT 040 Q CCX link D D D D D D D D 1 1 3 6 F 7 7 F 7 7 R R R R 6 6 6 6 V L X T 2 2 2 2 5 CX4 CX4 6 6 6 6 - - - - - - - - 6 6 6 6 5VLXT 2 2 2 2 180.00 6 6 6 6 R R R R 7 7 FF1136 7 7 150.00 D D D D D D D D D D D D R R R R B B B B 133 133 133 133 A A A A M M M M G G G G Q 4 4 4 4 0 S 4 H 0 - - D P P D DDR2 DIMM0 DDR2 DIMM0 - - 0 H 4 S 0 Master Node DDR2 DIMM1 DDR2 DIMM1 Q DDR2 DIMM2 DDR2 DIMM2 DDR2 DIMM3 DDR2 DIMM3 40.00 10.00 18.00 23.00 102.00 70.00 107.00 Master Node Ring Wiring QSH “cross-over” cable CCX link 22 www.opensparc.net RAMP Retreat-Aug 2008 Bee3 Multi-core Node Block Diagram Xilinx Embedded Xilinx CacheLink (XCL) FPGA Boundary Developer’s External DDR2 Dimm (EDK) Design Fast Simplex Links (FSL) MemCon CCX-FSL Microblaze Proc Microblaze Debug UART SPARC T1 Core Interface SPARC T1 UART RinRingg Ring WiWriniringg Wiring Ethernet Developed and FSL interconnect to other Working FPGA(s) New Maintain compatability between ml505_v5lx110t and Bee3 designs 23 www.opensparc.net RAMP Retreat-Aug 2008 Bee3 OpenSPARC Four-Core System Diagram (Full Mesh interconnect) 305.00 105.00 105.00 QSH “cross-over” board 25.00 25.00 Fujitsu 2x2 15.00 Fujitsu 2x2 30.00 RJ45 CX4 CX4 RJ45 35.00 20.00 40.00 P P P P J T DDR2 DIMM0 DDR2 DIMM0 C C C C A I I I I - - - - G E E E E 65.00 60.00 x x x x p p p p DDR2 DIMM1 DDR2 DIMM1 r r r r e e e e Ring Wiring s s s s s s s s 8 8 8 8 5 DDR2 DIMM2 DDR2 DIMM2 x x x x 0 p 4 4 4 4 i n G G G G M M M M 2 DDR2 DIMM3 DDR2 DIMM3 CCX link m A A A A B B B B m R R R R H D D D D D D D D e D D D D a 1 1 3 6 F 7 7 F 7 7 d R R R R e 6 6 6 6 r V L X T 2 2 2 2 5 5VLXT 100.00 6 6 6 6 - - - - 133 133 133 133 - - - - 6 6 6 6 2 2 2 2 6 6 FF1136 6 6 R R R R 7 7 7 7 D D D D D D D D D D D D R R R R 4 1 - 2 p B B B B V A A A A i CX4 CX4 n M M M M G G G G 21.00 QSH-DP- User1 User2 QSH-DP- 4 4 4 4 8 1 - 40x2 2 72* 40x2 p V i 040 5VLXT 5VLXT 040 n 180.00 29.00 CX4 CX4 PCI-E PCI-E 2 4 2 Ring Wiring 8X 8X 380.00 p .