USOO6331856B1 (12) United States Patent (10) Patent No.: US 6,331,856 B1 Van Hook et al. (45) Date of Patent: *Dec. 18, 2001 (54) VIDEO GAME SYSTEM WITH (56) References Cited COPROCESSOR PROVIDING HIGH SPEED EFFICIENT 3D GRAPHICS AND DIGITAL U.S. PATENT DOCUMENTS AUDIO SIGNAL PROCESSING Re. 31,200 4/1983 Sukonick et al. .................... 395/520 4,789,927 12/1988 Hannah ................................ 395/800 (75) Inventors: Timothy J. Van Hook, Menlo Park; Howard H. Cheng, San Francisco; (List continued on next page.) Anthony P. DeLaurier, Sunnyvale; FOREIGN PATENT DOCUMENTS Carroll P. Gossett, Mountain View; Robert J. Moore, Sunnyvale; Stephen O 632 407 A1 1/1995 (EP). J. Shepard, Cupertino; Harold S. O 633 533 A2 1/1995 (EP). Anderson, Morgan Hill; John Princen, (List continued on next page.) Sunnyvale; Jeffrey C. Doughty, Palo Alto; Nathan F. Pooley, Mountain OTHER PUBLICATIONS View; Byron Sheppard, Santa Cruz, all of CA (US); Genyo Takeda, Hirakata; Johnson, Matthew, A Fixed-Point DSP for Graphics Shuhei Kato, Otsu, both of (JP) Engines, Aug. 1989, IEEE, Los Alamitos, CA, pp. 63-77. Wayner, Peter, Silicon for 3-D, Sep., 1994, Byte, Peterbor (73) Assignees: Nintendo Co., Ltd., Osaka (JP); Silicon ough, NH, pp. 191-192. (US)Graphics, Inc., Mountain View, CA (ListLi continuedinued on next page.) Primary Examiner Kee M. Tung (*) Notice: This patent issued on a continued pros- (74) Attorney, Agent, or Firm Nixon & Vanderhye P.C. ecution application filed under 37 CFR 1.53(d), and is subject to the twenty year (57) ABSTRACT F.)(5 rm provisions of 35 U.S.C. A low cost high performance three dimensional (3D) graph ics System can model a world in three dimensions and Subject to any disclaimer, the term of this project the model onto a two dimensional viewing plane patent is extended or adjuste d under 35 Selected based on a changeable viewpoint. The viewpoint U.S.C. 154(b) by 0 days. can be changed on an interactive, real time basis by oper ating user input controls Such as game controllers. The System rapidly produces a corresponding changing image (21) Appl. No.: 08/561,718 (which can include animated cartoon characters or other (22) Filed: Nov. 22, 1995 animation for example) on the Screen of a color television Set. The richly featured high performance low cost System (51) Int. Cl." .................................................. G06F 15/16 gives consumers the chance to interact in real time inside (52) U.S. Cl. ........................... 345/503; 34.5/512; 34.5/520 magnificent Virtual 3D Worlds to provide a high degree of (58) Field of Search ..................................... 395/501-503, image realism, excitement and flexibility. An optimum fea 395/506, 507,512, 522, 2.09, 806, 807, ture set/architecture (including a custom designed graphics/ 122, 119, 125-127, 129-134, 140, 141; audio coprocessor) provides high quality fast moving 3D 345/115-116, 121, 122, 133, 141, 185, images and digital Stereo Sound for Video game play and 189, 501-503, 506, 507, 512, 522, 422, other graphics applications. 421, 400, 441, 419, 425-427, 429–434, 520, 521: 463/43, 44 2 Claims, 56 Drawing Sheets WDEO GAME 54 OVERAVIDEO GAME SYSTEM STORAGEDEVICE 300 CoPROCESSOR MAIN 40 MAN PROCESSOR SIGNAL FROCESSOR MEMORY SPLAY PROCESSOR VIDEO&AWEIO (LIR) MAN UNIT GAME CONROLLERS US 6,331,856 B1 Page 2 U.S. PATENT DOCUMENTS “The i7500R Video Processor: A Total Multimedia Solution', 4,799,635 1/1989 Nakagawa ............................ 395/442 K. Harney et al, Communications of the ACM, vol. 34, No. 4,823,120 4/1989 Thompson et al. ... 34.5/509 4, 4/91, pp. 64-78.* 4,824,106 4/1989 Ueda ...................................... 463/33 Computer Graphics Proceedings, Annual Conf. Series, 4,825,391 * 4/1989 Merz ...... ... 395/131 1993, Aug. 1–6, 1993. “LEO: A System for Cost Effective 4,951,232 8/1990 Hannah .......... ... 395/122 3D Shaded Graphics' Deering et al., pp. 101-108.* 4,970,636 11/1990 Snodgrass et al. .................. 345/434 IEEE CGSA, “A New VLSI Graphics Coprocessor The 5,038,297 8/1991 Hannah ................................ 395/122 Intel 82786” Glen Shires, 10/86, pp. 49-55.* 5,051,737 9/1991 Akeley et al. ... 395/134 Open GL Programming Guide, “The Official Guide to 5,070,479 12/1991 Nakagawa . 395/182.09 Learning OpenGL, Release 1,” OpenGL Architecture 5,113,490 5/1992 Winget ....... ... 395/119 5,193,145 3/1993 Akeley ................................. 395/123 Review Board, Jackie Neider, Tom Davis, Mason Woo, 5,227,863 7/1993 Bilbrey et al. ....................... 345/302 Copyright 1993 by Silicon Graphics, Inc. 5,230,039 7/1993 Grossman et al. ... 345/430 Open GL Reference Manual, “The Official Reference Docu 5,265,199 11/1993 Catlin ............. ... 395/122 ment for OpenGL, Release 1,” OpenGL Architecture 5,266,941 11/1993 Akeley et al. ... 34.5/201 REview Board, Copyright 1992 by Silicon Graphics, Inc. 5,291,189 3/1994 Otake et al. ... ... 34.5/201 MIPS Microprocessor R4000 User's Manual, First Edition, 5,307.450 4/1994 Grossman .. ... 395/123 by Joe Heinrich, Copyright 1993 by MIPS Technologies, 5,343,558 8/1994 Akeley ....... ... 395/126 Inc. 5,347,618 9/1994 Akeley ....... ... 395/121 MIPS Microprocessor R4000 User's Manual, Second Edi 5,357,604 10/1994 San et al. ............................... 463/31 5,369,739 11/1994 Akeley ....... ... 395/134 tion, by Joe Heinrich, Copyright 1994 by MIPS Technolo 5,388,841 2/1995 San et al. ............................... 463/44 gies, Inc. 5,394,170 2/1995 Akeley et al. ... 34.5/201 MIPS RISC Architecture, “Introducing the R4000 Technol 5,469,535 * 11/1995 Jarvis et al. ... ... 34.5/503 ogy,” by Gerry Kane and Joe Heinrich, Copyright 1992 by 5,491,498 2/1996 Koyama et al. ... 34.5/189 MIPS Technologies, Inc. 5,541,923 7/1996 Kato ... ... 395/427 MIPS Open RISC Technology, “R4000 Microprocessor 5,553,864 9/1996 Sitrick .................................... 463/31 Product Information,” by Satya Simha, MIPS Technologies, 5,561,746 * 10/1996 Murata et al. ... 345/419 Inc., Sep. 27, 1993. 5,701,444 12/1997 Baldwin ......... ... 34.5/506 INDY Product Guide, INDY-TMG-(09/93),Copyright 1993 5,732,224 * 3/1998 Gulicket al. ...... ... 345/302 by Silicon Graphics, Inc. 5,768,393 * 6/1998 Mukojima et al. .................... 381/17 OpenGL, It's Everywhere, Information Sheet, OPGL-BRO 5,797.028 * 8/1998 Gulick et al. .......................... 712/32 (07/93), Copyright 1993 by Silicon Graphics, Inc. FOREIGN PATENT DOCUMENTS Reality Engine/Reality Engine', Graphics Subsystems, Data O 649 118A2 4/1995 (EP). Sheet, Copyright 1993 by Silicon Graphics, Inc. O 676 719 A2 10/1995 (EP). INDY Technical Report, INDY-TR (06/93) Copyright 1993 O 676 726 A2 10/1995 (EP). by Silicon Graphics, Inc. O 627 699 A2 11/1995 (EP). Reality Engine in Visual Simulation Technical Overview, O 681 267 A2 11/1995 (EP). RE-Vissim-TR(8/92), Copyright 1992 by Silicon Graphics, 0 685 246 12/1995 (EP) ................................. A63F/9/22 O 268 419 5/1988 (EP). Inc. O 724 220 7/1996 (EP). Rambus Architectural Overview, DL0001-02, Copyright O 553 532 8/1993 (EP) ............................... G06F/15/72 1992, 1993 by Rambus Inc. 4-106594 8/1992 (JP). SH7600 Series Super H RISC Engine, Overview, Hitachi WO94/10685 5/1994 (WO). manual, Oct. 17, 1994. WO94/27205 11/1994 (WO). This is What It's Like to Give Your Next Product a 32-Bit WO 94/10641 5/1994 (WO) ............................. GO6F/15/62 RISC Controller, Hitachi America, Ltd. brochure, 1994. OTHER PUBLICATIONS SEGA Genesis Instruction Manual, SEGA, Hayward, Cali Wilson, Chip Vendors May Exterminate Frame Buffers, EE fornia, #3701-926–0–01 (1994). Times-News 1995, Mar. 27, 1995. SEGA Genesis 32X Instruction Manual, SEGA, Redwood Jung, Atari Jaguar Frequently Asked Questions, pp. 1-25, City California, #672–2116 (1994). 1996 (www.gue-tech.org/darkgrue/attic/atari/jaguar/jag Knuckles CHAOTLX Instruction Manual, SEGA, Redwood uarfaq.txt). City, California, #84503 (1995). Nat! and KKP, Jaguar, pp. 1-6, 1996 (www.gue-tech.org/ 3D BALLZ Instruction Booklet, Accolade, San Jose, Cali darkgrue/attic/atari/jaguar/jagdox/general.html). fornia, #3050-00231 Rev. A. Super Nintendo Entertainment System Vortex game car Sonic 2 The Hedgehog Instruction Manual, SEGA, Hay tridge for the SNES. ward, California, #672–0944 3701–925-0-01 (1992). “Battle Zone/Cabaret', Atari Inc. (1980), pp. 12–13 & VHS PR Newswire, Sony Enters the CD-ROM-Based Video Videotape. Game, New York, Ny, May 31, 1991. Perry et al., associate editors, “Special Report, Consumer Sony PlayStation Instruction Manual, and informational Electronics, Video games: the electronic big bang, IEEE materials, Sony Computer Entertainment Inc. 1995. Spectrum, pp. 20-32, Dec. 1982. 6 Photographs of Sony PlayStation: 1) top case and compact Johnstone, “Keeping Nintendo Competitive,” Wired, Jan. 1994, pp. 76–77. disk; 2) hand controller; 3) internal circuitboards (top view); “The Creation of Battlezone' from Microsoft Arcade 4) internal circuit boards (top view); 5) compact disk reader (Undated). (bottom view); and internal main circuit board (bottom “Battlezone by Atari Coin Op-True Facts” & “Army view). Battlezone” (memoranda). * cited by examiner U.S. Patent Dec. 18, 2001 Sheet 1 of 56 US 6,331,856 B1 U.S. Patent Dec. 18, 2001 Sheet 2 of 56 US 6,331,856 B1 U.S. Patent Dec. 18, 2001 Sheet 3 of 56 US 6,331,856 B1 U.S. Patent Dec. 18, 2001 Sheet 4 of 56 US 6,331,856 B1 U.S. Patent Dec. 18, 2001 Sheet 5 of 56 US 6,331,856 B1 008 HOSSEIO08’d00 01NILESA1HINW9SHETT0H1N00 U.S. Patent Dec. 18, 2001 Sheet 7 of 56 US 6,331,856 B1 ‘007MOSSIDOMJOO00Z005 MOSSÈDO?dNOSS3É) 801 U.S.