Case: 12-1252 Document: 46 Page: 1 Filed: 08/29/2012 NON-CONFIDENTIAL No. 2012-1252 United States Court Of Appeals For The Federal Circuit
MOTIVA, LLC, Appellant,
v.
INTERNATIONAL TRADE COMMISSION, Appellee,
and
NINTENDO CO., LTD. AND NINTENDO OF AMERICA INC., Intervenors. ______
ON APPEAL FROM THE UNITED STATES INTERNATIONAL TRADE COMMISSION IN INVESTIGATION NO. 337-TA-743 ______
ANSWERING BRIEF OF INTERVENORS NINTENDO CO., LTD. AND NINTENDO OF AMERICA INC. ______
Joseph S. Presta E. Joshua Rosenkranz Robert W. Faris Peter A. Bicks NIXON & VANDERHYE P.C. Alex V. Chachkes 901 North Glebe Road ORRICK, HERRINGTON & SUTCLIFFE LLP 11th Floor 51 West 52nd Street Arlington, VA 22203 New York, NY 10019 (703) 816-4000 (212) 506-5000
Mark S. Davies Katherine M. Kopp ORRICK, HERRINGTON & SUTCLIFFE LLP 1152 15th Street, N.W. Washington, D.C. 20005 (202) 339-8400 Case: 12-1252 Document: 46 Page: 2 Filed: 08/29/2012
CERTIFICATE OF INTEREST
Counsel for Intervenors Nintendo Co., Ltd. and Nintendo of
America Inc. certify the following:
1. The full name of the parties represented by us in this case are:
Nintendo Co., Ltd. and Nintendo of America Inc.
2. The name of the real parties in interest (if the party named in the caption is not the real party in interest) represented by us are:
Nintendo Co., Ltd. and Nintendo of America Inc.
3. The parent companies, subsidiaries (except wholly owned subsidiaries), and affiliates that have issued shares to the public of the parties represented by us are:
Nintendo Co., Ltd., whose stock is publicly traded in Japan, owns
100% of Nintendo of America Inc.
4. The names of all law firms and the partners or associates that appeared for the parties now represented by us in the agency or are expected to appear in this Court, are:
i Case: 12-1252 Document: 46 Page: 3 Filed: 08/29/2012
Peter A. Bicks Joseph S. Presta E. Joshua Rosenkranz Robert W. Faris Mark S. Davies NIXON & VANDERHYE P.C. Alex V. Chachkes 901 North Glebe Road Elyse D. Echtman 11th Floor Sarah E. Walcavich Arlington, VA 22203 Richard A. Rinkema (703) 816-4000 Jordan L. Coyle Nicholas H. Lam Lauren B. Muldoon Cyrus P.W. Rieck Katherine M. Kopp Steven E. Adkins (no longer with firm) ORRICK, HERRINGTON & SUTCLIFFE LLP 1152 15th Street, NW Washington, D.C. 20005 (202) 339-8400
5. No other appeal involving this civil action was previously before this or any other appellate court. There are no pending cases known to counsel that would directly affect or be directly affected by this Court’s decision in the pending appeal.
Date: August 29, 2012 /s/ Mark S. Davies Mark S. Davies
Attorney for Nintendo Co., Ltd. and Nintendo of America Inc./Intervenors
ii Case: 12-1252 Document: 46 Page: 4 Filed: 08/29/2012
TABLE OF CONTENTS
Page
CERTIFICATE OF INTEREST ...... i
TABLE OF AUTHORITIES...... vi
STATEMENT OF RELATED CASES ...... ix
INTRODUCTION...... 1
JURISDICTIONAL STATEMENT...... 3
STATEMENT OF THE ISSUES...... 3
STATEMENT OF THE CASE ...... 3
STATEMENT OF THE FACTS ...... 5
A. Motiva’s Research And Marketing Efforts: The Start And Finish...... 5
B. Motiva’s Patents...... 9
C. Motiva’s Patent Litigation...... 16
1. Motiva Files An ITC Action Against Nintendo ...... 16
2. The ALJ Grants Nintendo’s Motion For Summary Determination...... 18
3. The ITC Reverses And Remands With Questions ..... 19
4. The ALJ Holds A Hearing, Answers The ITC’s Questions, And Finds No Violation ...... 20
5. The ITC Affirms ...... 30
SUMMARY OF ARGUMENT...... 30
STANDARD OF REVIEW...... 33
ARGUMENT ...... 34 iii Case: 12-1252 Document: 46 Page: 5 Filed: 08/29/2012
TABLE OF CONTENTS (continued) Page
I. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’S FINDING THAT MOTIVA WAS NOT IN THE PROCESS OF ESTABLISHING ANY DOMESTIC INDUSTRY ...... 34
A. Motiva’s Stale Licensing Efforts Do Not Demonstrate Any Tangible Steps Toward Establishing A Domestic Industry...... 35
B. Motiva’s Lawsuit Against Nintendo Is Not A Tangible Step Toward Establishing A Domestic Industry...... 41
C. The Wii System’s Success Does Not Suggest Motiva Has Any Chance Of Establishing A Domestic Industry In The Future...... 46
II. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’S FINDING THAT THE WII SYSTEM DOES NOT INFRINGE MOTIVA’S PATENTS ...... 51
A. Substantial Evidence Shows That The Wii System Does Not “Track” Movement Of A User ...... 51
B. Substantial Evidence Shows That The Wii System Does Not Determine User “Movement”...... 54
1. Three Credible Witnesses Demonstrated That The Wii Sensors Cannot And Do Not Determine Position Or Orientation ...... 55
2. Motiva’s Hodgepodge Of Movement And Position Information Arguments Are All Meritless ...... 59
CONCLUSION ...... 75
iv Case: 12-1252 Document: 46 Page: 6 Filed: 08/29/2012
TABLE OF CONTENTS (continued)
Material has been deleted from pages 1, 2, 6, 22, 26, 29, 43, 46-47, 51-
53, 55-64, 68-70, 72 and 73 of the Non-Confidential Answering Brief of
Intervenors Nintendo Co. Ltd. and Nintendo of America Inc. This material is deemed confidential business information pursuant to 19
U.S.C. § 1337(n) and 19 C.F.R. § 210.5, and pursuant to the Protective
Order entered November 2, 2010. The material omitted from these pages contains confidential deposition and hearing testimony, confidential business information, confidential patent application information, and confidential licensing information.
v Case: 12-1252 Document: 46 Page: 7 Filed: 08/29/2012
TABLE OF AUTHORITIES
Page(s) FEDERAL CASES
Am. Silicon Techs. v. United States, 261 F.3d 1371 (Fed. Cir. 2001) ...... 50
Bally/Midway Mfg. Co. v. ITC, 714 F.2d 1117 (Fed. Cir. 1983) ...... 38, 48
Beneficial Innovations, Inc. v. Blockdot, Inc., Nos. 2:07-CV-263-TJW-CE, 2:07-CV-555-TJW-CE, 2010 U.S. Dist. LEXIS 54151 (E.D. Tex. June 3, 2010)...... 74
Erbe Elektromedizin GmbH v. ITC, 566 F.3d 1028 (Fed. Cir. 2009) ...... 33
Finnigan Corp. v. ITC, 180 F.3d 1354 (Fed. Cir. 1999) ...... 60
In re Katz Interactive Call Processing Patent Litig., 07-ml-01816-BRGK, 2008 WL 4952454 (C.D. Cal. Feb. 21, 2008)...... 74
In re Nintendo Co., Ltd., 589 F.3d 1194 (Fed. Cir. 2009) ...... 16
InterDigital Commc’ns, LLC v. ITC, No. 2010-1093, 2012 U.S. App. LEXIS 15893 (Fed. Cir. Aug. 1, 2012)...... 39, 41
John Mezzalingua Assocs. v. ITC, 660 F.3d 1322 (Fed. Cir. 2011) ...... 32, 33,34, 40
Ninestar Tech. Co. v. ITC, 667 F.3d 1373 (Fed. Cir. 2012) ...... 33
Nippon Steel Corp. v. United States, 458 F.3d 1345 (Fed. Cir. 2006) ...... 50
vi Case: 12-1252 Document: 46 Page: 8 Filed: 08/29/2012
Northrop Grumman Corp. v. Intel Corp., 325 F.3d 1346 (Fed. Cir. 2003) ...... 66
Retractable Technologies v. Becton, Dickinson & Co., 653 F.3d 1296 (Fed. Cir. 2011) ...... 65, 66, 67
SRAM Corp. v. AD-II Eng’g, Inc., 465 F.3d 1351 (Fed. Cir. 2006) ...... 74
St. Clair Intellectual Prop. Consultants, Inc. v. Canon Inc., 412 F. App’x 270 (Fed. Cir. 2011)...... 74
Tessera, Inc. v. ITC, 646 F.3d 1357 (Fed. Cir. 2011) ...... 34, 71, 72
TianRui Group Co. v. ITC, 661 F.3d 1322 (Fed. Cir. 2011) ...... 33
Vita-Mix Corp. v. Basic Holding, 581 F.3d 1317 (Fed. Cir. 2009) ...... 70, 71
ADMINISTRATIVE CASES
Certain Coaxial Cable Connectors and Components Thereof and Prods. Containing Same, Inv. No. 337-TA-650, USITC Pub. No. 4283 (Nov. 11, 2010)...... 41
Certain Digital Satellite Sys. Receivers and Components Thereof, Inv. No. 337-TA-392, USITC Pub. No. 3418 (Apr. 2011)...... 45, 46
Certain Multimedia Display and Navigation Devices and Sys., Components Thereof, and Prods. Containing the Same, Inv. No. 337-TA-694, USITC Pub. No. 4292 (Nov. 2011)...... 44, 45
Certain Stringed Musical Instruments & Components Thereof, Inv. No. 337-TA-586, USITC Pub. No. 4120 (Dec. 2009) ...... 34
In re Certain Rotary Wheel Printing Sys., Inv. No. 337-TA-185, USITC Pub. No. 1857 (May 1986)...... 38
vii Case: 12-1252 Document: 46 Page: 9 Filed: 08/29/2012
FEDERALRULES & STATUTES
Fed. Cir. R. 30(j) ...... 1
19 U.S.C. § 1337 ...... passim
28 U.S.C. § 1295(a)(6)...... 3
LEGISLATIVE HISTORIES
S. Rep. No. 100-71 (1987)...... 20, 34
H.R. Rep. No. 100-40 (1987)...... 20, 32, 34, 40
REGULATIONS
75 Fed. Reg. 68,379 (Nov. 1, 2010)...... 4
viii Case: 12-1252 Document: 46 Page: 10 Filed: 08/29/2012
STATEMENT OF RELATED CASES
No other appeal from the proceeding below was previously before this Court or any other appellate court.
The same parties and one of the patents involved in this appeal are involved in a district court action, Motiva, LLC v. Nintendo Co.,
Ltd., et al., No. 2:10-cv-00349 (W.D. Wash. filed Nov. 10, 2008). That action was filed in the Eastern District of Texas, but, on Nintendo’s petition to the Federal Circuit, this Court ordered the case transferred to the Western District of Washington. In re Nintendo Co., Ltd., et al.,
589 F.3d 1194, 1201 (Fed. Cir. 2009). In June 2010, the district court stayed the action pending completion of the International Trade
Commission investigation underlying this appeal and completion of the
U.S. Patent and Trademark Office’s reexamination of a patent.
ix Case: 12-1252 Document: 46 Page: 11 Filed: 08/29/2012
INTRODUCTION1
The individuals behind Motiva, LLC (“Motiva”) dreamed of riches resulting from an advanced type of exercise equipment that wirelessly and precisely tracked an individual’s movements. But the dreamers gave up that fantasy years ago once reality intruded. Creating a workable device required “vast amounts of money” (A7811), and
Motiva’s only funder walked away. The individuals moved on to other interests, and their “prototype,” which was “not close to being production ready,” lay fallow. A7820.
A7836. Today, only the dream of “‘winnings’ from a lawsuit against Nintendo” of
America Inc. and Nintendo Co., Ltd. (“Nintendo”) (A7822) keeps Motiva going. But, as the ALJ found both before (A7647) and after (A7837) an evidentiary hearing, Motiva “failed to demonstrate that a domestic industry ‘is in the process of being established,’” (id.), a necessary
1 In this brief, all “A” cites are to the parties’ Joint Appendix, and “Br.” cites refer to the appellant’s opening brief. “SVA” refers to the Supplemental Video Recording Media Appendix, which includes demonstratives submitted to the ALJ (Respondent’s Demonstrative Exhibit Nos. RDX-154, -156, and -158) and filed with this Court pursuant to Fed. Cir. R. 30(j). For the Court’s convenience, the relevant patents are reproduced at the back of this brief. The page numbers from the Joint Appendix are retained. Case: 12-1252 Document: 46 Page: 12 Filed: 08/29/2012
requirement for bringing a complaint before the International Trade
Commission (“ITC” or “Commission”), 19 U.S.C. § 1337(a)(2).
The ALJ also went on to conclude that Nintendo’s Wii System does not infringe Motiva’s patents. That conclusion is correct, and thus, to say the least, supported by substantial evidence. The Wii System does not “track” “movement” of a user, key limitations of Motiva’s patents. Crediting Nintendo’s witnesses, the ALJ found that the
“evidence adduced at the hearing demonstrates that the three key devices in the Wii Remote—the three-axis gyroscope, the three-axis accelerometer, and the DPD—do not, in fact, track the movement of the user or provide information regarding the position or orientation of the user.” A7773. The ALJ found Nintendo’s witnesses credible:
A7780.
The ITC’s order adopting the careful factual determinations of the
ALJ should be affirmed.
2 Case: 12-1252 Document: 46 Page: 13 Filed: 08/29/2012
JURISDICTIONAL STATEMENT
Motiva invoked the ITC’s authority under Section 337 of the Tariff
Act of 1930, as amended. A8746. See 19 U.S.C. § 1337(b)(1). The ITC’s final determination was issued on January 5, 2012. A7876. Motiva filed a petition for review with this Court on March 5, 2012. A30,562.
Motiva notes this Court’s jurisdiction to review ITC final determinations under 28 U.S.C. § 1295(a)(6). Br. 1.
STATEMENT OF THE ISSUES
1. Whether the ITC correctly found that Motiva failed to establish that “an industry in the United States, relating to the articles protected by” Motiva’s two patents (U.S. Patent No. 7,492,268 and U.S.
Patent No. 7,292,151) was “in the process of being established”
(19 U.S.C. § 1337)?
2. Whether substantial evidence supports the ITC’s factual finding that Nintendo’s Wii System does not track user movement nor determine the position or orientation of the user and thus does not infringe Motiva’s two patents?
3 Case: 12-1252 Document: 46 Page: 14 Filed: 08/29/2012
STATEMENT OF THE CASE
In September 2010, Motiva filed a proposed complaint at the ITC naming as respondents Nintendo Co., Ltd. and Nintendo of America
Inc. A8746. The ITC issued a Notice of Investigation. See 75 Fed. Reg.
68,379 (Nov. 1, 2010). Administrative Law Judge Robert K. Rogers, Jr. issued an Initial Determination granting Nintendo’s motion for summary determination. A7631-49. The ITC vacated the summary determination, posed a series of factual questions, and remanded for an evidentiary hearing. A7858-69. The evidentiary hearing took place from August 1 through August 5, 2011. A7662. The witnesses included the two named inventors, six fact witnesses (including two Nintendo employees), and six experts. A7663. Thereafter, the ALJ issued a second Initial Determination, again finding no violation. A7855.
Motiva petitioned the ITC for review. A8537. The ITC rejected the petition and ruled that “[t]he ALJ’s conclusion that Motiva has not proven a violation of section 337 is correct” and “is the Commission’s final determination.” A7876.2
2 Although the ITC determined to review portions of the Initial Determination on its own initiative, see A7871, those issues are not raised in this appeal.
4 Case: 12-1252 Document: 46 Page: 15 Filed: 08/29/2012
STATEMENT OF THE FACTS
A. Motiva’s Research And Marketing Efforts: The Start And Finish
Kevin Ferguson and Donald Gronachan had a dream. But like so many get-rich dreams, theirs never even came close to fruition. They had exactly one investor who abandoned them, got no traction from any identified licensee, and never created a product—or even a passable prototype of a product.
The dream was to develop an advanced technology that would precisely measure human movement for the fitness and rehabilitation market. A7810. The product they had in mind was a system consisting of a screen, a base station, and a handheld controller. A9519. They thought the product might be used for general exercise, athletic performance training, and physical therapy and research. A30,638.
The Ferguson-Gronachan collaboration began in October 2003.
They formed Motiva as an “informal partnership.” A7810. Mr.
Gronachan had a day job at a company called Biodex Medical Systems.
A7811. Mr. Ferguson devoted time to developing some prototypes— largely compressed in a 16-month period between October 2003 and
April 2005. A7823. During that period he tried to build a “proof-of-
5 Case: 12-1252 Document: 46 Page: 16 Filed: 08/29/2012
concept” prototype and a “demonstration” prototype. A10,490; see also
A10,179-80, 20,717. The proof-of-concept prototype no longer exists.
A20,717. “The demonstration prototype cannot be described as anything close to production-ready. . . . [T]he prototype lacks industrial design, fine-tooling, functional engineering, and test phase manufacturing.” A7819-20, 10,506-07. See also A7819 (describing the prototype as having “exposed circuit boards, wiring, and sensors”). It was the engineering equivalent of a stick figure, albeit an expensive one: Motiva planned to sell the device for See, e.g., A20,708-
09.
Motiva’s development work was funded—albeit only briefly—by the only investor the duo ever attracted. His name was David Smith.
Motiva claims that Mr. Smith contributed a sum total of to engineering, research, and development. A7807-08, 20,523-24. Mr.
Ferguson accepted the majority of this funding— —as salary.
A7807, 7811. Motiva spent the rest on supplies for the prototypes. Id.
Within a year, Mr. Smith pulled the plug on the investment. He told the duo in 2004 that he had lost interest in supporting the project.
A7811. He realized that Mr. Ferguson and Mr. Gronachan had no hope
6 Case: 12-1252 Document: 46 Page: 17 Filed: 08/29/2012
of bringing anything to market without “vast amounts of money”—far
“more money than he could afford to invest.” Id. By early 2005, Mr.
Smith abandoned the venture without even recovering his investment.
A7814. He did not bother retaining any ownership rights in the inventors’ work. A7810, 20,532, 20,909, 30,403, 30,404-06.
With their only investor gone, Motiva ground to a halt. Mr.
Ferguson took a job at Liebert Corporation. Now that he had a day job,
Mr. Ferguson had virtually no time to devote to developing the product.
A9534, 10,179-80. While Mr. Ferguson continued for a couple of years to dabble in the project when time permitted, by 2007 he had completely abandoned the pretense of working on the invention. A9539.
In the entire four-year period of their collaboration—from 2003 to
2007—Mr. Ferguson and Mr. Gronachan made exactly three pitches in the hopes of attracting a licensee.
The first pitch, in January 2005, was to James Reiss. Mr. Reiss was Mr. Gronachan’s boss at Biodex. A7811. The meeting took place in the basement of Mr. Ferguson’s home. A10,447. The pitch went nowhere because the technology was not compatible with Biodex’s products. A7812.
7 Case: 12-1252 Document: 46 Page: 18 Filed: 08/29/2012
Fifteen months elapsed before Motiva had an opportunity for a second pitch. In March 2006, Mr. Gronachan had several conversations about the Motiva technology with Gregory Highsmith. A7813. Mr.
Highsmith is one of Mr. Gronachan’s “oldest, long standing friends.”
A20,668. Mr. Highsmith was employed by a commercial fitness equipment manufacturer known as Life Fitness. “As a member of his company’s innovation committee, Mr. Highsmith was in a perfect position to introduce the Motiva technology to Life Fitness, but he did not do so.” A7814. Life Fitness just was not interested. Id. Mr.
Highsmith stated that he never would have even bothered talking about the Motiva device but for his “personal relationship” with Mr.
Gronachan. A20,668. See also A7814.
A good nine months later, in January 2007, came the third, and last, pitch. Mr. Ferguson and Mr. Gronachan met with Michael
Lannon, the CEO of Koko Fitness. A7816. The meeting was held on a
Saturday because, as Mr. Lannon explained, it “wasn’t important enough to take time out of our actual workweek.” Id. (quoting A10,458); see also A10,456. It was not even a real opportunity. Koko Fitness was
8 Case: 12-1252 Document: 46 Page: 19 Filed: 08/29/2012
“a small company” with “limited development resources.” A7816
(quoting A10,456).
After this third strike, having already abandoned their development efforts, Mr. Ferguson and Mr. Gronachan abandoned all further marketing efforts.
B. Motiva’s Patents
What Mr. Ferguson and Mr. Gronachan did manage to do was to secure two patents. In July 2004, they filed a provisional patent application. A7810. The application resulted in U.S. Patent No.
7,292,151, “Human Movement Measurement System,” A1-35, issued on
November 6, 2007, and U.S. Patent No. 7,492,268, A36-67, a continuation of the ‘151 patent, issued on February 17, 2009.
Both patents relate “to a system and methods for setup and measuring the position and orientation (pose) of transponders” for the purposes of “exercise” and “rehabilitation.” A14 at 1:11-13
(emphasis added); A50 at 1:13-15 (emphasis added). Before setting out the patent more fully, a few words are in order about the bolded terms.
The “position” of any object relative to another object at any particular time can be described by “x, y and z” coordinates. The three
9 Case: 12-1252 Document: 46 Page: 20 Filed: 08/29/2012
coordinates describe an object’s location in space relative to an origin point. Here is a graphical image of this coordinate system:
So, for example, using this Court as the point of origin, one can describe the position of the flag atop the White House in terms of its x, y, and z relative to the front door of 717 Madison Place. If west-east is the x-axis, north-south the y-axis, and up-down the z-axis, the position of the flag relative to the Court would be (in feet) x=+413, y=+734, and z=+95.
As paradoxical as it might at first sound, an object can move without changing its position. That is because a change in location is not the only sort of movement. A pigeon flying from the White House across Lafayette Square to the Court is obviously moving in the sense of position. But if the pigeon sits atop the flagpole and spins around, or
10 Case: 12-1252 Document: 46 Page: 21 Filed: 08/29/2012
tilts forward or to the side, it is moving even though its position has not changed.
An object’s rotation about an axis is referred to as “orientation.”
Orientation is measured as the amount of rotation along each of three axes at any particular time. “Roll” is the degree of rotation around the x-axis (e.g., a bird leaning left or right); “pitch” is the degree of rotation around the y-axis (e.g., a bird looking up or down); and “yaw” is the degree of rotation around the z-axis (e.g., a bird balancing on one leg and spinning in circles). See, e.g., A10,306.
“Pose” refers to measurement of both “position” and “orientation.”
See A14 at 1:11-13 (“This invention relates to a system and methods for setup and measuring the position and orientation (pose) of transponders.”).
In sum, the pose of an object is described by using six coordinates—three coordinates specific to position (x, y, z), and three coordinates specific to orientation (roll, pitch, yaw).
Although not a disputed term, by way of background a
“transponder” is a wireless device that responds to a signal with a signal of its own. The patent’s specification defines a transponder as a
11 Case: 12-1252 Document: 46 Page: 22 Filed: 08/29/2012
“wireless communication and monitoring device that receives a specific signal and automatically responds with a specific reply.” A14 at 2:18-
20. A transponder is often “hand-held.” A15 at 4:34-35.
The ‘151 patent discloses an invention that trains the user to manipulate “the pose of the transponders through a movement trajectory” for the purposes of exercise and rehabilitation. A14 at
1:13-17 (emphasis added). Immediately below is an image (a composite of patent Figures 4A, 4B, and 6A, see A10, 12) illustrating how the system determines the transponder’s position and orientation:
12 Case: 12-1252 Document: 46 Page: 23 Filed: 08/29/2012
In the diagram above, the user (represented by the figure) holds a transponder. A10. In response to a request from the computer, the transponder sends a signal (indicated by the blue lines) to the receiver.
To determine the transponder’s position relative to the receiver, the computer engages in a two-step process: (1) it measures how long it takes the signal to reach the different points on the receiver; and (2) it performs certain geometric and algebraic calculations to translate those measurements into the distance on the x, y, and z axes. A17 at 8:9-11,
23 at 20:12-16, 27 at 27:55-60, 28 at 29:6-7. With slightly different calculations, the computer can also determine the position of the transponder relative to an “origin” point other than the receiver. A28 at
29:31-32.
To determine the orientation of the transponder, the computer calculates the position of three separate points on the transponder. See
A28 at 29:39-45. The three positions define a unique plane, and that plane reflects the transponder’s roll, pitch, and yaw. Id.
13 Case: 12-1252 Document: 46 Page: 24 Filed: 08/29/2012
Transponder →
As explained above, position and orientation, or pose, are measurements at any particular moment in time. The patents refer to a “movement trajectory.” See, e.g., A14 at 1:14-15; A50 at 1:16-17. As the user moves the transponder, the system repeatedly generates and stores the position and orientation measurements:
By recording the transponder’s poses over time, the system tracks the user’s movement.
14 Case: 12-1252 Document: 46 Page: 25 Filed: 08/29/2012
The first claim in the patent reads as follows:
A system for tracking movement of a user, comprising:
a first communication device . . .;
a processing system . . . said processing system adapted to determine movement information for said first communication device and sending data signals to said first communication device for providing feedback or control data; and
wherein said first communication device receives and processes said data signals from said processing system and wherein the output device provides sensory stimuli according to the received data signals.
A31 at 35:39-54 (emphasis added). Claim 50 claims an “apparatus for use in tracking movement of a user.” A32 at 38:47.
The ‘268 patent is a continuation of the ‘151 patent. Claim 1 claims a system for “tracking position of a user.” A66-67 at 34:59-35:8.
Claim 10 claims an “apparatus for use in conjunction with a remote processing system for tracking position of a user.” A67 at 36:7-8.
Claim 15 is for a “system for tracking movement of a user.” A67 at
36:35.
In 2010, in two separate orders, the PTO ordered reexamination of each patent. The examiner canceled some claims and confirmed
15 Case: 12-1252 Document: 46 Page: 26 Filed: 08/29/2012
others.3 Nintendo has appealed to the Board of Patent Appeals as to both patents.
C. Motiva’s Patent Litigation
1. Motiva Files An ITC Action Against Nintendo
In November 2008, Motiva commenced a patent infringement action against Nintendo in the United States District Court for the
Eastern District of Texas. This Court ordered the litigation transferred to the Western District of Washington, where Nintendo of America is located. In re Nintendo Co., Ltd., et al., 589 F.3d 1194, 1198
(Fed. Cir. 2009). In June 2010, the district court in Washington stayed the action pending the outcome of the reexamination proceedings at the
Board of Patent Appeals. A few months later, Motiva sued Nintendo in the ITC, alleging that Nintendo violated Section 337 by importing and selling the Wii game system and related items. A8746.
Motiva’s allegations center on the Wii Remote, the primary controller for the Wii System. Here is a picture of the Wii Remote Plus:
3 As to the ’151 patent, the examiner canceled claims 1-11, 17-20, 26, 35, 36, 38-43, 45-56, 58-64, 66, 67, 69-72, 77, 78, 82, 83, and 85-91. The examiner confirmed claims 12-16, 21-25, 27-34, 37, 44, 57, 65, 68, 73-76, 79-81, and 84. As to the ‘268 patent, the examiner canceled claims 1, 3, and 13, confirmed claim 4, and found claims 2, 5-12, 14, and 15 patentable.
16 Case: 12-1252 Document: 46 Page: 27 Filed: 08/29/2012
As relevant here, the Wii Remote Plus has three key devices:
(1) a three-axis gyroscope
(2) a three-axis accelerometer
(3) a particular type of camera referred to as a Direct Pointing
Device (“DPD”).
The Wii System was released in the United States in November
2006, A8276, two full years after Mr. Smith stopped funding Motiva.
A7811. The Wii sells for approximately $149. A8278.
“Motiva argues that these three devices contained within the Wii send movement and position information” to the Wii console. A7773.
The complaint asserted that the Wii System infringes three independent claims and various dependent claims of Motiva’s patents.
17 Case: 12-1252 Document: 46 Page: 28 Filed: 08/29/2012
All the asserted ‘151 claims require “tracking movement of a user.”4
A8746. All but one of the asserted ‘268 claims require “tracking position of a user.”5 Id.
The ITC opened an investigation.
2. The ALJ Grants Nintendo’s Motion For Summary Determination
In Order No. 12, the ALJ granted Nintendo’s “motion for summary determination that the economic prong of the domestic industry requirement is not satisfied.” A7631. The ALJ noted that a patent complainant invoking the ITC’s authority must establish that an industry “relating to the articles protected by the patent . . . exists or is in the process of being established” in the United States, A7634
(quoting 19 U.S.C. § 1337(a)(2)).
After summarizing the research and marketing effort set forth by
Motiva, the ALJ ruled that “Motiva’s investments do not support a
4 The complaint, as corrected, asserted eight claims dependent on claim 1 (claims 16, 27-32, 44) and four claims dependent on claim 50 (57, 68, 81, and 84). 5 The complaint asserted claim 1 of the ‘268 patent and 7 claims (claims 2-6 and 8-9) dependent on claim 1, as well as claim 10 and 4 claims (11-14) dependent on claim 10. The complaint also asserted claim 15, which refers to “tracking movement of a user.”
18 Case: 12-1252 Document: 46 Page: 29 Filed: 08/29/2012
finding of either the existence of a domestic industry or that a domestic industry was in the process of being established at the time of filing of the complaint in this investigation.” A7637. The ALJ found that
“Motiva ceased investing in research and development, marketing, or any step towards creating a domestic industry, in 2007.” A7648. The
ALJ reasoned that: (i) “Motiva does not directly dispute that it had stopped investing in research and development and marketing for the asserted patents in 2007”; (ii) “Motiva’s patent litigation activities are not to be considered as part of the domestic industry analysis, and its prosecution of ‘related applications’ is not relevant to the domestic industry analysis”; and (iii) Motiva’s “assertion that once it wins its patent infringement litigations” it will “reenter the market” is
“insufficient” to establish a domestic industry because to “find otherwise would render the domestic industry requirement a nullity.” Id. The
ALJ granted summary determination to Nintendo. A7631.
3. The ITC Reverses And Remands With Questions
Motiva and the Office of Unfair Import Investigations petitioned for review. The ITC reversed and remanded. The ITC explained that the “legislative history indicates that an industry is ‘in the process of
19 Case: 12-1252 Document: 46 Page: 30 Filed: 08/29/2012
being established’” if the patent owner “can demonstrate that he is taking the necessary tangible steps to establish such an industry in the
United States,” A7863-64 (citing S. Rep. No. 100-71, at 130 (1987)), and there is a “significant likelihood that the industry requirement will be satisfied in the future,” A7864 (citing H.R. Rep. No. 100-40, at 157
(1987)). The ITC ruled that the “ALJ erred in declining to consider
Motiva’s activities that occurred before the issuance of the asserted patents.” A7863 (citing A7672).
The ITC also found that Motiva had raised a “genuine issue of material fact as to whether its district court litigation activities between
2007 and the present are related to licensing and/or product development.” A7865. The ITC directed the ALJ to address “to the extent necessary” several questions “relevant” either “to whether a domestic industry exists” or “is in the process of being established.”
A7867.
4. The ALJ Holds A Hearing, Answers The ITC’s Questions, And Finds No Violation
The ALJ holds the hearing. At the hearing, the ALJ heard from nine witnesses called by Motiva: the two inventors; the one short- term investor; two of the three targets of pitches (Mr. Highsmith and
20 Case: 12-1252 Document: 46 Page: 31 Filed: 08/29/2012
Mr. Reiss); three experts; and a Nintendo engineer. A7663. Nintendo called three experts, a Nintendo manager, and Mr. Barry French (the
CEO of Trazer Technologies and Mr. Ferguson’s former boss). Id.
At the hearing, Nintendo’s expert testified that “Motiva abandoned any domestic industry that it might have had in process.”
A10,488. He based his conclusion “on the fact that Mr. Ferguson’s work
. . . waned significantly . . . in early 2005, and then stopped entirely in
January 2007” and that “Mr. Gronachan . . . did not put in any significant effort at any time.” Id. In response to one of Motiva’s experts, Mr. Bakewell prepared the following graph to illustrate the point:
21 Case: 12-1252 Document: 46 Page: 32 Filed: 08/29/2012
The ALJ finds no ITC authority over Motiva’s suit. Based on a careful review of the trial record, the ALJ concluded that “Motiva has failed to prove that a domestic industry ‘exists’ pursuant to Section
337(a)(2).” A7832. The ALJ summarized the record:
(1) well before the release of the Wii, Motiva was facing a lack of funding due to the departure of David Smith;
22 Case: 12-1252 Document: 46 Page: 33 Filed: 08/29/2012
(2) prior to the Wii being released, Motiva met with potential partners, but they showed little to no interest in investing in the Motiva technology;
(3) the one company that Motiva met with after the release of the Wii—Koko Fitness—was not interested in investing in the technology for reasons other than the fact that the Wii was already on the market;
(4) Motiva has offered no evidence that any potential investor, partner, or licensee was dissuaded in investing in the Motiva technology due to the Wii;
(5) the Wii was not in the same market as the Motiva product, and the two products would not have competed;
(6) Motiva’s actions in litigation are not consistent with a company whose main purpose is to remove Nintendo from the market so that it can enter the market; and
(7) Motiva has not demonstrated that litigation was its only recourse to salvage its business in the face of Nintendo’s release of the Wii.
A7830-31 (emphasis added).
The ALJ also provided the following answers to the ITC’s questions relevant to the existence of a domestic industry (with all emphasis added):
Commission Question ALJ Findings
1[a]. What was the level of interest “I find that there was little from potential manufacturers, interest from potential investors, and licensees in Motiva’s manufacturers, investors, and technology prior to the release of licensees in Motiva’s technology the Wii? prior to release of the Wii.” A7814.
23 Case: 12-1252 Document: 46 Page: 34 Filed: 08/29/2012
1[b]. Did Nintendo’s release of the “I now find that Motiva has failed Wii cause this interest to decrease? to demonstrate that there was any decline in interest caused by the release of the Wii.” A7815.
1[c]. To what extent would the “I find that the product developed product(s) being developed by by Motiva would not compete Motiva compete with Nintendo’s with the Nintendo Wii.” A7817. Wii?
2. How close was Motiva’s “I find that Motiva’s technology technology to being was not close to being commercialized and/or production incorporated in a commercial or ready? production-ready product.” A7819.
3[a]. To what extent was Motiva’s “I find no evidence to support shift in production-oriented Motiva’s claim that the turn to activities to litigation-oriented litigation was the result of being activities a strategic business rejected by potential investors due decision not caused by Nintendo’s to the presence of the Wii.” A7822. activities? “Motiva’s litigation tactics strongly suggest that the purpose behind the litigation was to extract a monetary award either through damages or a financial settlement.” A7823.
3[b]. Could Motiva have continued “It was possible that Motiva could its commercialization efforts have continued its without resorting to litigation? commercialization efforts without resorting to litigation, but it would have taken a new source of money to do so.” A7823.
In addition to finding that no domestic industry existed, the ALJ concluded that “Motiva has failed to demonstrate that a domestic
24 Case: 12-1252 Document: 46 Page: 35 Filed: 08/29/2012
industry ‘is in the process of being established.’” A7832-37. The ALJ found that “[f]rom 2003 to 2007, Motiva was taking tangible steps to establish an industry in the United States.” A7833. But the “building” and “market[ing]” “activities ended in 2007.” A7833-34. “Therefore,” the ALJ found, “after 2007, Motiva abandoned its efforts to establish an industry in the United States.” A7834 (citing A10,488-90). “Mr.
Bakewell’s credible testimony supports this conclusion.” Id. The ALJ also rejected Motiva’s argument that the “litigation against Nintendo is evidence of Motiva taking the necessary tangible steps to establish an industry.” Id.
Not only was Motiva not taking tangible steps to establish an industry, the ALJ found that “Motiva has not demonstrated that there is a significant likelihood that the industry requirement will be satisfied in the future.” Id. “Motiva could not demonstrate that there was any significant interest in its technology prior to the Wii’s existence, and I find that there is no reason to believe that manufacturers of fitness and rehabilitation equipment will suddenly become interested in the technology because Nintendo has been excluded from the market.”
A7835. Moreover, the ALJ found that “Motiva has not shown that it
25 Case: 12-1252 Document: 46 Page: 36 Filed: 08/29/2012
has ever attracted interest from the video game industry.” Id. In addition, the ALJ found that
A7836. The ALJ found that “Motiva failed to demonstrate that a domestic industry ‘is in the process of being established,’ pursuant to Section 337(a)(2).” A7837.
In so holding, the ALJ answered the ITC’s questions relevant to any future industry:
3[c]. Was Motiva taking the “I find that after 2007, Motiva “necessary tangible steps to abandoned its efforts to establish” a domestic industry? establish an industry in the United States.” A7834. “I do not find that the litigation against Nintendo is evidence of Motiva taking the necessary tangible steps to establish an industry.” Id.
4[a]. Do the steps “taken [by “I conclude that Motiva has not Motiva] indicate a significant demonstrated that there is a likelihood that the industry significant likelihood that the requirement will be satisfied in the industry requirement will be future?” satisfied in the future, and this determination is not dependent on 4[b]. How likely is it that Motiva the Commission’s actions in this will have a domestic industry in investigation.” Id. the future (1) if no relief is issued against Nintendo or, alternatively, (2) if relief is granted against Nintendo?
26 Case: 12-1252 Document: 46 Page: 37 Filed: 08/29/2012
The ALJ also finds no infringement. In addition to rejecting
Motiva’s complaint for failure to establish a condition for an ITC action, the ALJ separately found that Nintendo’s Wii System does not infringe the Motiva patents. A7854-55.
In so doing, the ALJ’s Initial Determination construed certain claim terms. The phrase “tracking movement of a user” appears in the preamble of independent claims 1 and 50 of the ‘151 patent. A31-32.
The ALJ ruled that the phrase means “tracking changes of position and/or orientation of a user.” A7674.
The ALJ’s construction was different from Nintendo’s reading of the patent. Nintendo argued that “tracking movement of a user” required tracking the position and orientation of the user. Among other things, Nintendo noted that the patent describes the invention as
“measuring the position and orientation (pose) of transponders.” A7673
(emphasis added). Nevertheless, the ALJ found that “tracking movement of a user” can mean tracking either the position or orientation of the transponder. A7672.
In addition to construing the claims to require “tracking changes of position and/or orientation of a user,” the ALJ construed “position
27 Case: 12-1252 Document: 46 Page: 38 Filed: 08/29/2012
information.” The ALJ explained that the “parties’ proposed constructions do not vary greatly,” and “[t]he primary dispute centers on Nintendo’s inclusion of ‘3D space.’” A7678. The ALJ found that
“[t]he specification explains that the invention may be used ‘for the purposes of functional movement assessment for exercise, and physical medicine and rehabilitation,’” Id. (citing A14 at 1:16-17), and “[s]uch tracking requires knowledge of the user’s location in 3D space,” id.
(citing A10,330-31). Although Motiva invoked certain statements by the examiner made during the reexamination of the ‘151 patent, the
ALJ found them “not persuasive” in light of the “broader standard for claim construction during reexamination” and because the
“reexamination of the ‘151 patent is not complete.” A7679.
After construing the key terms, the ALJ found that “Motiva has failed to show by a preponderance of the evidence that the accused products infringe claim 1 of the ‘151 patent.” A7782. At the hearing, the ALJ viewed live demonstrations in which Nintendo expert
Dr. J. Edward Colgate and Keizo Ohta, Nintendo’s Manager of the
Technology Group in the Entertainment Analysis and Development
Division (A7663), played Wii games and explained how the Wii creates
28 Case: 12-1252 Document: 46 Page: 39 Filed: 08/29/2012
the appearance of tracking the user’s movement without knowing the user’s position or orientation. A21,593-618, 30,447-56, 30,552-53,
30,576-79, 30,581-83; see generally, SVA at RDX-154, -156, and -158.
The ALJ found that the “evidence adduced at the hearing demonstrates that the three key devices in the Wii Remote—the three-axis gyroscope, the three-axis accelerometer, and the DPD—do not, in fact, track the movement of the user or provide information regarding the position or orientation of the user.” A7773. The ALJ concluded:
A7780. Instead, the Wii
Id.
(emphasis added); see id. (“After careful review of the record evidence, I find that Motiva has failed to meet its burden to demonstrate that the
Wii accused products track the movement of a user.”).
The ALJ applied the same analysis to find no infringement of the
‘268 patent.
29 Case: 12-1252 Document: 46 Page: 40 Filed: 08/29/2012
5. The ITC Affirms
Motiva appealed again to the ITC, but this time the ITC affirmed.
A7871. The Commission determined that “[t]he ALJ’s conclusion that
Motiva has not proven a violation of section 337 is correct and is the
Commission’s final determination. The investigation is terminated.”
A7876.
SUMMARY OF ARGUMENT
The Court should affirm the ITC’s decision to adopt the ALJ’s careful determination. The ALJ gave two independent reasons for rejecting Motiva’s complaint. Nothing in Motiva’s brief calls these reasons into question.
I.A. Substantial evidence supports the ALJ’s finding that Motiva was not “in the process” of “establishing” any domestic industry related to these patents. In a detailed opinion issued after an extensive hearing, the ALJ found that “after 2007, Motiva abandoned its efforts to establish an industry in the United States.” A7834.
Motiva makes no effort to challenge the ALJ’s finding that its licensing efforts have “ended.” Id. Motiva does not deny that its main investor walked away, does not claim to have another funding source,
30 Case: 12-1252 Document: 46 Page: 41 Filed: 08/29/2012
and does not claim to have done anything since 2007 (except sue
Nintendo). Motiva has stopped trying to establish any industry, and its completed actions are not evidence that it will do any research or marketing in the future.
B. Against these and similar dispositive factual findings, Motiva makes a variety of limited and erroneous points. For example, much of
Motiva’s brief turns on its contention (at 44) that “[b]ut for the presence of the Wii in the market, Motiva was—and still is—extremely close to realizing its product-driven licensing goals.” The ALJ specifically rejected this fiction. A7812-13 (citing A20,633-34). The ALJ also ruled that “Motiva could not demonstrate that there was any significant interest in its technology prior to the Wii’s existence,” and “that there is no reason to believe that manufacturers of fitness and rehabilitation equipment will suddenly become interested in the technology because
Nintendo has been excluded from the market.” A7835.
This Court has explained that “Congress recognized that the
Commission is fundamentally a trade forum, not an intellectual property forum, and that only those intellectual property owners who are ‘actively engaged in steps leading to the exploitation of the
31 Case: 12-1252 Document: 46 Page: 42 Filed: 08/29/2012
intellectual property’ should have access to the Commission.” John
Mezzalingua Assocs. v. ITC, 660 F.3d 1322, 1327-28 (Fed. Cir. 2011)
(quoting H.R. Rep. No. 100-40, at 157 (1987)). Here, Motiva’s bald assertion that it is a “licensing company” is not sufficient to allow it to invoke the ITC’s jurisdiction. Motiva has nothing but its patents. If
Motiva can invoke the protections of the ITC, any and every patent owner can do so. This Court does not permit such a lax definition of the
ITC’s role.
II. The Wii System does not infringe Motiva’s patents. Because
Motiva did not establish any basis for bringing its complaint to the ITC, the ITC decision should be affirmed on that basis alone. Nevertheless, the ALJ reached, and the ITC adopted, a second rationale for rejecting
Motiva’s complaint: The Wii System does not infringe Motiva’s patents.
A. Substantial evidence supports the Commission’s determination that the Wii System does not “track” movement of a user. A7777-82.
The ALJ found that the Wii System does not take “complete measurements” of user movement, does not measure user movement with “reasonable accuracy,” and does not keep any “representation of the movement history in memory.”
32 Case: 12-1252 Document: 46 Page: 43 Filed: 08/29/2012
Motiva argues (at 25-26) that the ALJ’s construction of “tracking”
“directly contradicts the Commission’s apparent construction of
‘tracking movement of a user.’” Br. 24-25 (citing A7674 at ¶¶ 3-4). This is inexplicable. The cited paragraphs do not construe the term
“tracking.” In any event, the Wii System does not “track” movement.
B. Moreover, and independently, substantial evidence supports the ITC’s finding that the Wii System does not determine user movement. As three witnesses that the ALJ found persuasive explained, the Wii sensors cannot and do not determine the position or orientation of the transponder. Motiva’s hodgepodge of movement and position information arguments are all meritless.
STANDARD OF REVIEW
“[F]indings of fact [by the ITC] are reviewed to ascertain whether they were supported by substantial evidence on the record as a whole.”
Erbe Elektromedizin GmbH v. ITC, 566 F.3d 1028, 1033
(Fed. Cir. 2009). The Court routinely affirms ITC decisions that are supported by substantial evidence. See, e.g., Ninestar Tech. Co. v. ITC,
667 F.3d 1373, 1379 (Fed. Cir. 2012); TianRui Group Co. v. ITC, 661
F.3d 1322, 1337 (Fed. Cir. 2011); Mezzalingua, 660 F.3d at 1330.
33 Case: 12-1252 Document: 46 Page: 44 Filed: 08/29/2012
Claim construction is an issue of law this court reviews de novo.
Tessera, Inc. v. ITC, 646 F.3d 1357, 1364 (Fed. Cir. 2011). Infringement decisions are a question of fact reviewed for substantial evidence. Id.
ARGUMENT
I. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’S FINDING THAT MOTIVA WAS NOT IN THE PROCESS OF ESTABLISHING ANY DOMESTIC INDUSTRY
As the ALJ realized from the get-go, Motiva is not entitled to invoke the ITC’s authority. Motiva could not prove that a domestic industry is “in the process of being established,” 19 U.S.C. § 1337(a)(2), without establishing: (1) “that [it] is taking the necessary tangible steps to establish such an industry in the United States,” and (2) that
“there is a significant likelihood that the industry requirement will be satisfied in the future.” Certain Stringed Musical Instruments &
Components Thereof, Inv. No. 337-TA-586, USITC Pub. No. 4120, at 13
(Dec. 2009) (Comm’n Op.) (quoting S. Rep. No. 100-71, at 130 (1987) &
H.R. Rep. No. 100-40, at 157 (1987)).
Motiva’s brief reads as if the ALJ made no findings. But in a detailed opinion issued after an extensive hearing, the ALJ found that
“after 2007, Motiva abandoned its efforts to establish an industry in the
34 Case: 12-1252 Document: 46 Page: 45 Filed: 08/29/2012
United States.” A7834. The decision is supported by substantial— indeed, overwhelming—evidence and should be affirmed.
A. Motiva’s Stale Licensing Efforts Do Not Demonstrate Any Tangible Steps Toward Establishing A Domestic Industry
The ALJ’s two opinions here call to mind the Monty Python “dead parrot” sketch. There, John Cleese returns to the pet shop to register a complaint about his dead Norwegian Blue parrot that had been nailed to its perch. The shop owner insists that the bird is “resting,” “pining for the fjords,” or simply “stunned.” Cleese responds by banging “Polly
Parrot” on the counter and rattles off several metaphors for death: the bird “is no more,” “has ceased to be,” “bereft of life, it rests in peace.”
The sketch is one of the most famous in the history of British television comedy.
The propped-up parrot here is Motiva’s technology, and the ALJ is playing the role of Cleese by repeatedly and emphatically declaring the technology dead. As the ALJ found both before (A7648) and after
(A7835-36) an evidentiary hearing, Motiva’s technology was “bereft of life” by the time Motiva filed its complaint in 2010. Motiva’s only investor quit in 2004. The ALJ noted that the “Motiva technology has
35 Case: 12-1252 Document: 46 Page: 46 Filed: 08/29/2012
not been updated or improved since at least December 2007.” A7820
The ALJ found that Motiva’s licensing efforts “ended in 2007.” A7834-
35. “Thus, on October 1, 2010, the relevant date for determining a domestic industry, it had been at least 3.5 years since the end of the
[sic] Motiva’s engineering, research and development, and commercialization activities.” A7826. Motiva has no future products and no potential customers. Instead, Motiva exists solely to seek a cut in the hoped-for contingent lawsuit “‘winnings.’” A7822. The ALJ notes that Motiva does not “directly” challenge this version of events. A7648.
Instead, Motiva emphasizes (at 40) the ALJ’s ruling that “Motiva was taking tangible steps to establish an industry in the United States” from 2003 to 2007.6 Motiva argues this finding is evidence that it was seeking to establish a domestic industry when it filed this complaint in
2010.7 And these same steps form the basis of Motiva’s claim (at 48) that the “public interest” supports ITC authority here.
6 On appeal, Motiva has abandoned its argument that ITC authority could be based on the “existence” of a relevant domestic industry. Compare A8023 with Br. at 10. 7 For purposes of this appeal, Nintendo does not dispute the ALJ’s finding that Motiva’s activities before the end of 2007 were steps toward establishing a licensing program. But the record is far from clear that the sporadic effort set forth by Motiva amounts to a “substantial”
36 Case: 12-1252 Document: 46 Page: 47 Filed: 08/29/2012
But Motiva cannot turn old news into future prospects. The ALJ found that “all” of Motiva’s licensing efforts “ended in 2007.” A7834.
“Motiva lost its primary source of funding when David Smith ended his investment in late 2004, and Motiva has never found a suitable investor to replace him,” id., nor did it try, A10,489. “Since the end of 2007,
Motiva has focused on nothing except suing Nintendo.” A7834. Motiva did not have any executed licensing agreements, term sheets, letters of intent, confidentiality agreements, licensing negotiations, trade show demonstrations, or any offers made or received to license Motiva’s technology. See A8751 (“Motiva has not licensed the Asserted
Patents.”). See also A20,686, 20,687, 20,689, 20,690, 20,706, 20,722,
20,723. The ALJ found that “after 2007, Motiva abandoned its efforts to establish an industry in the United States.” A7834; see also id. (“Mr.
Bakewell’s credible testimony supports this conclusion” (citing A10,488-
90)).
Motiva makes no effort to challenge the ALJ’s finding that its licensing efforts have “ended.” A7834. Motiva does not deny that Mr.
Smith walked away, does not claim to have another funding source, and investment in licensing. Cf. Br. 51 (asserting there is “no dispute” regarding pre-2008 activities).
37 Case: 12-1252 Document: 46 Page: 48 Filed: 08/29/2012
does not claim to have done anything since 2007 (except sue Nintendo, discussed below in section I.C). Motiva has stopped trying to establish any industry, and its completed actions are not evidence that it will do any research or marketing in the future. Compare Bally/Midway Mfg.
Co. v. ITC, 714 F.2d 1117, 1122 (Fed. Cir. 1983) (“if there was an existing . . . [domestic industry] when the complaint was filed, section
337(a) was satisfied.”) (emphasis added); see In re Certain Rotary Wheel
Printing Sys., Inv. No. 337-TA-185, USITC Pub. No. 1857, at 42 (May
1986) (holding that Bally/Midway “was attempting to take account of the situation where an industry is destroyed in the course of a
Commission investigation”).
Although it is not even attempting to license its patents, Motiva continues to refer to itself (at 39, 43) as a “product-driven licensing company.” But Motiva has no products, no licenses, and no drive. It is done. Motiva began marketing its technology as early as January 2005, yet it has never come close to negotiating a license with a third party, let alone having its technology incorporated into a commercial product.
A7820, 8751, 10,486-87, 20,686, 20,687, 20,689, 20,723. After Nintendo observed that “Motiva has never offered to license its patents to
38 Case: 12-1252 Document: 46 Page: 49 Filed: 08/29/2012
Nintendo,” Motiva wrote: “Nintendo has it right. Motiva’s litigation is not about licensing Nintendo.” A30,364 (emphasis added).
Also meritless is Motiva’s claim (at 39) to a focus on the “video game environment.” Its patents disavow any real interest in the video game business. See A21 at 15:38-43 (“It is important to note that only primitive forms of video game challenges would be considered, to take into account the user’s cognitive awareness and physical limitations, and the economics of software development for photo realistic virtual environments and animation.”) (emphasis added); A57 at 15:29-34
(same). Motiva has made no effort to compete with Nintendo in the video game market. A20,708-09.
But even if Motiva is a licensing company, it has not shown the required “substantial licensing activities related to the asserted patent” necessary to satisfy the future domestic industry requirement.
InterDigital Commc’ns, LLC v. ITC, No. 2010-1093, 2012 U.S. App.
LEXIS 15893, at *32 (Fed. Cir. Aug. 1, 2012). Since 2007, the ALJ found that Motiva engaged in no licensing activities related to any patents. A7834. Motiva complains that the ALJ “faults Motiva at length for its lack of consummated licensing deals.” Br. 46 (citing
39 Case: 12-1252 Document: 46 Page: 50 Filed: 08/29/2012
A7814) (emphasis added). That is rather like referring to a monk’s lack of consummated marriages. The ALJ’s finding was not just that the deals were not “consummated,” but that they never got past the first date: The ALJ found that “nothing ever came of those meetings.”
A7814. They do nothing to show the required tangible steps toward a future domestic industry.
As for public policy, “Congress recognized that the Commission is fundamentally a trade forum, not an intellectual property forum, and that only those intellectual property owners who are ‘actively engaged in steps leading to the exploitation of the intellectual property’ should have access to the Commission.” Mezzalingua, 660 F.3d at 1328
(quoting H.R. Rep. No. 100-40, at 157 (1987)). Motiva’s bald assertion that it is a “licensing company” is not sufficient to allow it to complain to the ITC. Motiva has nothing but its patents. If Motiva can invoke the protections of the ITC, any and every patent owner can do so. That is not the scheme Congress established. Id.
40 Case: 12-1252 Document: 46 Page: 51 Filed: 08/29/2012
B. Motiva’s Lawsuit Against Nintendo Is Not A Tangible Step Toward Establishing A Domestic Industry
The ALJ rejected Motiva’s argument that its district court
“litigation against Nintendo is evidence of Motiva taking the necessary tangible steps to establish an industry.” A7834.
In Coaxial Cable, the Commission held that for litigation activities to constitute “exploitation” of a patent under Section 337(a)(3)(C), the complainant must “clearly link” its litigation efforts to licensing.
Certain Coaxial Cable Connectors and Components Thereof and
Products Containing Same, Inv. No. 337-TA-650, USITC Pub. No. 4283, at 51 (Apr. 14, 2010) (Comm’n Op.). As the Commission explained,
“[a]llowing patent infringement litigation activities alone to constitute a domestic industry would place the bar for establishing a domestic industry so low as to effectively render it meaningless.” Id. at 46.
Motiva does not dispute that this settled agency practice is the correct understanding of Section 337(a)(3)(C). See InterDigital, 2012 U.S. App.
LEXIS 15893, at *32 (noting that “[i]f there were any ambiguity” about the scope of Section 337(a)(3)(C), the Commission’s “consistent interpretation of the statute . . . would be entitled to deference”).
41 Case: 12-1252 Document: 46 Page: 52 Filed: 08/29/2012
Here, the ALJ concluded that “the litigation against Nintendo does not relate to exploitation of the asserted patents.” A7832. Among the ALJ’s reasons for so holding were:
“Emails between the inventors before the start of any litigation show that they were interested in their potential ‘winnings’ from a lawsuit against Nintendo.” A7822 (citing A10,548); A30,547 (“The money is suppose [sic] to come from Laniers [sic] cut of the winnings and not from our cut.”) (emphasis added).
“Motiva’s decision not to file a complaint at the ITC from the outset or seek a preliminary injunction against Nintendo shows that Motiva was not concerned with taking swift actions to remove Nintendo from the market.” A7823.
“The inventors could have continued to seek investors, licensees, or partners instead of, or in conjunction with, suing Nintendo.” A7824.
Moreover, the ALJ held that even if the litigation is related to the patents, Motiva has not made a significant investment in the district court litigation. “Motiva’s lawyers are working under a contingent fee agreement.” A7831. “Motiva has not paid any attorney’s fees or expenses related to Motiva’s litigation against Nintendo.” Id.
Despite all this, Motiva writes (at 52) that “the undisputed evidence establishes that Motiva did not pursue litigation as a mere patent owner.” According to Motiva, “it is important to remember that
Motiva’s litigation effort was in further service of its licensing goals.”
42 Case: 12-1252 Document: 46 Page: 53 Filed: 08/29/2012
Br. 52. But it is Motiva that has amnesia. The ALJ found that “the litigation against Nintendo does not relate to exploitation of the asserted patents.” A7832. If this litigation is “not about licensing,”
A30,364, then it cannot be evidence of Motiva’s “substantial investment in . . . licensing,” 19 U.S.C. § 1337(a)(3)(C).
Motiva’s litigation-inspired arguments also forget a host of other adverse factual findings. Motiva says (at 54) that the evidence showed
“Motiva and several business partners alike all saw the Wii as a substantial roadblock to Motiva’s success.” But, as detailed further below (at 47-49), the ALJ rejected this view of events. Motiva “failed to demonstrate that there was any decline in interest caused by the release of the Wii.” A7815. Motiva invokes the wisdom of its founders on the necessity of litigation. Br. 54. But the ALJ specifically rejected this assertion: “I do not concur with the inventors’ assertion that because of the Wii, they had no other recourse but to sue Nintendo.”
A7824. Motiva notes that it “invested in the litigation,” Br.
58, but the ALJ ruled this time “insubstantial.” A7831; see also A7817
(“I give no weight to this wholly unsupported testimony from clearly interested witnesses.”).
43 Case: 12-1252 Document: 46 Page: 54 Filed: 08/29/2012
Motiva’s “public policy” points in support of its ‘litigation alone is enough’ theory of ITC authority misconstrues what the ALJ held.
Motiva states that the ALJ “implicitly held that when a company seeks financial compensation for patent infringement, as well as injunctive relief, it necessarily belies a bona fide domestic industry.” Br. 55. But the ALJ made no such global ruling, “implicitly” or otherwise. The ALJ pointed to Motiva’s failure to invoke certain procedures, such as a preliminary injunction or exclusion order. Those procedures are designed to protect industry. Motiva’s decision not to invoke those protective procedures is a decision that confirms the lack of an industry to protect. Where, as here, the complainants are on record as excited about their litigation “winnings” and have ended any efforts to license or commercially exploit the patents, a patent owner cannot cite litigation focused on monetary relief as the sole evidence of steps toward establishing an industry.
Motiva cites Certain Multimedia Display and Navigation Devices and Systems, Components Thereof, and Products Containing the Same,
Inv. No. 337-TA-694, USITC Pub. No. 4292 (Nov. 2011) (Final), for the premise that its litigation efforts should have been given more weight in
44 Case: 12-1252 Document: 46 Page: 55 Filed: 08/29/2012
the ALJ’s domestic industry analysis. In Certain Multimedia, the ITC held that “section 337(a)(2) . . . requires complainant’s activities relate to licensing.” Id. at 13. Here, however, the ALJ found as a matter of fact that Motiva failed to establish any connection between its litigation efforts and any licensing. No matter how frequently Motiva asserts that it is in the licensing business, the ALJ found as a matter of fact that its litigation against Nintendo had nothing to do with licensing.
A7830-31. Litigation unrelated to licensing, as Motiva concedes, cannot establish a domestic industry. Br. 52.
Motiva’s reliance on Certain Digital Satellite System Receivers and
Components Thereof is similarly misplaced. Inv. No. 337-TA-392,
USITC Pub. No. 3418, at 11 (Apr. 2001) (Initial and Recommended
Determinations). The patentee in Certain Digital Satellite had reached license agreements to the patent-in-suit with four licensees and every single employee of the patentee was “responsible for maintaining [its] system of identifying, approaching, and negotiating with prospective licensees.” Id. at 10-11. As Motiva acknowledges, the Certain Digital
Satellite litigation efforts were “an extension of its licensing program.”
45 Case: 12-1252 Document: 46 Page: 56 Filed: 08/29/2012
Id. at 11 (emphasis added). But here, there is no licensing program other than the litigation effort.
C. The Wii System’s Success Does Not Suggest Motiva Has Any Chance Of Establishing A Domestic Industry In The Future
The ALJ ruled that “Motiva could not demonstrate that there was any significant interest in its technology prior to the Wii’s existence,” and “there is no reason to believe that manufacturers of fitness and rehabilitation equipment will suddenly become interested in the technology because Nintendo has been excluded from the market.”
A7835. In addition, the ALJ found that
A7836.
Against these dispositive findings, Motiva makes three limited and erroneous points.
Much of Motiva’s brief turns on its contention that “[b]ut for the presence of the Wii in the market, Motiva was—and still is—extremely close to realizing its product-driven licensing goals.” Br. 44; see Br. 41-
42, 58-59. For example, Motiva cites the testimony of Reiss (he “would still be willing to invest in a license to Motiva’s technology today if
Motiva has the power to exclude Nintendo”) and of Highsmith (“there is
46 Case: 12-1252 Document: 46 Page: 57 Filed: 08/29/2012
potential for a product that has [Motiva’s] characteristics”). Br. 44.
Motiva argues that “[s]ubstantial evidence supports a conclusion that
Motiva’s invention competes with the Wii.” Br. 50.
The ALJ explained why “the Wii Fit [an exercise game] would not compete with the Motiva product.” A7818. “Even though the Wii Fit may be viewed by some as a fitness product, it is not a serious fitness product like the one described in the Motiva documentation.” Id.
“Based” on this review of the evidence, the ALJ found “that the Motiva product was intended to be an expensive tool used by people in the physical rehabilitation and fitness industries, while the Nintendo Wii is a relatively inexpensive video game system for home consumers that is intended to appeal to a mass market.” A7819. See, e.g., A20,708-09 (“Q:
Now, do you think a normal American consumer is going to pay for a device and put it in their living room to play video games? A: No.”).
Motiva makes no effort to dispute these substantial facts the ALJ relied on to find no competition between the Wii System and any Motiva product.
Moreover, any argument that the November 2006 introduction of the Wii System doomed Motiva’s domestic industry ignores the ALJ’s
47 Case: 12-1252 Document: 46 Page: 58 Filed: 08/29/2012
findings. “The proper date for determining whether [Motiva’s licensing efforts] constituted an ‘industry’ entitled to protection under section 337 was the date on which the complaint was filed.” Bally/Midway, 714
F.2d at 1121. Motiva’s complaint was filed in September 2010. A8759.
But, as detailed above at 25, the ALJ properly found that Motiva’s efforts, if any, to establish a domestic industry had ceased by 2007.
A7829. Any effect of the Wii System on Motiva’s prospects must have likewise ended by 2007, three years before the complaint was filed here.
The ALJ also specifically rejected the fiction that Motiva’s licensing efforts were hurt by the introduction of the Wii System.
“Neither Mr. Reiss nor Motiva has offered any evidence to support the assertion that [Mr. Reiss’s company] would license the Motiva patents, but for the presence of the Wii.” A7812. Instead, the ALJ found that
Mr. Reiss’s “real interest lies in excluding Nintendo from the market” to protect his “physical therapy device” from competition from the Wii Fit game. A7812-13 (citing A20,633-34). And the ALJ found that Mr.
Highsmith had “little interest” in Motiva; he never viewed the prototype or patent application. A7814; see also A7815 (“[E]ven though Mr.
Highsmith was aware of Motiva’s technology prior to the Wii’s release,
48 Case: 12-1252 Document: 46 Page: 59 Filed: 08/29/2012
he never showed enough interest to view the prototype, view Motiva’s patents or patent applications, hold a meeting with the inventors in
[his] offices, or present the invention to [his company’s] innovation committee.”).8
Motiva incorrectly states (at 59-60) that the ALJ’s findings are
“premised largely” on its noninfringement ruling. According to Motiva, should this Court find that the Wii System infringes the Motiva patents
(it does not, see below at Section II), then the ALJ’s analysis of the prospects of Motiva’s products also falls. But the ALJ expressly stated that its factual findings were independent of the infringement ruling:
“Motiva has not demonstrated that there is a significant likelihood that the industry requirement will be satisfied in the future, and this determination is not dependent on the Commission’s actions in this investigation.” A7834 (emphasis added). The ALJ noted testimony by Motiva’s inventors to the contrary, but the ALJ did “not find this testimony to be persuasive,” A7835, dismissing it as “self- interested testimony of the inventors.” A7834. In short, regardless of
8 Among other things, the ALJ also rejected Motiva’s “claim that the technology is currently ready to be commercialized.” A7820 (“the Motiva prototype was not close to being production-ready”).
49 Case: 12-1252 Document: 46 Page: 60 Filed: 08/29/2012
whether the Wii System does not or does infringe Motiva’s patents, the
ALJ found that Motiva still had no prospects of marketplace success.
No one was interested before the Wii System arrived, and no one would be interested if the Wii System suddenly disappeared from the market.
For all these reasons, the ALJ was correct to find that the Wii
System’s success had nothing to do with the failure of Motiva’s technology. But in any event, the Court need only find that substantial evidence supports that conclusion. This Court affirms agency findings where substantial evidence supports several competing conclusions.
Nippon Steel Corp. v. United States, 458 F.3d 1345, 1352
(Fed. Cir. 2006) (“‘[E]ven if it is possible to draw two inconsistent conclusions from evidence in the record, such a possibility does not prevent [the Commission’s] determination from being supported by substantial evidence.’” (quoting Am. Silicon Techs. v. United States, 261
F.3d 1371, 1376 (Fed. Cir. 2001))).
50 Case: 12-1252 Document: 46 Page: 61 Filed: 08/29/2012
II. SUBSTANTIAL EVIDENCE SUPPORTS THE ALJ’S FINDING THAT THE WII SYSTEM DOES NOT INFRINGE MOTIVA’S PATENTS
In light of how the Wii System actually operates, substantial evidence supports the ITC’s conclusion that the Wii System does not infringe Motiva’s patents and must be affirmed. First, the Wii System does not “track” movement of the user—a fact which, by itself, is a basis on which to affirm. Second, and independently, the Wii System does not determine user “movement.” It does not determine the position or orientation of the user (under the ALJ’s construction). Nor does it determine the position and orientation of the user (under the proper construction Motiva incorrectly says the ALJ actually applied).
Motiva’s scattershot effort to challenge these rationales is unpersuasive.
A. Substantial Evidence Shows That The Wii System Does Not “Track” Movement Of A User
Substantial evidence supports the Commission’s determination that the Wii System does not “track” movement of a user. A7777-82.
There was no dispute as to what one of skill in the art understood
“tracking” movement to mean. The only witness to testify on this topic,
Professor Colgate, explained that
51 Case: 12-1252 Document: 46 Page: 62 Filed: 08/29/2012
A7777
(citing A30,437-38); see also A10,320-21. This is also how the patents and others of skill in the art use the term “tracking.” A10,321-30.
The ITC’s holding—that the Wii lacks each of the three elements that define tracking—is supported by the documents and the testimony.
First,
A7777; see A30,430-33.
A7777-78.
A30,430.
Second, the ALJ found that the Wii does not measure user movement with A7777.
A7777.
52 Case: 12-1252 Document: 46 Page: 63 Filed: 08/29/2012
Third, the ALJ found that the Wii System did not keep any
Id. In an effort to contradict this finding, Motiva invokes its expert, Dr. Jaswinder Pal
Singh,
A9920; see also A7779. But the ALJ found that Dr. Singh was not qualified as an expert on motion-sensing devices (A20,154); he was an expert in computer systems and electrical engineering. A20,153.
The ALJ was justified in finding that Nintendo manager
A7779.
Id. (citing A30,566, 30,592, 30,593-94, 30,596).
A7779.
Id. (emphasis in original).
53 Case: 12-1252 Document: 46 Page: 64 Filed: 08/29/2012
Simply put, tracking a fictional position in a three-dimensional virtual world is not the same as tracking the human playing the game.
Motiva argues that the ALJ’s construction of “tracking” “directly contradicts the Commission’s apparent construction of ‘tracking movement of a user.’” Br. 24-25 (citing A7674 at ¶¶ 3-4). Nonsense.
The cited paragraphs do not construe the term “tracking.” The paragraphs construe “tracking movement of a user” to mean “tracking changes of position and/or orientation of a user.” A7674 (emphasis added). So as relevant here, all that one can glean from this claim construction is that the Commission said “tracking” means “tracking.”
The bottom line is that the Wii System does not “track” movement, and nothing in Motiva’s brief casts doubt on that fact. This alone requires affirmance.
B. Substantial Evidence Shows That The Wii System Does Not Determine User “Movement”
In light of the above, this Court need not proceed to the next step and address whether the Wii System determines user “movement” as required by Motiva’s patents. But should the Court reach the issue, the answer is “No”: The ALJ correctly found that the Wii sensors do not determine user “movement,” i.e., “changes of position and/or orientation
54 Case: 12-1252 Document: 46 Page: 65 Filed: 08/29/2012
of a user” (or, consistent with Nintendo’s construction, changes of position and orientation of a user). A7772. Motiva’s attacks on that finding are meritless.
1. Three Credible Witnesses Demonstrated That The Wii Sensors Cannot And Do Not Determine Position Or Orientation
To understand why the Wii System does not determine the position or orientation of the user, it is useful to understand how the
Wii System actually works. As anyone who has played it or watched the videos submitted with this brief might ask, isn’t the whole fun of the
Wii System that it responds to the player’s movement? See A7752
(“90% of consumer purchase decisions are based upon the Wii’s ability to track motion”). In fact, the Wii System “trick[s]” the user into thinking it knows the user’s movement. A20,439. The ALJ found that the Wii “create[s] a convincing display of character movement on the screen through animations based on real time computations that merely give the appearance of tracking player movements.” A7780 (emphasis added).
The ALJ found Nintendo’s witnesses credible:
55 Case: 12-1252 Document: 46 Page: 66 Filed: 08/29/2012
A7780. Mr. Ohta participated in the development of the Wii prototype and designed many of the Wii’s most popular games, including Wii
Sports. A21,395, 21,401. He explained, in testimony endorsed by the
ALJ, how the illusion of tracking user movement was implemented in the popular Wii Tennis game.
56 Case: 12-1252 Document: 46 Page: 67 Filed: 08/29/2012
Thus, the Wii System determines which tennis animation to display without determining either the position or orientation.
A7762 (citing A21,479-80, 21,482, 30,575). See also SVA at RDX-156.
The record includes various other examples that confirm the Wii
System works without tracking the user’s position or orientation.
57 Case: 12-1252 Document: 46 Page: 68 Filed: 08/29/2012
Dr. Colgate explains:
A30,452.
So too with boxing. As demonstrated in the Wii Boxing video,
SVA at RDX-154, the
58 Case: 12-1252 Document: 46 Page: 69 Filed: 08/29/2012
In sum, the Wii System does not determine either the position or orientation of the user.
2. Motiva’s Hodgepodge Of Movement And Position Information Arguments Are All Meritless
Regarding the alleged infringement of the movement claim term,
Motiva raises a host of scattered arguments that are all meritless.
1. In an attempt to circumvent the ALJ’s findings, Motiva now emphasizes that the gyroscope measures the “changes” in orientation by measuring angular velocity. Motiva asserts that the ALJ erred by evaluating “infringement based on whether the Wii determines the
‘position or orientation’ of the user instead of changes in the position and/or orientation of a user.” Br. 22 (Motiva’s emphasis). Motiva argues that the Wii’s gyroscope detects “changes in orientation over time.” Id. at 23 (Motiva’s emphasis).
59 Case: 12-1252 Document: 46 Page: 70 Filed: 08/29/2012
But it is Motiva’s arguments that have changed. Before trial and at the hearing, Motiva argued that the Wii System “determine[s]”
(A30,129) or “describes” (A9797) the orientation of the transponder.
Only later, in the petition to the ITC, did Motiva advance its newfound theory that what matters is that the Wii System determines orientation
“changes.” See, e.g., A8602. The law is clear that arguments not raised before the ALJ are waived. Finnigan Corp. v. ITC, 180 F.3d 1354, 1363
(Fed. Cir. 1999) (“The argument at the trial and appellate level should be consistent, thereby ensuring a clear presentation of the issue to be resolved.”).
In any event, the Wii System does not measure changes in the orientation or position of the user.
It is undisputed that gyroscopes detect only “angular velocity.”
Br. 13. Angular velocity is the speed of rotation around an axis. A7755; see A20,160; 20,174.
A20,160, 20,174; see also A30,423.
60 Case: 12-1252 Document: 46 Page: 71 Filed: 08/29/2012
A7775 (citing A20,412-15). Motiva agrees that the gyroscope “cannot detect where an object is facing at a particular moment in time.” Br. 13. The ALJ also embraced testimony from
A7776-77; see also A30,423-24, 30,430-31.
A7776-77.
Motiva argues that the accelerometers determine “changes in position.” Br. 14 (emphasis added). It is undisputed that accelerometers do nothing but measure acceleration. As with the orientation argument, Motiva’s “changes” position argument is new and
61 Case: 12-1252 Document: 46 Page: 72 Filed: 08/29/2012
waived. See, e.g., A20,194
Moreover, the ALJ cited substantial evidence in support of the finding that the accelerometers do not determine position or any changes in position.
A7775.
A20,285
The ALJ also again credited Dr. Colgate’s testimony.
A7774.
A7775.
A7774.
Gravity provides a constant acceleration on every object; all objects
62 Case: 12-1252 Document: 46 Page: 73 Filed: 08/29/2012
register acceleration even when not moving. Id. (citing A30,417-18).
Therefore, one cannot simply calculate out the gravitational acceleration because, for example,
A30,553.
A7775 (citing A20,234).
To the extent Motiva argues that the accelerometers determine orientation,
A7776. Motiva’s patents concede that accelerometers can only be used to determine pitch and roll when the device is “static.” A30 at 33:60-61; see also A30,418.
Furthermore, the ALJ found that Mr. Rabin
A7775. The ALJ noted that Mr. Rabin had explained his presentation to Wii developers
63 Case: 12-1252 Document: 46 Page: 74 Filed: 08/29/2012
A7775-76. But the ALJ noted that Mr. Rabin had
A7776. In other words,
Id.
And the ALJ found that
Id.
Id. Of course, measuring only pitch and roll of a stationary transponder cannot determine user movement.
In short, Motiva’s last-minute change in argument would be unpersuasive even if it had not been waived.
Moreover, Motiva’s new infringement argument still requires
“tracking”—something the Wii System does not do. See above at II.A.
2. Next, Motiva argues that “movement” and “position” do not necessarily involve movement or position in all three spatial
64 Case: 12-1252 Document: 46 Page: 75 Filed: 08/29/2012
dimensions. Br. 15-18, 26. It is enough, Motiva argues, if a device can figure out “that an object is ‘above’ or ‘to the left of’ another object.” Id. at 26.
The ALJ’s opinion demonstrates why Motiva is wrong to insist that two-dimensional detection is enough under the claims. The ALJ noted that the patent “specification explains that the invention may be used ‘for the purposes of functional movement assessment for exercise, and physical medicine and rehabilitation.’” A7678. “Such tracking,” the ALJ held, “requires knowledge of the user’s location in 3D space.”
Id. A “user” is a human being, not a cartoon, which means that the user resides, and moves, in three-dimensional space. Tracking in fewer than three dimensions is not tracking “the user” as the claims require.
“Motiva cites no evidence from the ‘151 patent that supports a finding that the claimed invention may track position in only one or two dimensions.” A7679. Thus, the ALJ found “it appropriate to include express reference to 3D space in the proposed construction.” A7678.
Motiva contends that the plain and ordinary meaning of position and movement information does not require three-dimensional tracking. In support of this view, Motiva invokes (at 26) Retractable
65 Case: 12-1252 Document: 46 Page: 76 Filed: 08/29/2012
Technologies v. Becton, Dickinson & Co., 653 F.3d 1296, 1306
(Fed. Cir. 2011), and (at 19) Northrop Grumman Corp. v. Intel Corp.,
325 F.3d 1346, 1355-56 (Fed. Cir. 2003). Those cases stand for the rule that “statements from the description of the preferred embodiment . . . are just that—descriptions of a preferred embodiment” and that such statements do not disavow claim scope absent a clear disclaimer.
Northrop Grumman, 325 F.3d at 1356; see also Retractable, 653 F.3d at
1306 (specification disclosed an embodiment that operates by “cutting” and mere statements about difficulties with prior art “cutting” type devices did not disclaim claim scope).
This rule, however, has no bearing here because the ALJ did not find that the asserted patents disclosed and then disclaimed embodiments that track in one or two dimensions. Rather, the ALJ found that every embodiment of “the invention” tracks the user in three dimensions, and Motiva cited “no evidence . . . that the claimed invention may track position in only one or two dimensions.” A7679.
This finding is confirmed by the specification’s statement that “[t]he present invention described can be used . . . to measure frequency and amplitude of body sway in three dimension (3D) space.” A21 at 16:8-13.
66 Case: 12-1252 Document: 46 Page: 77 Filed: 08/29/2012
And it is further confirmed by the statement that “[t]he present invention” can be used to determine whether an “individual’s movement trajectory varied from the intended two dimensional (2D) reference movement trajectory by deviation from the planar path into the uninvolved spatial dimension,” which would require tracking in three dimensions. A22 at 18:3-16. Thus, where the specification recites that
“the invention” measures in three dimensions and only discloses embodiments that track the user in three dimensions, tracking movement and position must be in three dimensions. That is the only construction that properly “tether[s] the claims to what the specifications indicate the inventor actually invented.” Retractable, 653
F.3d at 1305.
None of Motiva’s arguments come close to overcoming the conclusion compelled by the specification. Motiva cites (at 15-16, 20) to a “dual axis inertial sensor” in a preferred embodiment. But the specification explains that this sensor is optional; the sensor is used to provide supplemental pitch and roll information. A28 at 30:43-31:50; see also A30,538. Moreover, the asserted patents acknowledge that such dual axis inertial sensors can only measure pitch and roll when
67 Case: 12-1252 Document: 46 Page: 78 Filed: 08/29/2012
“static,” making them useless for tracking movement. A30 at 33:60-61.
Motiva mischaracterizes a “Limits of Stability” test by asserting that it measures body sway in just two dimensions. Br. 16. The specification states twice that “body sway [is measured] in three dimension [sic] (3D) space.” A21 at 16:12-13, 22 at 7:23-24.
See A30,539-40
Motiva also mischaracterizes the specification by stating that it
“describes tracking using ‘mechanical, inertial, acoustical or electromagnetic radiation sensors.’” Br. 19. The passage that Motiva cites, however, concerns the prior art—not the claimed invention:
“Known are commercial tracking and display systems that employ . . . mechanical, inertial, acoustical or electromagnetic radiation sensors to determine a mobile object’s position and orientation, referred to collectively as pose.” A14 at 1:18-21.
68 Case: 12-1252 Document: 46 Page: 79 Filed: 08/29/2012
Motiva also cites to three prior art references that allegedly disclose tracking humans in fewer than three dimensions. Br. 16-18.
As the ALJ recognized, however, Motiva’s citations to the prior art merely shift the focus away from the central inquiry: what one skilled in the art would understand “tracking” a “user” to mean in the asserted patents. A7678-79.
As explained above, supra at 60-64 neither the gyroscopes nor the accelerometers determine, much less track, position or orientation.
Motiva also emphasizes the Wii System’s camera, the DPD. Br. 37-39.
But, again, the ALJ’s fact findings are dispositive.
The ALJ had substantial evidence to support the finding that the
DPD does not determine position or orientation in any dimension. Dr.
Colgate
A7773.
A7774.
Thus, as demonstrated at the hearing,
69 Case: 12-1252 Document: 46 Page: 80 Filed: 08/29/2012
A7773. Indeed,
Id.
3. Motiva’s brief includes an attempt to circumvent its burden under the substantial evidence standard by claiming that the ALJ first construed “movement information” correctly but then “applied a construction for ‘movement information’ that was inconsistent with its initial construction.” Br. 32.
Motiva is wrong on the law, wrong on what the ALJ did here, and wrong on the correct claim construction.
Motiva asserts that when an ALJ construes terms in one fashion and then applies them in an arguably different fashion, the de novo standard of review applies. Id. at 32. In support of its argument,
Motiva incorrectly describes the holding of Vita-Mix Corp. v. Basic
Holding, 581 F.3d 1317 (Fed. Cir. 2009). Motiva contends that Vita-Mix stands for the proposition that “a lower court’s application of a construction that is ‘inconsistent with its earlier claim construction’ is
70 Case: 12-1252 Document: 46 Page: 81 Filed: 08/29/2012
[reviewed] de novo to determine correct construction.” Br. 32. Vita-Mix, however, says no such thing. Vita-Mix was an appeal from a summary judgment of non-infringement, and thus the Court merely applied the settled rule that grants of summary judgment are reviewed de novo.
Vita-Mix, 581 F.3d at 1323.
More instructive here is Tessera, Inc. v. ITC, 646 F.3d 1357 (Fed.
Cir. 2011), which Motiva asserts, without explanation, does not apply.
Br. 32-33. Like Motiva, Tessera won its claim construction, but the ITC nevertheless found no infringement. Tessera, 646 F.3d at 1363-64.
Like Motiva, Tessera appealed to this Court and argued that the ITC
“initially adopted a correct claim construction, but ‘halfway through its infringement analysis, the ITC inexplicably switched to an incorrect claim construction’” and, thus, the ITC’s finding of noninfringement should be reviewed “de novo, as an error in claim construction.” Id. at
1364. The Court, however, found that Tessera could not argue that the
ITC applied the wrong construction because it adopted Tessera’s proposed construction and thus its “contention at best [was] a disagreement over the Commission’s application of Tessera’s construction to the accused . . . devices.” Id. Thus, the Court held that
71 Case: 12-1252 Document: 46 Page: 82 Filed: 08/29/2012
Tessera’s appeal was a challenge of the ITC’s “infringement determination,” and should be “review[ed] for substantial evidence.” Id.
For the same reasons, the substantial evidence burden applies here.
In any event, the ALJ in fact applied the construction of “tracking movement of a user” as “tracking changes of position and/or orientation of a user.” The ALJ adopted Motiva’s construction of “movement information,” construing it to mean “information about changes in position and/or orientation.” A7674, 7676. The ALJ referred to this construction at the outset of the infringement discussion: “The question becomes whether or not the accused products ‘track changes of position and/or orientation of a user.’ I find that they do not.” A7772. The ALJ then described Motiva’s arguments regarding the gyroscope, accelerometer, and DPD, and held they “do not, in fact, track the movement of the user or provide information regarding the position or orientation of the user.” A7773 (emphasis added).
A7777. And, indeed, that is Nintendo’s view. But the ALJ ruled differently. A7674.
72 Case: 12-1252 Document: 46 Page: 83 Filed: 08/29/2012
Nothing in this accurate recitation of Dr. Colgate’s testimony suggests that the ALJ had a change of heart.
A7777.
The ALJ was citing Dr. Colgate for the point that the Wii System determines neither position nor orientation.
Nintendo continues to disagree with the ALJ’s construction of movement. But the Court can and should resolve this appeal without deciding whether the claims require determining both position and orientation. As the ALJ held, the Wii System does not track either a user’s position or a user’s orientation. A7773. There is no need to decide whether a device that tracks only position would infringe the claims.
4. Motiva repeatedly invokes the reexamination process of its patents. Br. 14, 16, 18, 19-20, 27-31. But the reexamination is ongoing and in any event not meaningfully relevant here.
Motiva emphasizes its “own actions” during reexamination. Br.
27-30. According to Motiva, its decision to press on with rejected claims rather than canceling or amending the claims suggests that it did not disavow the broader claim scope. Br. 28. But nothing in the ALJ’s
73 Case: 12-1252 Document: 46 Page: 84 Filed: 08/29/2012
claim construction turns on whether or not Motiva disavowed any claim scope.
Motiva complains (at 28) that the ALJ should not have disregarded the (still incomplete) reexamination record. Motiva cites cases that stand for the proposition that a patent owner’s statements during an incomplete prosecution can limit the claims.9 The cases also suggest that any points raised by an examiner in a prosecution can be evaluated for substantive persuasiveness.10
Under this authority, the ALJ’s decision to give little weight to the incomplete examination was proper. Motiva seeks to use incomplete proceedings to expand rather than narrow its claims. And, as the ALJ
9 See, e.g., In re Katz Interactive Call Processing Patent Litig., 07-ml- 01816-BRGK (FFMx), 2008 WL 4952454, at *5 (C.D. Cal. Feb. 21, 2008) (“[T]he prosecution history can often inform the meaning of the claim language . . . making the claim scope narrower than it would otherwise be.”) (emphasis added), aff’d 639 F.3d 1303 (Fed. Cir. 2011); Beneficial Innovations, Inc. v. Blockdot, Inc., Nos. 2:07-CV-263-TJW-CE, 2:07-CV- 555-TJW-CE, 2010 U.S. Dist. LEXIS 54151, at *8 (E.D. Tex. June 3, 2010) (“[S]tatements made by a patentee during reexamination to distinguish a claim from the prior art may serve to limit the scope of the claim.”) (emphasis added). 10 See, e.g., St. Clair Intellectual Prop. Consultants, Inc. v. Canon Inc., 412 F. App’x 270, 276-77 (Fed. Cir. 2011); SRAM Corp. v. AD-II Eng’g, Inc., 465 F.3d 1351, 1357, 1359 (Fed. Cir. 2006) (rejecting construction adopted by examiner after three reexaminations).
74 Case: 12-1252 Document: 46 Page: 85 Filed: 08/29/2012
found, “the statements made by the Examiner regarding claim construction are not persuasive.” A7679; see also supra II.A. Motiva repeatedly notes (at 14, 16, 18, 19-20) that the examiner adopted
Nintendo’s definition of “movement.” But the statements that Motiva cites were made under the express qualification that these are not
Nintendo’s arguments for the purposes of litigation because federal courts use a narrower standard than the PTO in reexamination. The
ALJ’s treatment of the reexamination proceedings provides no basis for reversal.
CONCLUSION
For the foregoing reasons, the ITC’s decision should be affirmed.
75 Case: 12-1252 Document: 46 Page: 86 Filed: 08/29/2012
Date: August 29, 2012 Respectfully submitted,
/s/ Mark S. Davies Mark S. Davies Katherine M. Kopp ORRICK, HERRINGTON & SUTCLIFFE LLP 1152 15th Street, N.W. Washington, D.C. 20005 (202) 339-8400 [email protected]
E. Joshua Rosenkranz Peter A. Bicks Alex V. Chachkes ORRICK, HERRINGTON & SUTCLIFFE LLP 51 West 52nd Street New York, NY 10019 (212) 506-5000
Joseph S. Presta Robert W. Faris NIXON & VANDERHYE P.C. 901 North Glebe Road 11th Floor Arlington, VA 22203 (703) 816-4000
Attorneys for Nintendo Co., Ltd. and Nintendo of America Inc./Intervenors
76 Case: 12-1252 Document: 46 Page: 87 Filed: 08/29/2012
CERTIFICATE OF COMPLIANCE UNDER FEDERAL RULES OF APPELLATE PROCEDURE 32(A)(7) AND FEDERAL CIRCUIT RULE 32
Counsel for Intervenors Nintendo Co., Ltd. and Nintendo of
America Inc. certify that the brief contained herein has a proportionally spaced 14-point typeface, and contains 13,791 words, based on the
“Word Count” feature of Word 2007, including footnotes and endnotes.
Pursuant to Federal Rule of Appellate Procedure 32(a)(7)(B)(iii) and
Federal Circuit Rule 32(b), this word count does not include the words contained in the Certificate of Interest, Table of Contents, Table of
Authorities, Abbreviations, and Statement of Related Cases.
Dated: August 29, 2012 Respectfully submitted,
/s/ Mark S. Davies Mark S. Davies
77 Case: 12-1252 Document: 46 Page: 88 Filed: 08/29/2012
CERTIFICATE OF SERVICE
I certify that on the 29th day of August, 2012, the foregoing Non-
Confidential Answering Brief of Intervenors Nintendo Co., Ltd. and
Nintendo of America Inc. was electronically filed using the Court’s
CM/ECF System, which will automatically serve all counsel of record.
I further certify that two copies of the foregoing Non-Confidential
Answering Brief of Intervenors Nintendo Co., Ltd. and Nintendo of
America Inc. will be served at the time the paper copies of the brief are submitted to the Court:
Clark S. Cheney Wayne W. Herrington Dominic L. Bianchi U.S. International Trade Commission 500 E Street, S.W., Suite 707 Washington, D.C. 20436 [email protected]
Counsel for Appellee International Trade Commission
Christopher D. Banys Lanier Law Firm, P.C. 2200 Geng Rd. Suite 200 Palo Alto, CA 94303 [email protected]
Counsel for Appellant Motiva, LLC
/s/ Mark S. Davies Mark S. Davies Case: 12-1252 Document: 46 Page: 89 Filed: 08/29/2012
Addendum Case: 12-1252 Document: 46 Page: 90 Filed: 08/29/2012
Motiva, LLC v. International Trade Commission Fed. Cir. Appeal No. 2012-1252
Index to Addendum
DOCUMENT PAGE NO.
Patent No. 7,292,151, Dated November 6, 2007...... A1-35
Patent No. 7,492,268, Dated February 17, 2009 ...... A36-67 Case: 12-1252 Document: 46 Page: 91 Filed: 08/29/2012
JX-001 ,-,
UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office
June 08, 20lO
THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COpy FROM THE RECORDS OF THIS OFFICE OF:
U.S. PATENT: 7,292,151 ISSUE DATE: November 06,2007
By Autbority of tbe Under Secretary of Commerce for Intellectual Property and Director of tbe United States Patent and Trademark Office
A000001 Case: 12-1252 Document: 46 Page: 92 Filed: 08/29/2012
JX-001.0002
111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111111111111111111111111111111111111111111111111111111111 US007292151B2
(12) United States Patent (10) Patent No.: US 7,292,151 B2 Ferguson et al. (45) Date of Patent: Nov. 6,2007
(54) HUMAN MOVEMENT MEASUREMENT 4,912,638 A 3/1990 Pratt ...... 600/595 SYSTEM 4,925,189 A 5/1990 Braeunig ...... 273/148 B 5,148,154 A 9/1992 Mackay et al...... 3401712 (76) Inventors: Kevin Ferguson, 8156 Camp den Lakes 5,184,295 A 2/1993 Mann ...... 364/410 Blvd., Dublin, OH (US) 43016; Donald 5,214,615 A 5/1993 Baur ...... 367/128 Gronachan, 4 Spiral Rd., Holtsville, 5,227,985 A 7/1993 DeMenthon ...... 364/559 NY (US) 11742 5,229,756 A 7/1993 Kosugi et al...... 340/706 5,239,463 A 8/1993 Blair ...... 463/3 ( *) Notice: Subject to any disclaimer, the tenn of this 5,255,211 A 10/1993 Redmond ...... 365/401 patent is extended or adjusted under 35 5,288,078 A 2/1994 Capper et al...... 273/148 B U.S.c. 154(b) by 47 days. 5,320,538 A 6/1994 Baum ...... 434/307 5,347,306 A 9/1994 Nitta ...... 348/578 (21) Appl. No.: 111187,373 5,372,365 A 12/1994 McTeigue et al...... 273/182.2 5,375,610 A * 12/1994 LaCourse et al...... 600/595 (22) Filed: Jul. 22, 2005
(65) Prior Publication Data (Continued) US 2006/0022833 Al Feb. 2, 2006 FOREIGN PATENT DOCUMENTS Related U.S. Application Data WO PCTlUS96117580 5/1997 (60) Provisional application No. 60/592,092, filed on luI. 29,2004. OTHER PUBLICATIONS (51) Int. CI. G08B 23/00 (2006.01) Reality built for two: a virtual reality tool, Symposium on Interac (52) U.S. CI...... 340/573.1; 340/407.1; tive 3D Graphics, ACM Press webpages from http://portal.acm.org/ citation.cfm?id~91385.91409&dl+ACM&type~series&i (Jun. 10, 434/114 2004) 1-4. (58) Field of Classification Search ...... 340/573.1, 340/573.4,539.12,539.13,539.22,407.1, (Continued) 340/825.36; 3811315; 4341112,114 See application file for complete search history. Primary Examiner-Toan N. Pham (74) Attorney, Agent, or Firm-Standley Law Group LLP (56) References Cited (57) ABSTRACT U.S. PATENT DOCUMENTS
4,337,049 A 611982 Connelly ...... 434/247 4,375,674 A 311983 Thornton ...... 364/559 A system for measuring the position of transponders for 4,627,620 A 1211986 Yang ...... 273/1 GC testing and training a user to manipulate the position of the 4,631,676 A 1211986 Pugh ...... 364/413 transponders while being guided by interactive and sensory 4,645,458 A 211987 Williams ...... 434/251 feedback through a bidirectional communication link to a 4,695,953 A 911987 Blair et al...... 364/410 processing system for the purpose of functional movement 4,702,475 A 1011987 Elstein et aI ...... 273/1 GC assessment for exercise and physical rehabilitation. 4,751,642 A 611988 Silva et aI ...... 364/413 4,817,950 A 411989 Goo ...... 273/148 B 91 Claims, 10 Drawing Sheets
A000002 Case: 12-1252 Document: 46 Page: 93 Filed: 08/29/2012
JX-001.0003
US 7,292,151 B2 Page 2
u.s. PATENT DOCUMENTS 6,400,452 Bl 6/2002 Maynard ...... 356/141.1 6,430,997 Bl 8/2002 French et al...... 73/379.04 5,385,519 A 111995 Hsu et al...... 482/57 6,487,906 Bl * 12/2002 Hock ...... 73/379.01 5,405,152 A 411995 Katanics et al ...... 273/438 6,515,593 Bl * 212003 Stark et al...... 340/870.07 5,423,554 A 611995 Davis ...... 273/437 6,720,876 Bl 4/2004 Burgess ...... 340/568.1 5,429,140 A 711995 Burdea ...... 600/587 6,749,432 B2 6/2004 French et al...... 434/247 5,466,200 A 1111995 Uerich ...... 482/4 6,765,726 B2 7/2004 French et al...... 359/630 5,469,740 A 1111995 French et al ...... 73/379.04 6,774,885 Bl 8/2004 Even-Zohar ...... 345/156 5,474,083 A * 1211995 Church et al...... 600/546 6,834,436 B2 * 1212004 Townsend et al...... 33/512 5,485,402 A * 111996 Smith et al...... 702/160 6,876,496 B2 412005 French et al...... 359/630 5,495,576 A 211996 Ritchey ...... 395/125 200210183961 Al 12/2002 French et al...... 702/150 5,516,105 A 511996 Eisenbrey et al...... 273/148 B 5,524,637 A 611996 Erickson ...... 1281779 OTHER PUBLICATIONS 5,577,981 A 1111996 Jarvik ...... 482/4 5,580,249 A 1211996 Jacobsen et al...... 434/11 Europe is Bursting with Virtual Reality Ideas, But Developers Are 5,584,700 A 1211996 Feldnian et al ...... 434/247 Critically Strapped for Cash, webpages from https:llwww/lexis. comlresearch/retrieve? _m~66dI7057c 1b77fl97 aledb9f5 fadb87 d 5,587,937 A 1211996 Massie ...... 364/578 5,591,104 A 111997 Andrus ...... 48217 &_browseType~Text, (Jan. 1993) 1-2. 5,597,309 A 111997 Riess ...... 434/258 Allard, P., et al, Three-Dimensional Analysis of Human Movement, 5,616,078 A 411997 Oh ...... 463/8 Human Kinetics (1995) 3, 8-14. 5,638,300 A 611997 Johnson ...... 364/551.01 Brownstein, B., et al, Functional Movement in Orthopedic and 5,641,288 A 611997 Zaenglein, Jr...... 434/21 Sports Physical Therapy, Churchill Livingstone (1997), 15. 5,645,077 A 711997 Foxlin ...... 1281774 Brugger, W., et ai, Computer-aided tracking of body motions using 5,656,904 A 811997 Lander ...... 318/568.12 a c.c.d.-image sensor, Med. BioI. Eng. & Comput, (Mar. 1978), 5,659,691 A 811997 Durward et al...... 395/329 207-210. 5,702,323 A 1211997 Poulton ...... 482/8 Codella, c., et ai, Interactive Simulation in a Multi-Person Virtual 5,703,623 A 1211997 Hall ...... 345/158 World ACM (May 3-7, 1992), 329-334. 5,704,837 A 111998 Iwasaki et al ...... 463/38 DeLoura, M., et ai, Game Programming Gems, Charles River Media, (2000) 200-204. 5,711,304 A * 111998 Dower ...... 600/523 5,715,834 A 211998 Bergamasco et al...... 1281782 Greenleaf, W.J., DataGlove, DataSuit, and virtual reality Advanced 5,720,619 A 211998 Fisslinger ...... 434/336 technology for people with disabilities, Proceedings of the Seventh 5,759,044 A 611998 Redmond ...... 434/307 Annual Conference 'Technology and Persons with Disabilities,' 5,785,630 A 711998 Bobick ...... 482/4 (Mar. 1992) 211-214. 5,785,631 A 711998 Heidecke ...... 482/5 Kasvand, T., et al, Computers and the Kinesiology of Gait, Comput. 5,790,076 A 811998 Sypniewski ...... 342/365 BioI. Med. Pergamon Press (1976) vol. 6 111-120. 5,790,124 A 811998 Fischer et al...... 345/435 Kenmochi, A., et al, A network virtual reality skiing system-system 5,792,031 A 811998 Alton ...... 482178 overview and skiing movement estimation, Symbiosis of Human 5,812,257 A 911998 Teitel et al ...... 356/141.4 and Artifact, (Jul. 1995) 423-428. 5,838,816 A 1111998 Holmberg ...... 382/157 Kraus, A., Matrices for Engineers, Hemisphere Publishing Corp. 5,846,086 A 1211998 Bizzi et al...... 434/247 (1987) 118-120, 124-126. 5,850,201 A 1211998 Lasko-Harvill ...... 345/8 Lengyel, E., Mathematics for 3D Game Programming & Computer 5,872,438 A 211999 Roston ...... 318/568.11 Graphics, Charles River Media (2004) 76-78, 467-468. 5,888,172 A 311999 Andras et al...... 48217 Medved, v., Towards a virtual reality-assisted movement diagnos 5,890,995 A 411999 Bobick ...... 482/4 tics-an outline, Robotica (Jan.-Feb. 1994) vol. 12, 55-57. 5,913,727 A 611999 Ahdoot...... 463/38 Mulder, A., Human movement tracking technology, School of Kinesiology, Simon Fraser University (Jul. 1994) 1-14. 5,929,782 A * 711999 Stark et al...... 340/870.01 5,963,891 A 1011999 Walker et al...... 702/150 Ruby, D., Biomechanics-how computers extend athletic perfor 5,989,157 A 1111999 Walton ...... 482/4 mance to the body's far limits, Popular Science (Jan. 1982) 58-60. 6,004,243 A 1211999 Ewart ...... 482/8 Sandweiss, J., et al, Biofeedback and Sports Science, Plenum Press 6,028,593 A 212000 Rosenberg ...... 434/11 New York (1985) 1-201. 6,043,873 A 3/2000 Ramer...... 356/139.03 Scarborough, E.L., Enhancement of Audio Localization Cue Syn 6,050,822 A 4/2000 Faughnn ...... 434/11 thesis by Adding Environmental and Visual Cues, Air Force Inst. Of 6,050,963 A 4/2000 Johnson et al...... 600/595 Tech., Wright-Patterson AFB, OH School of Engineering (Dec. 6,054,951 A 4/2000 Sypniewski ...... 342/465 1992) 1-4. 6,066,075 A 5/2000 Poulton ...... 482/8 Smith, J., et ai, Virtual Batting Cage and Human Model, Virtual 6,073,489 A 6/2000 French et al ...... 73/379.04 Human http://www.cs.berkeley.edu/rcdavis/classes/cs294/, (Jun. 6,077,201 A 612000 Cheng ...... 482/57 17, 2004)1-5. 6,088,091 A 7/2000 Ramer ...... 356/141.5 Zetu, D., et al, Extended range tracking for remote virtual reality 6,098,458 A 8/2000 French et al ...... 73/379.04 aided facility management, Department of Mechanical Engineering 6,100,896 A 812000 Strohecker et al...... 345/427 The University of Illinois at Chicago, http://alpha.me.uic.eduldan/ 6,119,516 A * 912000 Hock ...... 73/379.01 NsfPaper/nsfl.htrnl, (Apr. 19, 2004)1-9. 6,132,337 A 1012000 Krupka ...... 482/8 Codamotion: The science of real-time motion capture and analysis, 6,152,856 A 1112000 Studor ...... 482/8 webpages from http://www.charndyn.comlindex.htrnl. (Apr. 17, 6,162,191 A 1212000 Foxlin ...... 600/595 2004) 1. 6,164,973 A 1212000 Macri ...... 434/247 Irex, Virtual Reality Technologies, webpages from http://www. 6,181,343 Bl 112001 Lyons ...... 345/358 irexonline.comlhow_it_works.htrn, (Apr. 19, 2004) 1-2. 6,183,259 Bl 212001 Macri ...... 434/247 Polhemus, PATRIOT: The Fast and Affordable Digital Tracker, 6,198,528 Bl 3/2001 Maynard ...... 356/141.1 www.polhemus.com. (Feb. 2004) 1-2. Polhemus, LIBERTY: The Forerunner in Electromagnetic Tracking 6,244,987 Bl 6/2001 Ohsuga ...... 482/4 Technology, www.polhemus.com. (May 2003) 1-2. 6,308,565 Bl 10/2001 French et al ...... 73/379.04 Success Story Profile: Innovative Sports Training, Motion Monitor, 6,346,045 B2 212002 Rider ...... 463/31 (2002) 1-2. 6,361,507 Bl 3/2002 Foxlin ...... 600/595 6,366,272 Bl 4/2002 Rosenberg ...... 345/156 * cited by examiner
A000003 Case: 12-1252 Document: 46 Page: 94 Filed: 08/29/2012
JX-001.0004 u.s. Patent Nov. 6,2007 Sheet 1 of 10 US 7,292,151 B2
10
y ------) FIG-1A
y FIG-1B
A000004 Case: 12-1252 Document: 46 Page: 95 Filed: 08/29/2012
JX-001.0005 u.s. Patent Nov. 6,2007 Sheet 2 of 10 US 7,292,151 B2
(-l \-----1----.....) 1 1 1 1 1 1 1 1 1 1 1 ) ~ ..... _-", FIG-2A ~ I FIG-2B
1 kg I , ! I \ FIG-2C
FIG-2D
A000005 Case: 12-1252 Document: 46 Page: 96 Filed: 08/29/2012
JX-001.0006 u.s. Patent Nov. 6,2007 Sheet 3 of 10 US 7,292,151 B2
T • Evaluate msplay PIN Entry I Security & Navigation Requirements Controls "," Request CI) rn +[reqUiredl Authentication ---) .c0 a... [not required] User Session b [identification] 'c :::J 0 CI) Authenticatef _____ J (/) User + User Session [loaded] .. Configure Display D Session UI Graphical Incons of Name & ID • Descriptor -,,' 1 ,,-" ,-" Display Graphical ~ Request Incons of Workload Program ---" Intensity & Limits, Selection .. Measurement User Session Criteria, Stimulus Properties, ect. [setup] rh
I I Process I i D CI) / Program ...---_./ Display rn I 0 I Request Graphical .c Request a... Incons of Session 0.. ---"" " " Parameters ...-:::J Setup " " CI) ~ Request " & Instruction (/) Program ---" Text User Session Setup .. [setup] rh User Session Process I J [setup] rh Session +-----" Process Parameters l Program ...---_./ Parameters l J + + [A or B] ( Deployment Phase ) FIG-3A
A000006 Case: 12-1252 Document: 46 Page: 97 Filed: 08/29/2012
JX-001.0007 u.s. Patent Nov. 6,2007 Sheet 4 of 10 US 7,292,151 B2
( Deployment Phase) i
User Session ------, [save] rh Program [load] Process 1M Deployment ------, Requirements
Evaluate Secondary 1M Display Graphical Requirement Incon for Detail of Modular Length, Weighted, or [not required] Augmentative Force Resistance [required] Attachments
CI) _.c._-.../ CI) o l 1 .J:: / i 0... / i ..... / i c ...... CI) 0 r-~~----~I i --- i i E S i ~ .. "'i5.. CI) + Program Cl Program [Deployment] [Deployment] i Fasten I Fasten i Primary 1M i i i Secondary 1M i i i i i , i i \ i \.. J \. i --r \ .. I \ --T-' '\. Display Graphical Display Incon of Graphical User Body Incon of User Body Location & [ready] [ rea d]y Strap Holster Location & Strap Holster ~echan;sm Mechanism
Calibration Phase FIG-3B
A000007 Case: 12-1252 Document: 46 Page: 98 Filed: 08/29/2012
JX-001.0008 u.s. Patent Nov. 6,2007 Sheet 5 of 10 US 7,292,151 B2
( Calibration Phase)
Process Requirements of 1M Calibration
Determine Secondary 1M Requirement Display GraphicalD. Incon of Primary 1M ~[ not requ;red] Calibration Status [required] Display GraphicalD.
Request se~noc~;a;f 1M .---. Primary 1M Pose ---" ",----+< Calibration Status I .- :B Modification ;" c I r----=-...... --:--"'" ;/;'" E Request? o 1 Secondary 1M Pose ---"" c: Process 1 Q) '--' Program .---. Modification ff) Primary 1M ..c I c .....u ..c Pose [Calibration] c c.. E t 1 o Process Program 1 c: Secondary 1M Modulate 1 '--' J Pose [Calibration] Aural/Visual/Tactile ----' Stimuli rh I I Modulate 1 Aural/Visual/Tactile ---) Evaluate Stimuli Primary 1M rh
Pose tatch.... (_ ... _"'_'"_,, ...... __,.. Evaluate Secondary 1M '-----<0 Match Pose to D. Intial Reference pose~MatCh P--,..."_"_"_"..;" _____ Movement Trajectory '----__--< Match Pose to D. [match] Intial Field
'--__~~ ..... ___ .....,~---[m-t.:.t-c-h] Position
( Execution Phase) FIG-3C
A000008 Case: 12-1252 Document: 46 Page: 99 Filed: 08/29/2012
JX-001.0009 u.s. Patent Nov. 6,2007 Sheet 6 of 10 US 7,292,151 B2
Query ,,---[Iearn]------.. Leam Modality
[normal]
---,, Program [Record]
User Session ---'" I [Setup] I I Q) I In c I .s::. I Process c.. Primary 1M Program• I:: 0 Pose [Execution] :;:::; :::l I 0 Q) I x Process I w Learn I I Terminus I I I [not done] I ---J [done]
Process Program Terminus
Program [not done] [Save] [done] •)
FIG-3D Data Analysis Phase
A000009 Case: 12-1252 Document: 46 Page: 100 Filed: 08/29/2012
JX-001.0010 u.s. Patent Nov. 6,2007 Sheet 7 of 10 US 7,292,151 B2
------L-~ _~--- --=---J-~) ---=--~==---] ,------~ I I I I I / I I I / / / / ..- .,- --,..'"
FIG-4B
A000010 Case: 12-1252 Document: 46 Page: 101 Filed: 08/29/2012
JX-001.0011 u.s. Patent Nov. 6,2007 Sheet 8 of 10 US 7,292,151 B2
Remote Position Processor
Analog Signal Digital Signal Processor Interface Processor Interface
Receiver Processor Constellation
r.t, ~--I TImebase «access» 2 I Sensor &: AID .s I Preamplifier Conversion Peak Detect I Comparator I r---- .... &: I I Sample-Hold ( ------'" Parallel/Serial I «access» «access» I I~- I I/o I I I I I I I r.t, I ~------Digital Filter Communication Interface I --_./ r------I I «1M status» J I I _./ Radio Link L..-U_S_B_2_.0_J..... ------L...-___---I --- FIG-5 20 30 A000011 Case: 12-1252 Document: 46 Page: 102 Filed: 08/29/2012 JX-001.0012 u.s. Patent Nov. 6,2007 Sheet 9 of 10 US 7,292,151 B2 14 FIG-6A FIG-6B FIG-6e A000012 Case: 12-1252 Document: 46 Page: 103 Filed: 08/29/2012 JX-001.0013 u.s. Patent Nov. 6,2007 Sheet 10 of 10 US 7,292,151 B2 Transponder Stimuli Interface Processor I r===n. r.L, 28MHz "k ~ ~ White LED ~ I .... D Timebase r-l & Driver r.L, e..... Vibrator 8 .c::::::;L Motor «clock»,t ~ ~ r.L, & Driver )t. PWM r l) r-l r.L, «access>~ Audio «configure» Annuciator f----' "" & Driver r===n.l ~ r===il T II> r.L, CDu r.L, u Software ,.-- AID c Conversion ~" Calculation Sensor Interface - Engine r.. r===il <)g. r.L, E Ultrasonic =jJ ~""f;9""'J, :::;; ~ Transmitter ------r.L, Parallel/Serial «access» ~ess»T I/o " J r-l r.L, T ~ ':- r.L, Accelerometer ~( :::l 01 Heart Rate t;:: ,.. <: / Receiver 0 "',, u ,.... ¥ T I I Dual Axis I low g I I I I I Communication Interface I Polar compatible r----' ~ uS~»T ~ ----, ?«uLr I/O» r.L, r.L, Graphic LCD ~ & r- Switch I/O Touch Screen ( ) r-l «1M msg» r.L, 20-...." ~'\ n Radio Link FIG-7 30 (1)------_ _ 900MHz ISM A000013 Case: 12-1252 Document: 46 Page: 104 Filed: 08/29/2012 JX-001.0014 US 7,292,151 B2 1 2 HUMAN MOVEMENT MEASUREMENT intrinsic strengths and weaknesses of the tracking technol SYSTEM ogy employed, ultimately the user's satisfaction with the system's utilization and efficacy, including the production of This application claims the benefit of u.s. Provisional reliable, easily understood, measurable outcomes, will dic Application No. 60/592,092, filed luI. 29, 2004, which is tate the overall success of the device. hereby incorporated by reference in its entirety. This invention's system and methods facilitates biome BACKGROUND AND SUMMER OF THE chanical tracking and analysis of functional movement. In INVENTION the preferred embodiment, this invention is low cost, robust, 10 easy to deploy, noninvasive, unobtrusive, and conveys intui This invention relates to a system and methods for setup tive and succinct information to the user to execute move and measuring the position and orientation (pose) of tran ment properly and provides performance indicators of said sponders. More specifically, for training the user to manipu movement for feedback purposes. One feature of the present late the pose of the transponders through a movement invention provides for an interactive tracking system trajectory, while guided by interactive and sensory feedback 15 because the sensor functionality, or referred to herein as means, for the purposes of functional movement assessment active transponders or transponders, is integrated with local for exercise, and physical medicine and rehabilitation. user input control, and real-time sensory interfaces on the Known are commercial tracking and display systems that same device. The transponder is a wireless communication employ either singularly, or a hybrid fusion thereof, and monitoring device that receives a specific signal and mechanical, inertial, acoustical or electromagnetic radiation 20 automatically responds with a specific reply. In one embodi sensors to determine a mobile object's position and orien ment, the invention provides functional movement assess tation, referred to collectively as pose. ment based upon the relative measures of limb pose with The various commercial tracking systems are broadly respect to two positions defined by the transponders. The classified by their relative or absolute position tracking transponders can operate independently or work in unison to capability, in which system the pose of a mobile object is 25 process and share computational tasks and information measured relative to a fixed coordinate system associated between the local databases. This decentralized, distributed with either combination of receiver(s) or passive or active processing scheme allows the configuration and coordina transmitter( s) housing mounted on the user. The tracking tion of the training session, and processing and analysis of system's components may be tethered with obvious inherent the measurements to occur without requiring expensive movement restrictions, or use wireless communication 30 auxiliary computer and display systems to manage the same, means to remotely transmit and process the information and and without relying on costly software development of allow for greater mobility and range of movement. complex synthetic environments for visualization purposes. Typically these tracking systems are utilized for biome Also, the user can manage the applications and performance chanics and gait analysis, motion capture, or performance databases off-line on a remote computer system with Inter- animation and require the sensors to be precisely mounted 35 net connectivity to customize and configure the system on the joints. Various means of presenting the tracking parameters in advance of their session. information in a visual display are employed, such as The present invention is designed to provide such system Heads-Up Display (HUD) , that provide occluded or see and methods for high-fidelity tracking or registration of the through visibility of the physical world, or Fixed-Surface poses of active transponders and engage the user to pur- Display (FSD), such as computer desktop monitors, depend 40 posely manipulate the transponders' pose along a prescribed ing upon the simulation and immersive quality required for or choreographed movement trajectory in order to train and the application. The application may require various degrees assess functional movement capability. In the preferred of aural, visual, and tactile simulation fidelity and construct embodiment, the system is comprised of two subsystems: direct or composite camera views of the augmented or three (1) a subsystem comprised of one or more active transpon- dimensional (3D) virtual reality environment to elicit inter 45 ders, which, in its most sophisticated implementation, active user locomotion and/or object manipulation to responds to periodic requests from another component of the enhance the user's performance and perception therein. The system to radiate or transmit a signal for purposes of tracked object may be represented in the virtual environment absolute position tracking; processes an embedded inertial in various forms, i.e., as a fully articulated anthropoid or sensor for relative orientation tracking and absolute tracking depicted as a less complex graphical primitive. The render 50 refinement; and provides an essentially real-time aural, ing strategy employed depends upon the degree of photo visual, and tactile sensory interfaces to the user, and (2) a realism required with consideration to its computational cost subsystem comprised of a centralized position processor and the application's proprioception requirements. system or unit and receiver constellation unit, collectively Tracking technologies possess certain inherent strengths referred to as the processor unit, which is essentially a signal and limitations dependent upon technology, human factors, 55 processor that synchronizes the transponders' periodicity of and environment that need consideration when discussing radiating signal and other operational states; collectively their performance metrics. Regardless of differentiating receives and processes the radiated signal; iteratively cal resolution and accuracy performance benchmarks, many culates the transponders instantaneous pose and convolu implementations suffer from varying degrees of static and tion, thereof; and continually exchanges this information, dynamic errors, including spatial distortion, jitter, stability, 60 and its analysis thereof, with the transponders and/or aux latency, or overshoot from prediction algorithms. Some iliary host computer system in essentially real-time via a human factors include perceptual stability and task perfor combined wireless and tethered communication means. This mance transparency, which are more subjective in nature. real-time bidirectional exchange of information allows for And environmental issues such as line-of-sight, sensor proper transponder identification, coordination, and the attachment, range, and multiple-object recognition, need to 65 accurate measurement of pose, thereof, and timely actuation be considered when selecting the optimal technology for the of the sensory interfaces for optimal user regulated closed most robust application development. Irrespective of the loop control. A000014 Case: 12-1252 Document: 46 Page: 105 Filed: 08/29/2012 JX-001.0015 US 7,292,151 B2 3 4 The transponder is broadly classified by its level of of movement control or smoothness. In summary, one hardware and software configuration that define its scope of embodiment of the present invention is comprised of: intelligence, sensory support, and configuration. The degree 1) a means to create a single movement vector whose of intelligence is detennined by its capability to locally endpoints are defined by the locations of at least two access, process, and modifY the database. Further, either transponders, wherein, the expansion and contraction of transponder classification can be sub-classified by its the vector's length is calculated, analyzed, and reported in manipulative requirements. In one embodiment, where mul essentially real-time; tiple transponders are used, a principle transponder is con 2) a means to create a single movement vector whose sciously and deliberately moved along the reference move endpoints are defined by the locations of two transpon ders, wherein, a representative point along the vector ment trajectory, while a subordinate transponder serves as an 10 anchor or secondary reference point elsewhere on the loco length is referenced and its higher-order derivatives are motion system whose kinematics are not necessarily con computed by mathematical numerical processes, wherein trolled by the user's volition. the result is calculated, analyzed, and reported in essen tially real-time; and, An interactive transponder, preferably, has significant 15 3) a means to correlate said vector's length and at least one intelligence; supports relative and absolute tracking capa other measure consisting of a higher-order derivative, to bilities; provides complete sensory stimuli support; provides the reference movement trajectory, wherein the result is for functional enhancement through attachment of modular, calculated, analyzed, and reported in essentially real-time. extension pieces; and provides a user display and input A registration system for practical functional movement system to control the training session. In the preferred 20 applications should clearly convey information to the user embodiment, the interactive transponder is primarily held in regarding his movement quality while he perfonns the task, the hand to facilitate more complex user input and greater without compromising or distracting from said execution by sensory intimacy. Conversely, in another embodiment, the uunecessary head movements or change in eye gaze and fixed transponder has limited intelligence; supports only the normal focus. Poor visualization strategies that distract the absolute pose tracking capability; provides no sensory 25 user are ineffectual for promoting heads-up, immersive stimuli support; and is usually mounted to a fixed site on the interaction, and the alphanumerical information it imparts limb or trunk. often can not be consciously processed fast enough to elicit A combination of transponder deployment strategies may corrective action. This system provides for both a local, be required depending on the training session's objectives, standalone sensory interface as a primary feedback aid, or such as two interactive transponders grasped by each hand; 30 alternatively, an interface to a remote fixed-surface display or alternatively, an interactive transponder, and a fixed for greater visualization and simulation capabilities. The transponder attached to the limb or trunk; or lastly, two fixed visual stimulus could be modulated to warn of range viola transponders attached to the limb(s) and/or trunk. tions, or provide signals for purposes of movement cadence In one embodiment, this invention proposes to elicit and directional cueing. A principle interactive transponder is movement strategies based on the deployment of at least two 35 typically hand-held, which is naturally in close proximity to transponders that define the endpoints of a movement vector the user's aural and visual sensory field during most upper whose relative translation and rotation is measured and extremity movements, or, conversely, the visual stimulus evaluated for the assessment of functional movement capa may be viewed through a mirrored or reflective means if not bility, including but not limited to, limb range of motion and in optimal line-of-sight. A remote fixed-surface display its control thereof, limb strength conditioning, and overall 40 might augment the immersive quality of the user's experi proprioception and hand-eye coordination skills, and overall ence by providing control of a view camera of a simulated body movement. This registration system measures a single computer environment, and display of the transponders movement vector whose endpoints are comprised of an and/or interactive objects' static or dynamic poses within the anchor point, i.e. one that is located in a less dynamic frame computer display's skewed through-the-window perspective of reference, e.g., such as the trunk or abdomen, and another 45 projection. In summary, one embodiment of the present more distal location fixed on or held by a limb or extremity, invention is comprised of: e.g., the hand, ann, or leg. As this movement vector is 1) a means for modulating an embedded luminescent display translated and rotated through space by the act of the user organized and oriented into a directional-aiding pattern, modifYing the pose of the principle transponder in concert by varying its degree of intensity and color, or other with the reference movement trajectory, the vector's length 50 physical characteristics, to provide a visual display stimu will expand and contract relative to the proximity of prin lus. This sensory interface is excited at a rate, repetition, ciple transponder with respect to the subordinate transpon or pattern proportional to the pose error of the transpon der. The vector's length conveys unique and explicit infor ders' movement trajectory compared to the reference mation regarding the user's movement efficiency and movement trajectory; biomechanical leverage. For example, by attaching a fixed 55 2) a means to view said visual display stimulus with the aid subordinate transponder at the hips and a fixed principle of a mirror(s) or other reflective means; transponder on the upper ann, the biomechanics of the act of 3) a means for the real-time projection of sound or speech lifting a box or similar object can be elegantly qualified. If commands through an audio device to provide warning, the user assumes a poor lifting technique, i.e. legs locked alann, instructional, and motivational aid, and/or addi with the trunk severely flexed with head down and the arms 60 tional cueing upon encroachment of static and dynamic stretched out beyond the basis of support, the vector's length limitlboundary conditions defined by the reference move- would consistently be measured longer than compared to a ment trajectory; good lifting technique, i.e., legs bent at knees with the back 4) a means for real-time tactile feedback including, but not straight, head gaze up, and arms close to body. Also, the limited to, modulation of the rotational properties of a measurement(s) of higher-order derivatives derived from 65 vibrator motor proportional to the pose error of the numerical mathematical processes of a reference point transponders' movement vector compared to the refer- described by the vector would provide additional indication ence movement trajectory; A000015 Case: 12-1252 Document: 46 Page: 106 Filed: 08/29/2012 JX-001.0016 US 7,292,151 B2 5 6 5) a means for combining the excitation of said stimuli be used such as SmartMedia, Compact Flash, etc. Addition proportional to the pose error of the transponders' move ally, the memory device interface should not be limited to ment vector compared to the reference movement trajec internal, but may include external media devices, such as tory; and, USB FLASH Key or other portable media means, that may 6) a means to coordinate the real-time, periodic parametric have inter-operability with other computerized devices. The update and modulation of the stimuli imparted by the data structures may include: sensory interfaces within the transponders from a pro Modulation & Feedback Thresholds/Triggers Properties cessing unit by means of a wireless communication linle the aural, visual, tactile interfaces require threshold set This invention addresses the need for an intuitive, inter- tings which determine their excitation or stimulation active method to instruct, create, and deliver a movement 10 characteristics. These settings can be derived from pre trajectory command without necessarily relying on pre vious performance data or defaults determined from nor programmed, regimented movement trajectories. The regis mative data, or modified in real-time, by algorithmic tration system can be configured via remote setup at the methods including moving averages, standard deviations, principle transponder to pre-record and choreograph a free interpolation based upon goal-oriented objectives, etc. form movement trajectory of the principle transponder with 15 Normative Performance-performance data collected over a the intent of the user mimicking the same said path. This large population of users through controlled studies, that impromptu learning modality can expedite the session down is distilled down into specific user categories based upon time between different users and movement scenarios, and certain demographics that the user may compare and rank accommodate users' high anthropometric variability in hislher results. This data may be initially embedded range of movement. In summary, one embodiment of the 20 within the transponders or position processor non-volatile present invention is comprised of: memory and may be augmented or modified auto mati - 1) a means is to provide a movement trajectory learning cally or by user volition when connected to the Internet. modality that allows the user to calibrate and create the Competitive Ranking-applications which have a predomi desired endpoints, midpoints, and/or total reference nate point goal-oriented purpose would allow access to a movement trajectory through user progranliller entry of an 25 global ranking file archive accessed through the Internet input device resident on the transponder; or automatically via updated executive files. This ranking 2) a means to process and save a movement trajectory using file would be created through an analysis of user partici a computationally efficient Catmull-Rom spline algorithm pation and publishing of his/her results through Internet or other similar path optimizing algorithms to create Web-based services. control points along key points of the movement trajec- 30 Downloadable Executive Programs & Configurations-new tory that define the optimally smoothest path intersecting software programs, including new features, enl13nce the control points; ments, bug fixes, adjustments, etc., could be downloaded 3) a means to provide database management by a processing to the transponder through an Internet connection. Graph unit via a wireless communication link or, alternatively, ics images would be stored in compressed or uncom through user data entry of an input device resident on the 35 pressed binary forms, i.e., bitmap, gif, jpeg, etc. This new interactive transponder; and, programs could be transferred to any suitable computer 4) a means to access, edit, and store the program and/or ized position processor unit located at a remote facility via the transponder's wireless link. Therefore, the user's databases to nonvolatile memory operably coupled to the principle transponders for the purpose of automating the transponder is the node that establishes the portable network capabilities of the system, not necessarily the creation, delivery, storage, and processing of movement 40 trajectories. Customized user programs and databases computerized position processor. would be downloaded from a central repository or rel Custom Menu Interfaces-specialized activities may require evant website in advance of the training session to the more advanced (or simplified) interfaces dependent upon transponder from the user's home location via the Internet the users' cognitive abilities and interactive specificity. This menu may include interactive queries or solicit or other convenient locales having networked Internet 45 access, and transported to the systems remote physical information regarding the user's daily goals, subjective location, and uploaded into the system's memory, and opinions or overall impression of the activity and ones executed as the application program. This a priori process performance which could be incorporated in the Motiva of remote selection, download, and transfer of program tion Index described below. Report Generation Tools and Templates-XML, HTML or matic content and database would minimize the user's 50 decision making and input during product utilization by other authoring language used to create documents on the offering only relevant and customized progranlilling Web that would provide an interactive browser-based user material of their choosing targeted for their specific exer interface to access additional performance data analysis cise, fitness, or rehabilitation goals. Performance data and report generation tools and templates that may not be available or offered with the standard product. could be saved indefinitely in the database's nonvolatile 55 memory, until an upload process was performed through Custom Performance Algorithms---certain application-spe the said network so the database could be transferred to cific performance analysis may require dynamically another location for purposes of, but not limited to, linked algorithms that process and calculate non-standard registration, processing, archival, and normative perfor- or specialized information, values, units, physical mea mance evaluation, etc. 60 surements, statistical results, predictive behaviors, filter ing, numerical analysis including differentiation and inte An exemplary list of specific data structures contributing to gration, convolution and correlation, linear algebraic or affecting the means for automating the creation, delivery, matrices operations to compute data pose and scaling storage, and processing of movement trajectories described transformation, and proprietary types. One example of a below may be stored within the non-volatile memory of the 65 proprietary type is Motivation Index, a composite numeri transponder or position processor which may use high cal value derived from a weighted average of statistical density serial FLASH, although other types of memory may performance indicators and subjective user input includ- A000016 Case: 12-1252 Document: 46 Page: 107 Filed: 08/29/2012 JX-001.0017 US 7,292,151 B2 7 8 ing relative scoring improvements, conformity to ROM optimal tracking location based upon collectively maxi pattern, lengthy activity access duration, high access rate, mizing the ultrasonic source's energy received at the relative skill level improvement, daily goal achievement, transducer interface. etc., that could represent the overall level of enthusiasm This invention addresses the practicality and robustness of and satisfaction, the user has for a particular activity. the registration system when used in either indoor or outdoor Range of Motion (ROM) Pattern Generator-the ROM environments, and especially when the tracking volnme pattern requires some key control points to be captured likely contains potentially occluding objects, i.e., an unin along the desired trajectory and stored in order that the volved limbs or clothing, that become potential sources of algorithm can calculate an optimally smooth path, in competing, reflected paths. The preferred embodiment of the real-time, during the comparative analysis phase. 10 registration system utilizes the time of flight (TOF) mea surement of ultrasonic acoustic waves due to its immunity ROM Pattern Capture & Replay-the ROM pattern can be from interference from the visible and near-visible electro can saved to memory in real-time by discrete position magnetic spectrum and its superior ability to overcome most samples versus time depending upon the resolution multi-path reflections problems by simple gated timing of desired and memory limitations and later played back on 15 the initial wave front. Upon command from the processor the transponder or remote display for analysis. unit, the transponders produce a few cycles burst of ultra- Activity Specific Attributes-includes Reps/Sets, Duration, sonic energy and the transducers of the receiver constella Pause, Heart Rate Limits, intra-activity delay, level, point tion unit are stimulated and mechanically resonate accord scalars, energy expenditure, task-oriented triggers, etc., ingly, upon the wave front arrival. The processor unit's and other parametric data that controls intensity, execu 20 analog signal processing circuits transform the mechanical tion rate and scoring criteria for the activity. energy into electrical signals that resemble tapered sinusoi Instructional Information-textual, graphical, or animation dal waveforms, which another electronic circuit triggers based instruction, advice, coaching, activity description, upon using an adaptive threshold technique which, in turn, diagramed transponder deployment and intra-device con the processor unit detects and calculates TOF timestamps nectivity, etc. that facilitates the intuitiveness, understand 25 indicating the wave front arrival. In the preferred embodi ing, and usage of the system. The form of instruction may ment, the system overcomes the ultrasonic technology's include music files saved in various formats, including intrinsic challenge of precisely triggering on same the wave Wave, MP3 or other current or future audio data com form location and provides consistent unambiguous trigger pression formats, and video files saved in MPEG or other detection by complementing the adaptive threshold tech- current or future video data compression formats. 30 nique with a software timestamp correction algorithm, Real-time Data Management-proprietary data manage which includes in part, a digital over-sampling and averag ment protocols that reside above the communication ing timestamp algorithm, a relative timestamp correction driver layer that manage the real-time, synchronous and scheme utilizing a predictive algorithm of higher-order asynchronous exchange of data between transponder(s) Taylor series based derivatives, and an absolute timestamp and position processor. This would provide an essential 35 correction scheme that minimizes the range error based upon real-time sharing of activity data, analysis, and feedback discrete biasing of timestamps. stimulus thresholds, or coordination of multiple transpon Further, in the preferred embodiment, the processor unit der configurations, or for a collaboration of same or utilizes the absolute and relative trigger timestamps in a different user requirements to complete a similar activity multi-modal trilateration algorithm for the measurement of objective. 40 three-dimensional (3D) translations and rotations of the This invention addresses the need for adaptability of the transponders. The primary trilateration calculation is derived registration system to different movement measurement by an application of Pythagoream theorem involving a point scenarios. In one embodiment, it utilizes a versatile, modular position solution based-upon range measurements from at configuration and mounting of the transponders onto the least three (3) points, versus the well-known triangulation user. The efficient deployment of the transducers between 45 method which uses bearing angles of two cameras of known different users' and from task to task requires a universal pose. Additionally, the system's main accuracy limitation is mounting scheme to provide consistent localization and pose mostly affected by the temperature variability of outdoor of the transponders at the desired measurement sites on environments and its influence on the speed of sound in air user's body. Also, to compensate for the receivers' finite value. This algorithm mitigates this problem by mathemati- tracking volume when stationary, the receiver constellation 50 cally computing the speed of sound every analysis period unit may be mechanically modified to optimize its tracking provided at least five (5) receivers and a transponder syn properties by conveniently repositioning it in closer prox chronizing means are utilized. If the integrity of the syn imity to the expected transponders movement trajectories chronizing signal is temporarily compromised, the system and line-of-sight, thereof. In summary, one embodiment of automatically employs a variation of the trilateration algo- the present invention is comprised of: 55 rithm that uses the last known speed of sound value. In the preferred embodiment, the maximum update rate, 1) a means to quickly and effIciently alter the location of the and hence the major contributor to the latency of the position transponders using a fastening system designed to quickly calculation, is determined by the typical acoustical rever attach and dispose various forms of transponder assem beration, typically between 20 to 100 ms, encountered in an blies; 60 indoor environment. Since the transponders are held or fixed 2) a means to augment the physical properties, i.e., weight on the user's body and, therefore, are mobile, the TOF and length, of the principle transponder with adjunct measurements will experience an additional latency effect. A electromechanical components that provide variations in Kalman filter is used as a prediction/estimation strategy to biomechanical leverage for isotonic and isometric utili minimize and compensate for the latency effect. The pre- zation; and, 65 diction algorithm uses a higher-order Taylor series based 3) a means to allow the user to manually alter the geometry derivatives and augmentative inertial sensor data. Its pre and pose of the receiver constellation unit to facilitate an dictive refinement is dependent upon predefined models of A000017 Case: 12-1252 Document: 46 Page: 108 Filed: 08/29/2012 JX-001.0018 US 7,292,151 B2 9 10 expected movement conditions. Because functional move FIGS. 2A-2D illustrates example extension pieces for the ment is episodic, having periods of stillness interspersed present invention; with bursts of motion activity, a multi-modal filtering strat FIGS. 3A-3D illustrate one example of process flows for egy is preferably employed to handle the unpredictable the present invention; jerkiness at the start of motion and relatively predictable, FIGS. 4A and 4B illustrate a sample application of the smooth motion afterwards. In summary, the preferred present invention; embodiment of the present invention is comprised of: FIG. 5 illustrates a block diagram of the remote process 1) a means to detect the same carrier wave cycle of ultra ing system of the present invention; sonic energy using a software correction algorithm requir FIGS. 6A-6C illustrate example receiver configurations of ing multiple, consecutive TOF acquisitions as input for 10 the present invention; and the digital over-sampling and averaging algorithm, the FIG. 7 illustrates a block diagram of the components of calculation of a higher-order numerical differentiation of one embodiment of the transponder of the present invention. the past and current TOF information as input for the predictive algorithm of higher-order Taylor series based DETAILED DESCRIPTION OF INVENTION derivatives used for the relative TOF correction, and a 15 measurement of the intra-pulse time intervals of consecu The present invention provides a practical, versatile mea tive TOF acquisitions as input for the absolute TOF surement tool for the assessment of the user's manipulation correction scheme that minimizes the range error based strategy of the transponder 10 or transponders along a upon selective biasing of the TOFs; reference movement trajectory. Moreover, the system and 2) a means to utilize a dual matrix formulation of the 20 methods measure and analyze the kinematics of the relative trilateration algorithm, and a calculation strategy thereof, translations and rotations of the limbs or extremities with which decision is dependent upon the integrity of the respect to each other or to a more inertial reference location system's communication link, synchronization condition, on or off body as the transponders are manipulated. This and the desired measurement accuracy; and, information provides useful insight on biomechanical 3) a means to coordinate the information transfer between 25 demands and anthropometric factors that influence human transponders and the processor unit so that their contri movement efficiency and control. Although measurement bution to the resultant movement vector calculation can performance metrics are important design criteria, it's be measured without intra-signal interference. equally important to provide intuitive and motivating pro These goals will be attained by such system and methods gram instruction and administration, and to provide com that are comprised of the user's interaction described by the 30 prehensive analysis and integration of the motion data in a following steps as set forth as the preferred embodiment: form that is objective and easily interpreted. This system 1) Authenticate user access and open user session from a improves upon the practicality and user interactive aspects local or remote database; of setup, deployment, calibration, execution, feedback, and 2) Setup user training session, i.e., workload limitations, data interpretation of a tracking system designed for func- measurement criteria, and audio/visual/tactile stimuli; 35 tion human movement. 3) Select training program and configure its options; Human movement is a response to external environmental 4) Deploy the transponders as instructed to predefined forces which requires the accurate coordination of the distal locations of users locomotion system to create at least one segment( s) to compensate for these forces. Skillful coordi transponder movement vector; nation of human movement is dependent upon the cohesive 5) Calibrate the transponder movement vector to establish its 40 interaction of multiple sensory systems, including visual, reference pose; vestibular, with the musculoskeletal system. More specifi- 6) Create a movement trajectory using learn mode, if cally, the challenges and goals of cognitive spatial mapping, required; (2) minimization of energy expenditure, (3) maintaining 7) Initiate the start of session; stability, (4) steering and accommodation strategies for 8) Determine the instantaneous pose of transponder move- 45 various environments, (5) dynamic equilibrinm, (6) active ment vector relative to its reference pose from a periodic propulsion and weight support, and (7) core locomotion temporal iteration of this step; pattern should be relationally considered to properly assess 9) Perform qualitative and quantitative statistical analysis of hnman movement. Therefore, it is preferable to engage the accumulated measured poses of the transponder move interaction of these sensory systems during a training ses ment vector relative to the pattern of instantaneous poses 50 sion to promote the desired functional movement outcome. defined by the reference movement trajectory; Because many movements persist for 400-500 ms, enough 10) Update the major transponders sensory interfaces to time is allowed for the initiation of the movement and for modulate said system parameters in a periodic temporal user correction based upon visual and kinesthetic informa iteration of this step; tion acquired during the time of the movement. However, 11) End the session once program objectives have been 55 the implemented means of visual feedback should be not be obtained; distracting or interfering with the task at hand. In the 12) Analyze the results by interacting with local and/or preferred embodiment, this system engages the sensory remote databases; systems with non-distracting, intuitive, embedded aural, 13) Provide numerical, graphical, and/or animated informa visual, and tactile stimuli which provide real-time indication tion indicating desired performance measurements. 60 of the principle transponder pose error with respect to the reference movement trajectory. BRIEF DESCRIPTION OF THE DRAWINGS In order to conduct a time efficient training session, this registration system attempts to minimize the encumbering FIG. lA illustrates one example of a deployment appa experimental setup and calibration procedures characteristic ratus of the present invention; 65 of more complex and higher cost motion analysis technol FIG. IB illustrates one example of hand-held form for the ogy. These complementary systems serve important aca transponder of the present invention; demic or clinical oriented research needs or for motion A000018 Case: 12-1252 Document: 46 Page: 109 Filed: 08/29/2012 JX-001.0019 US 7,292,151 B2 11 12 capture for computer animation purposes and strive for ment of musculature strength of the limb, while not com highly accurate measurement of joint motion data in terms promising the user's endurance with a potentially fatiguing of angular displacement. Therefore, the integrity and reli hand grasp requirement. ability of their motion data is dependent upon proper sensor In one embodiment (FIG. 2B), the tactile type provides setup and calibration. force feedback functionality by controlling the rotational For instance, single axis goniometer-based systems usu speed of an embedded vibrator motor in the shaft. Alterna ally require specially designed harnesses to hold the monitor tively, the visual type (FIG. 2C) may be comprised of a and are firmly strapped or taped over the joint to avoid series of light emitting diodes that could be uniformly relative motion artifacts. Usually these devices are tethered 10 embedded along the length of the handle or transponder and and their fit, weight, and constraining mechanical linkages their intensities variably controlled therein. It should be can impose limitations on the joint motion and cause dis appreciated that a simple, economical mirrored or reflective comfort for the user. Most optical or video-based systems surface placed in front of the user's visual field could require the placement of numerous active or passive markers provide sufficient real-time indication of the user's subjec- over landmarks, such as the joints' center of rotation. These 15 tive conformity to the said movement trajectory while systems should guarantee sufficient environmental illumi allowing non-distracting viewing of this visual stimulus. For nation and contrast between markers and background to example, a program that requires the user to reposition the function optimally. Also, these systems are severely affected principle interactive transponder through an arc-like move by occluded markers that may disappear for long periods of ment trajectory in the midsagittal plane through out a range time due to rotations and line-of-sight limitations. Other 20 of motion begiuning from the waist upwards until parallel to video-based systems do not use markers but require the shoulder height. As the user performs the movement, the assignment of the body's joints manually or through com visual sensory interface could be proportionally excited if puterized automation during data analysis, making real-time the user moves too quickly, or hesitates too long, or produces analysis arduous and real-time feedback virtually impos shaky or erratic episodic motions, or is beyond the pre- sible. 25 scribed bounds of the movement arc. The light stimulus is In the preferred embodiment, the system doesn't require easily viewed in the mirror and would indicate corrective complicated, time consuming sensor setup and calibration action in his or her movement strategy, while appropriate by virtue of it minimalist sensor requirements and uncom aural commands may be issued simultaneously to encourage plicated sensor mounting. Instead, it requires only the the same correction. Regardless of the sensory interface deployment of a sensor on the body (in one embodiment a 30 type, its control and excitation properties will be determined dual sensor group on a combination oflimb(s) and or trunk) by some statistical aspect of the user's conformity to and and doesn't enforce stringent movement protocol, but progression through the movement trajectory. encourages free-form, unrestrictive movement of the tran The hand-held transponder may include a modular exten sponders. sion piece with an embedded graphic display device and The transponder's preferred deployment means, include 35 associated input means to allow the user to setup, operate, either insertion into a universal strap and holster apparatus provide visual feedback, and view performance results of the (FIG. lA) that secures on the user's limb, extremity, or device usage without additional remote display means. More trunk, including, but not limited to, the hip, ankle, knee, specifically, a software-controlled user interface could pro wrist, upper arm, neck, waist or an augmentative mechanical vide certain visual prompts in a menu oriented presentation, attachment to one or a combination of modular extension 40 to instruct the user on (1) device setup, i.e., aural, visual, and pieces shaped into a hand-held form (FIG. IB). A strap or tactile feedback parameters, types of program start and torx-like clip and holster design provides a firm, yet light termination cues, program intensity based on ratio of weight and comfortable mounting location away from areas amount of repetitions, sets, and rest periods or categorical that clothing and or uninvolved limbs may occlude. gradation of challenge, learn mode behavior, etc., (2) scrol- The modular extension piece is either an instrumented 45 lable program selection with brief descriptions including sensory type designed to support alternative tactile stimulus objective, desired measurement, i.e., range of motion, device or alternative configurations of aural, visual, and energy, accuracy, speed, etc., and instructive information, tactile feedback types, or non-instrumented, weighted exten and (3) alphanumeric and/or graphical display of measured sion pieces as shown in FIGS. 2A-2D. All modular exten performance data and other biophysical data and its analysis sion pieces may be of various physical dimensions and 50 thereof, displayed in standard plotted forms including line, intrinsic weight, with a captive handle design that preferably bar, and pie charts, etc. It is important to note that the user requires zero grip strength to grasp. Alternatively, the modu input process is intuitive and streamlined so as not to detract lar extension piece may be coupled to the transponder from the practicality and user friendliness of the system. through a fixed or flexible, segmented, articulated coupling Only relevant applications and its control thereof will be to accommodate attachment of additional transponders and/ 55 sequestered from the database and presented to the user. or other modular extension pieces. These components would In one embodiment, two or more transponders and exten quickly assemble to each other using a spherical snap joint sion pieces, or combinations thereof, may be assembled at or twist snap latch, or similar mechanism, to provide quick their endpoints with a universal spring coupling. The alteration of form and function when used for different assembled device could be grasped in both hands and bent movement trajectory scenarios. 60 in various rotational angles about the spring coupling's axis. In one embodiment, the weighted extension attachments Isotonic strength conditioning programs can be developed (FIG. 2A) are offered in fixed gradations of one (1) kilogram due to the force resistance feedback supplied by the spring. increments or other convenient unit of measure and either be A multi-transponder assembly in the form of a flexible rod indicated as such with a numerical label, quantitative mark, or staff could provide an indication of balance of upper or color-code feature, or combination thereof. For upper 65 extremity strength and proprioceptive function dependent extremity evaluation, the weighted extension piece inte upon the angular closure rate and rotational imbalance and grated into a zero-grip handle would enhance the improve- orientation deviation from initial starting position. A000019 Case: 12-1252 Document: 46 Page: 110 Filed: 08/29/2012 JX-001.0020 US 7,292,151 B2 13 14 Additionally, in the preferred embodiment, the modular limited to, workload intensity, measurement criteria, sensory extension pieces have provisions for other attachable appa interface properties, and reporting features. A program menu ratus (FIG. 2D) that can augment the program's intensity or list would indicate name, ID, and a brief description, or difficulty. For example, an eyelet is embedded in the end of alternatively, be represented by a detailed graphical icon. the extension piece and is designed to attach an elastomeric When the program is selected, other program-specific band, such as the type manufactured by Theraband®. By options can be setup. securing the other open stirrup end of the band to the user's During the Deployment Phase (FIG. 3B), and dependent foot, isotonic strength conditioning programs can be devel upon the program's objectives, a suitable combination of oped due to the force resistance feedback supplied by the transponder types will be mounted on the user's body as elastomeric band. Moving the transponder through a move 10 instructed by the program. This example requires the assem ment trajectory is now made more restrictive and challeng bly of a hand-held interactive transponder with graphical ing. display, and a weighted extension piece coupled therein to be grasped by the hand on the same side as the affected APPLICATION EXAMPLES shoulder. Another subordinate transponder 12 is placed into 15 a holster assembly strapped around the lower quadriceps on An example training session deploying a dual transponder the same side. This setup is shown in FIG. 4. group is now described that may be designed to improve the During the Calibration Phase (FIG. 3C), a simple cali range of motion, strength, and coordination of shoulder bration procedure may be requested to evaluate transponder abduction in a user. The training session would primarily function and specific user range of motion constraints. serve as an exercise aid that provides essential feedback to 20 Typically, this information is determined beforehand and the user so that he/she learns to progressively improve the saved in the system's database. Also, practicality of this manipulation of the transponder through the reference system is claimed for lack of extensive calibration require movement trajectory, while benefiting from increased shoul ments. der range of motion and strength improvement. Dependent upon the program's options, a user-defined In advance of the training session, a software application 25 movement trajectory may be created prior to program start is operated from a host computer that provides a utility for in lieu of executing the predefined version. The learn mode baseline configuration and management of the system's and could be utilized to quickly choreograph free-form move transponder's local databases, and/or access to other remote ment trajectories and save them into the transponder's databases, and for the real-time interface to the data flow non-volatile memory for later execution. The learn mode between the system's components. The application's navi 30 would be accessed through the user interface and instruct the gation and selections are presented to the user through a management of the control point assignment by pressing the typical graphical user interface like Microsoft Windows® push button switch at the appropriate junctures of movement XP operating system. A generalized step-wise procedure discontinuity or, preferably, allowing automated assignment requires the administrator or user to (1) select the desired by the software. In the preferred embodiment, a computa- program and features from a menu screen list, and (2) to 35 tionally efficient Catmull-Rom spline is used to define a initiate a communication process that causes the program three dimensional (3D) curve that passes through all the parameters to be transferred to the processor unit through a control points along the movement trajectory path. Ifmanu standard computer communication protocol, i.e. serial, USB, ally interceding, the user is instructed to press the push ethernet, etc., whereupon, (3) the information is subse button once at each major juncture in the movement trajec- quently processed and transferred into the transponders local 40 tory, but, preferably, for no more than a few locations, until memory via a wireless communication link, and, finally, (4) the desired end of range of motion is reached as shown in the transponder's software program accesses this database to FIG. 4B. Similarly, the return path may be similarly defined manage the device utilization and configuration of the local or he/she may elect to use the same forward path in reverse. display means. Alternatively, a Compact FLASH-based These control points are registered by the processor unit and memory card, embedded serial FLASH, or a similar non 45 transferred and saved to the transponders' memory to serve volatile memory device provides the user an additional as the control points for the real-time calculation of a specialized database supporting remote data collection capa Catmull-Rom spline. The Catmull-Rom spline is calculated bilities. This database would be preprogrammed in advance in real-time from the desired initial starting point to provide and the resultant performance data retained, even if the a continuous set of position points representing the device's power is lost, or for extended unsupervised exercise 50 "learned" reference movement trajectory. sessions conducted remotely from the host computer system After the program is selected or the learn mode complete, or when the host computer system is unattached or unavail the user may be instructed to alter the pose of the transpon able. After the session is completed, the user would be ders to satisfY the initial starting conditions of the program. queried if the results are to be saved for later analysis or Either one or a combination of sensory interfaces could be would automatically be saved, dependent upon device setup. 55 excited by the principle transponder to cause the user to This data could be retrieved at a later time when the system direct or steer it towards the desired start point. For instance, is once again attached to a host computer system, and the the visual sensory interface could sequentially extinguish or software utility could be commanded to upload the database. dim its peripheral light sources to converge to a central light Henceforth, the following procedural description refers to source as the principle transponder is positioned closer to the the activity dependencies diagrammed in FIGS. 3A-3D that 60 desired start point. Alternatively, the aural sensory interface the user would encounter while operating the system. could change its tonality and loudness as the start point is During the Security Phase (FIG. 3A), the user may be approached. Or alternatively, the tactile sensory interface requested to provide a security authentication code for could be modulated to provide less force feedback as the validation, which opens access to his/hers custom programs start point is approached. in the training session. Next, during the Setup Phase (FIG. 65 During the Execution Phase (FIG. 3D), the transponders 3A), the user can configure global options or select the are continually manipulated along the reference movement desired program. The global options may include, but are not trajectory to the best of the user's skill and fidelity, within A000020 Case: 12-1252 Document: 46 Page: 111 Filed: 08/29/2012 JX-001.0021 US 7,292,151 B2 15 16 the bounds of the user's physical limitation, until an aural, increased risk for falls, balance disorders can shorten atten visual, or tactile response is given that indicates the activity tion span, disrupt normal sleep patterns, cause excessive volume has been successfully completed or a sufficient fatigue, increase dependence on others and reduce quality of number of confonnity violations or failures have been life. It is not uncommon for individuals with a history of registered. The processor unit calculates the instantaneous balance problems to regain their balance control through pose coordinates of the transponders every analysis interval accurate diagnosis followed by specific medical treatment and periodically communicates this information to the tran and/or rehabilitation exercises. sponders via the wireless communication linle As the prin The present invention described can be used as a testing ciple transponder is moved in mimicry to the reference and training device for balance improvement under both movement trajectory the confonnity error between the actual 10 static and dynamic conditions. and reference movement trajectory is calculated periodically One testing and training scenario for postural stability in real-time to determine the characteristics of feedback would be to measure frequency and amplitude of body sway quality to be elicited by the sensory interfaces for the user's in three dimension (3D) space while feet remain in a fixed closed-loop control to correct his/her manipulation strategy. position. This task can be performed in both a double or For example, the conformity error may be calculated from 15 single leg stance to test for bilateral symmetry relating to statistical processes based upon the standard deviation of the balance. Another modification of the test would be to least mean squared (LSM) principle transponder's position perfonn each test with eyes both open and closed to help error compared to the reference movement trajectory, or determine the contribution of the visual component to over based upon, or combination thereof, a threshold magnitude all balance ability. Tracking body sway while creating the of some multi-order numerical differentiation of said move- 20 illusion of motion through proper visual cueing on a display ment to indicate a "smoothness" quality of translation and means would be another test to help detennine the reliance rotation along the movement trajectory path. on specific sensory components of balance. Delivering rep Alternatively, a host computer system could provide an etition of protocols with increasing difficult oscillation auxiliary processing and display means to allow another thresholds with biofeedback of successes and failures of software program to access the transponder's calculated 25 such predetermined goals is one way to train to improve positional data through an application programmer's inter balance. face and use this data to alter the pose of a graphical The transponder can deliver aural, visual, and tactile primitive in proportion to the motions of the transponders stimuli to queue the individual to the degree of frequency within the context of computer generated virtual environ and amplitude of body oscillations. The aural and tactile ment. The dynamic control of objects in the computer 30 components provide the only means of feedback when the generated virtual environment could be used to augment the testing and training are perfonned with eyes closed or the local sensory interfaces of the transponders through an visual field is compromised. Examples include, but are not interactive, goal-oriented video game modality. The video limited to, (1) an audio signal increasing and decreasing in game challenges could be increased over time based upon volnme proportional to the amplitude of body sway, (2) a some scoring criteria of successful manipulation of the 35 vibration action proportional to frequency of body oscilla principally controlled on-screen graphical object with tions, and (3) a light source illuminated when both frequency respect to cueing derived from other secondary static or and amplitude goals are achieved. Multiple transponders can dynamically moving objects. It is important to note that only be used to evaluate and reinforce proper balance posture by primitive fonns of video game challenges would be consid communicating position information of certain body seg ered, to take into account the user's cognitive awareness and 40 ments in relationship to others. An example would be the physical limitations, and the economics of software devel comparison of position of the head with respect to the hips opment for photo realistic virtual environments and anima while generating a vibration action if an excessive forward tion. Also, this auxiliary computer display means would lean of the head as compared to the hips is recognized. offer an alternative visualization method of interactive and Another test for balance would be to test ones Limits of immersive video feedback aid to enhance the application 45 Stability (LOS). This test refers to ones ability to effectively presentation. operate within their sway envelope. The sway envelope or Additional examples of how the present invention may be LOS is defined as the maximal angle a person's body can applied are described as follows: achieve from vertical without losing balance. An individual with healthy balance is capable ofleaning (swaying) within Balance 50 a known sway envelope and recover back to a centered The body has the ability to maintain balance under both position without the need for a secondary maneuver such as static and dynamic conditions. In static conditions, such as a step, excessive bend at the torso or ann swinging. LOS for in standing, the body strives to efficiently maintain posture bilateral stance in nonnal adults is 8 degrees anterior, 4 (often referred to as postural stability) with minimal move degrees posterior and 8 degrees laterally in both directions. ment. In dynamic conditions such as in walking or sports 55 The present invention described can be used as a testing play, the body strives to maintain balance while perfonning and training device for balance control during movement or in an ever changing environment. The ability to maintain perturbations within a desired sway envelope. Through balance is a complex process that depends on three major proper visual queuing represented on the display means that sensory components. The sensory systems include visual, defines a normal sway envelope, the amount of body dis- vestibular and proprioception. For example, we rely on our 60 placement can be measured from vertical stance. visual system (eyes) to tell us if the environment around us The transponder can deliver aural, visual, and tactile is moving or still; we rely on our vestibular system (inner stimuli to queue the individual as to when he or she has ears) to tell us if we are upright or leaning, standing still or achieved the desired range of their sway envelope, then moving; and we rely on our proprioceptive system (feet and assess the individual's ability to return back to a vertical joints) to tell us if the surface we are standing on is uneven 65 stance. Examples include, but are not limited to, (1) a or moving. If balance problems develop, they can cause vibration action when the user varies (meanders) from the profound disruptions in your daily life. In addition to desired movement path, (2) an array of lights change inten- A000021 Case: 12-1252 Document: 46 Page: 112 Filed: 08/29/2012 JX-001.0022 US 7,292,151 B2 17 18 sity and pattern as the individual successfully approaches the the display means to instruct and motivate the individual intended target, (3) an audio signal is generated when the through the proper testing procedure. individual has maintained a stable position with respect to The present invention described can be used as a testing proper visual queuing represented on the display means for and training device for individuals involved in physical a selected period of time. Multiple transponders can be used rehabilitation or general fitness to improve ROM. Proper to evaluate and reinforce proper balance posture by com visual queuing can be represented on a display means to municating position information of certain body segments in motivate individuals to extend their range of motion beyond relationship to others. An example would be the comparison their current capabilities. of position of the head with respect to the hips while The transponder can deliver aural, visual, and tactile generating a vibration action if an excessive forward lean of 10 feedback that alerts the individual to successes or failures in the head as compared to the hips is recognized. proper execution of each repetition. An example of tactile Dynamic balance can be evaluated while having the feedback would be the transponders are vibrated if the individual perform coordinated movements which specifi individual's movement trajectory varied from the intended cally challenge the various components of balance in a two dimensional (2D) reference movement trajectory by 15 deviation from the planar path into the uninvolved spatial dynamic nature. Such movements include, but are not lim ited to jumping, hopping, and walking. These movements dimension. An array of light sources could increase illumi can be performed with eyes both open and closed, during nation in intensity and repetition as the ROM goal was interaction with static or dynamic visual queuing on the approached and an audio tone could signal the individual display means. The ability to perform these dynamic balance they have achieved the desired pause time at the proper 20 ROM. tasks with comparisons to others of similar sex, age or disability can be assessed. Example measurements may Multiple transponders can be deployed to determine the include, but are not limited to, (1) amount of body sway in contribution of each joint or anatomical structure where three dimension (3D) space, (2) time to complete specific more then one joint is involved in the ROM movement task, and (3) effects of fatigue on balance ability. (example; shoulder and scapular in overhead reaching). The 25 vector sum of each transponder movement in a specific axis Balance training in both static and dynamic conditions can be added together to determine the total ROM. The can be easily achieved by providing specific visual queuing ROM of one joint in a two joint motion can be subtracted on the display means, which challenge the individual to from the total ROM to determine the contribution of a single perform repetitive and progressively more difficult balance joint in a two joint movement. drills. Performance reports can be generated to establish a 30 baseline, isolate specific strengths and weaknesses within Human Performance Testing and Training the specific sensory and motor control aspects of balance, There are many devices that test the strength and speed of and document progression and improvements. isolated joint movements, for example, the leg extension and The transponder can deliver aural, visual, and tactile bicep curl. This information has value in testing both healthy stimuli to queue the individual as to when he or she has 35 individuals, athletes and individuals whose strength and achieved the desired balance task. By example, a vibration speed capabilities may be compromised by injury, disease, action is produced proportional to the frequency of a body poor conditioning or simply age. Recently in the field of segment oscillation after the user lands from a hop test and human performance, it has been recognized that the analysis attempts to stabilize and maintain proper postural balance. of the mobility of the isolated joint, although providing some When the individual finally stabilizes and achieves correct 40 value, does not offer enough information to determine how postural balance, an audio signal indicates the task has the body will perform during functional movements. Func successfully completed. Multiple transponders can be used tional movements are defined as movements equal to those to evaluate and reinforce proper balance posture by com encountered on the athletic field, in the work environment or municating position information of certain body segments in while performing activities of daily living. Functional move- relationship to others. An example would be the comparison 45 ments involve the movement and coordination of multiple of position of the head with respect to the hips while joints and muscle groups acting together to perform a more generating a vibration action if an excessive forward lean of complex task then a single, isolated joint movement. the head as compared to the hips is recognized. The present invention described can be used as a testing and training device for functional movement improvement. Range of Motion (ROM) 50 By tracking various registration points on the body with The present invention described can be used as a testing respect to each other or to an off-body registration point, and training device to determine the range of motion within performance measurements of functional movements can be a joint. Range of Motion is the normal distance and direction assessed, such as jumping, cutting, turning, bounding, hop- through which a joint can move. Limited ROM is a relative ping, shuttling, etc. term indicating that a specific joint or body part cannot move 55 The present invention described can be used as a testing through its normal and full ROM. Motion may be limited by and training device for individuals involved in physical a mechanical problem within the joint that prevents it from rehabilitation, general fitness or sports performance moving beyond a certain point, by swelling of tissue around enhancement to improve their functional movement abili the joint, by spasticity of the muscles, or by pain. Diseases ties. Proper visual queuing can be represented on the display that prevent a joint from fully extending, over time may 60 means to instruct and motivate individuals to perform spe produce contracture deformities, causing permanent inabil cific functional movements. ity to extend the joint beyond a certain fixed position. The transponder can deliver aural, visual, and tactile The present invention described can be used to test the feedback of proper movement execution. Examples include, starting point and end point which an individual is capable but are not limited to, (1) an audio signal alerting the user of moving a body part, typically a limb and associated 65 that the desired performance stance is incorrect, (2) the light joint(s). Comparisons to age and sex based normative data sources illuminate when the desired speed is achieved in a can be made. Proper visual queuing can be represented on first step quickness drill, (3) a vibration action to indicate the A000022 Case: 12-1252 Document: 46 Page: 113 Filed: 08/29/2012 JX-001.0023 US 7,292,151 B2 19 20 limits of tracking range, (4) a vibration action proportional oriented perpendicular to the horizontal plane. Further, as to the magnitude of a biophysical measurement during the indicated in the preceding figures, the transducers vertical interaction with visual queues represented on the display axes are oriented 90° with respect to the typical vertical axis means, (5) a vibration action when the body or limb position orientation of the transponder's transmitter to improve does not correlate well to the desired body or limb position acoustic coupling in the vertical plane, a consideration for of the visual queuing represented on the display means, (6) overhead, upper extremity tracking. Although this causes an audio signal indicating start, stop and pause periods or some reduction in the lateral registration bounds, the com other controlling events, (7) an audio signal indicating promise provides a more symmetric field about the middle proper body alignment or posture has been compromised, or primary location of tracking interest. and (8) an audio signal indicating the relationship of desired 10 In the preferred embodiment, the overall size of the target heart rate to a desired threshold. receiver constellation unit is predicated on a phenomenon referred to as Geometric Dilution of Precision (GDOP). The Hardware Description solution of a unique three-dimensional location based upon In the preferred embodiment, the processor unit is com trilateration requires the precise resolution of the common prised principally of a constellation of five (5) ultrasonic 15 intersection of multiple spheres circumscribed by the dis transducers and signal processing circuitry, thereof, and a tance between each transmitter and receiver transducer. signal processor that interfaces to this receiver group, per Each sphere has an inexact radius due to system noise and forms the pose calculations, and interfaces to the transpon measurement resolution. Therefore, the intersection ders and host computer databases. The following interface becomes a volnme instead of a point and the size of the descriptions for the processor unit are based upon the 20 volnme is dependent upon the radii of the intersecting dependency flow represented by FIG. 5. spheres as well as the distance between the spheres' centers. The sensors 14 preferably used for the receiver constel As the radii get larger with respect to the distance between lation unit are cylindrically-shaped ultrasonic transducers, the centers, i.e., the transmitter is farther down range, the for example, the model US40KR-0l 40 kHz PVDF ultra spheres begin to appear more and more tangential to one sonic receivers manufactured by Measurement Specialties 25 another and the intersection volnme increases, although not Inc., which provide adequate acoustic pressure sensitivity necessarily symmetrical in all dimensions. Therefore, to and exhibit 360 degree onmidirectional broad beam minimize position uncertainty, the receiver transducers response along the horizontal plane. The onmidirectional should be separated from each other as much as practical characteristic, albeit in one plane only, is very desirable to proportions allow with respect to the confines of the tracking minimize line-of-sight occlusion. Because of its low reso 30 field volume as the above said geometric examples provide. nance Q value, the rising and decay times are much faster This receiver constellation unit can be repositioned with than conventional ceramic transmitters. This reduces its respect to the tracking field by a simple mechanical adjust power requirements since less burst drive duration is needed ment as shown in the preceding figures. The mechanical to achieve sufficient triggering thresholds at the receiver. adjustment raises and lowers and changes the length and This transducer type is also utilized similarly in the tran 35 pivot axis of the cantilever ann which is fixed to a ground sponders to provide the potential for the most optimal base support. acoustic coupling. Because the receiver constellation unit operates a distance The receiver constellation unit is preferably mounted on from the processor unit, each receiver preferably has an a fixed support base, and has a pivoting and/or swiveling associated pre-amplifier circuit to convert the high-input mechanical linkage which provides an adjustable mecha 40 impedance piezoelectric signal into a low-level voltage nism for configuration of the receiver constellation unit's proportional to the acoustic signal energy impinging the inertial frame of reference relative to the tracking field. In transducers sufficient in order to accurately transmit the the preferred embodiment, it is strategically positioned and signals to the processor unit. In one embodiment, a high oriented in proximity to the tracking field in order (1) to input impedance AC amplifier design with 30 dB gain can be minimize line-of-sight degradation with respect to the 45 utilized. The preferred operational amplifier is the OPA373 expected transponder orientation, (2) to optimize registra manufactured by Texas Instruments. It was chosen for its tion resolution with respect to field volume size, and (3) to low 1 pA input bias current, high 6 MHz GBW, and satisfY the mathematical restrictions of performing trilatera low-voltage single supply operation. The amplifier is con tion calculations based upon the solution of simultaneous figured as a non-inverting type with the high-pass cutoff linear equations. It should be noted that the trilateration 50 frequency set at 1 kHz. The overall circuitry is preferably matrices may be solved if the matrices have a rank of five, enclosed in a metal shield to minimize electromagnetic noise and are non-singular, i.e., the matrix detenninant is non coupling into the highly sensitive amplifier inputs. In addi zero. In the preferred embodiment, the geometric parameters tion, a local, regulated power supply is included to allow for and their coordinate location of the receiver constellation a wide range of input voltage supply and provide sufficient must insure linear independence of the columns of the 55 power supply rejection to compensate for the noise suscep matrices and to avoid the matrices from becoming singular. tibility of remote power distribution. All the pre-amplifier One example geometrical pennutation of the receiver circuits' power and signal counections preferably originate constellation unit that satisfies these rules is shown in FIG. from the processor unit. 6A. It occupies a volnme of approximately 8 cu. ft. and The processor unit subsystem preferably consists of an essentially fixes the transducers in a way that defines two 60 analog signal processing interface that provides (1) addi primary orthogonal, bisecting planes defined by three non tional voltage amplification and filtering of base band signal collinear points each. Another preferred implementation that from the preamplifiers, (2) absolute value function, (3) peak occupies nearly the same volume is shown in FIG. 6B and detection function, and (4) analog-to-digital comparator is characterized by its S-shaped curve and tilted with respect function to provide support for an adaptive threshold means. to the horizontal plane. Another preferred implementation 65 The adaptive threshold technique provides robust triggering that occupies nearly the same volume is shown in FIG. 6C of the most proximal ultrasonic source at a precise temporal and is characterized by its helical or logarithmic spiral shape point along the traversing sinusoidal waveform of the elec- A000023 Case: 12-1252 Document: 46 Page: 114 Filed: 08/29/2012 JX-001.0024 US 7,292,151 B2 21 22 trical signal. Essentially, a new threshold signal is recalcu sample capacitor which faithfully tracks the rising voltage. lated each analysis period based upon a small percentage Once the input voltage diminishes in magnitude, the first reduction of the last peak wavefonn detected. Therefore, the blocking diode reverse biases and the sample capacitor holds tracking range is not necessarily restricted due to an arbi an accurate replica of the highest voltage attained with trarily high threshold setting and the noise immunity is minimal droop because of the low input bias current of the improved as the threshold tracks the wavefonn envelope and amplifier and elimination ofleakage altogether in the second not transient disturbances. An alternative automatic gain blocking diode by bootstrapping its cathode at the same control strategy for the amplification function is unnecessary potential provided by the low-impedance buffer of the since the trigger threshold will adjust to the signal level second output stage. An electronic switch and bleed resistor instead. In the preferred embodiment, the threshold faith 10 allow the voltage across the sample capacitor to be reset by fully tracks the peak to minimize integer period phase errors, the processor during power up and after the triggering event so the amplifier's gain is set to prevent signal saturation from is recorded so the adaptive threshold value can be refreshed occurring when the receiver constellation unit and transpon each cycle. A 1st order Butterworth filter may be used at the ders are in closest proximity during normal use. input to smooth false in-band transients that could disrupt In one embodiment, an amplifier and BW (band width) 15 the peak accuracy detection. filter circuit receives the output from the sensor and pream In the preferred embodiment, a comparator circuit plifier circuit and provides additional amplification and receives the output from the peak detect and sample-hold low-pass filtering to condition it for reliable threshold trig circuit to convert the analog signal to digital form for gering and input to other analog signal processing circuitry. high-speed triggering operation of the processor. The pre- A dual amplifier configuration may be used to provide an 20 ferred device is the MAX941 which is manufactured by additional gain of 40 dB, AC coupling to remove DC offsets Maxim. A percentage of the peak threshold is used to set the of the preamplifier outputs and long cable losses, and inverting input. When the non-inverting voltage exceeds the low-pass filter to reject noise beyond the interest signal's inverting voltage, the comparator's output will trip and bandwidth. The first stage amplifier may be configured as a produce a high-true logic pulse that triggers the processor. A non-inverting type with a gain of20 dB. The low-impedance 25 latch control input allows the processor to disable the DC input signal is effectively blocked by the coupling comparator action to prevent urmecessary triggering during capacitor in series at its non-inverting input with a high-pass the reverberation phase and to prevent potentially disruptive frequency cutoff set at 20 kHz. This gain stage feeds a noisy output chattering near threshold crossover beyond its second amplifier configured as low-pass, 2nd order Butter hysteretic immunity. The percentage of threshold level is worth MFB filter. This filter type provides smooth pass band 30 predetermined through the scaling resistors to be set low response and reduced sensitivity to component tolerances. enough to trigger on the rising edge of the signal's first crest The second stage low-pass frequency cutoff is set at 80 kHz at the furthest range of transponder operation, but high with a pass band gain of 20 dB. enough above the intrinsic system noise level and external An absolute value circuit receives the output of the noise caused by reverberation and other ultrasonic sources. amplifier and BW filter circuit and converts the bipolar 35 Once the first crest is registered, subsequent crests may be signal into a unipolar form for magnitude detection. A dual triggered at their zero-crossing representing the most precise amplifier configuration may be used to provide highly timing registration by momentarily disabling the sample accurate full wave rectification of the millivolt-level signal. hold circuit. Because of the longer duration trigger recep The first stage amplifier feedback switches to control the tivity window, early multiple reflections are mitigated by distribution of input current between the two signal paths 40 transducer placement at least 3.5 em away from adjacent dependent upon the input signal polarity. For a positive input planar surfaces, so the reflected acoustic energy doesn't voltage the input current will be positive which forward produce a canceling effect of the direct acoustic energy of biases Dl and reverse biases D2. This configures the 1st the later crests. Once a sufficient number of crests have been stage as an inverter driving the inverting input resistor of the registered, then the triggering window is blanked for the nd 2 stage, which is also configured as an inverter because its 45 remainder of the analysis period by latching the compara non-inverting input is held at virtual ground due to the tor's value. non-conducting path of D2. This effectively creates a com In the preferred embodiment, a digital signal processing bined circuit of two cascaded inverters for an overall gain of interface is connected to the analog signal processing inter +1. For a negative input signal its input current is negative face to transfonn the analog trigger processing into digital which forward biases D2 and reverse biases Dl. This 50 position information. configures the 1st stage as an inverter driving the non The digital filter circuit receives output from the com inverting input of the 2nd stage which changes the sign of the parator circuit and preferably consists of a digital low-pass circuit gain. In this mode, the input current is shared between filter implemented in a complex programmable logic device two paths to the input of the 2nd stage, where _2/3 of the input (CPLD) that serves to precondition the comparator circuit's current flows around the 1st feedback stage and -l!3 flows in 55 digital outputs. The preferred device is an AT1504ASVL the opposite path around the 2nd stage feedback path for a net CPLD which is manufactured by Atmel. Base band system gain of -1. noise or other glitches potentially occurring in the analog In the preferred embodiment, a peak detect and sample signal processor interface, but prior to the actually arrival of hold circuit receives the output of the absolute value circuit the ultrasonic signal, could cause a threshold disruption that and registers a peak value that is required to set a magnitude 60 registers a "runt" pulse as a false trigger condition. The threshold precisely at some percentage of full-scale of the "runt" pulse would be misinterpreted as the actual TOF peak. A dual amplifier configuration may be used to provide trigger and cause serious error in the position calculation. An the highest ratio of high output slew rate to low droop. The ANDINOR one-hot state machine design may be used to first stage is typically in negative saturation until the input ignore level transitions that are not stable for at least Ih voltage rises and exceeds the peak previously stored on the 65 system clock frequencyx8 states, so only transitions of 4 flS sample capacitor at the inverting input. Now the amplifier or greater are passed through. The system clock delays acts as a unity gain buffer and the input voltage charges the introduced by the digital filter's synchronous state machine A000024 Case: 12-1252 Document: 46 Page: 115 Filed: 08/29/2012 JX-001.0025 US 7,292,151 B2 23 24 affect all channels the same and are, therefore, effectively ber and Product Description strings. A host computer may eliminated by the inherent dependency on relative measure enumerate and access this device utilizing the manufactur ment. er's virtual COM port device drivers using a USB channel. In the preferred embodiment, the processor and digital In the preferred embodiment, the radio link circuit is filter circuits receive the output from the analog processor comprised of a wireless bidirectional communication inter and provide controlling signals therein. The preferred pro face to preferably (1) broadcast a synchronization signal to cessor circuit is the MC9S08GB60 which is manufactured control the transponders interoperability, (2) to receive other by Motorola Inc. It is a low-cost, high-performance 8-bit transponder sensor data, including, but not limited to, accel microcontroller device that provides all the aforementioned erometer, heart rate, battery, user I/O status, (3) to provide hardware circuits integrated into one convenient device. The 10 control messages for the transponders' sensory interfaces, calculation circuit is abstracted from embedded 60 KB and (4) to provide means to configure transponders' local FLASH for program memory with in-circuit programmable databases. The preferred wireless communication link is capability and 4 KB RAM for data memory. The time base based upon the AT86RF211, a highly integrated, low-power circuit is preferably comprised of an external, high-noise FSK transceiver optimized for license-free ISM band opera- immnnity, 4.0 MHz system clock, which multiplies this by 15 tions from 400 MHz to 950 MHz. and manufactured by the internal frequency-locked loop for a bus clock of 40.0 Atmel. It supports data rates up to 64 kbps with data clock MHz and single instruction execution time of 25 llS. This recovery and no Manchester Encoding required. The device clock also provides all the capture and control timing has a three wire microprocessor interface that allows access fnnctionality for the other specified circuits. Multiple par of read/write registers to setup the frequency selection, allel I/O ports and dedicated asynchronous serial commu 20 transmission mode, power output, etc. or get infonnation nication signals provide for the digital control of the analog about parameters such as battery, PLL lock state, etc. In signal processing and commnnication interfaces, respec normal mode, any data entering its input channel is imme tively. diately radiated or any desired signal collected by the aerial The timing capture-control circuit receives the output is demodulated and transferred to the microprocessor as from the digital filter circuit representing the arrival of the 25 reshaped register bit information. In wake-up mode, the TOF triggers to detennine the relative TOF propagation of device periodically scans for an expected message sequence the ultrasonic acoustic wave as it passes through the receiver and broadcasts an interrupt if a correct message is detected. constellation nnit. More specifically, it is comprised of a five In the preferred embodiment, at least three (3) consecutive channel 16-bit timer input capture module with program TOF timestamps are registered for each receiver during the mable interrupt control that provides edge detection and 50 30 acquisition phase. Preferably, the transponder's transducer llS timing precision to automatically register the TOF trig emits a multi-cycle ultrasonic acoustic burst of at least ten gers timestamps asynchronously without using inefficient cycles in duration so that sufficient energization of the and less accurate software polling means. receiver transducer is realized and at least three crests of the The phase-locked loop circuit receives the output from the waveform can be properly registered. At low signal levels timing capture-control circuit and is preferably comprised of 35 when ultrasonic acoustic coupling is poor, this requirement a three channel, 16-bit timer compare module is imple may fail and an invalid tracking status is asserted. Prefer- mented as an all-digital phase locked loop (ADPLL), which ably, the reference receiver transducer of the receiver con synchronizes the capture window and blanking functions stellation unit is positioned in closest proximity to the with respect to the reference input channel. It is comprised acoustic signal source so that it is the first transducer to be primarily of a free-running 16-bit timer configured to peri- 40 affected by the initial wave front. This reference receiver odically interrupt the processor dependent upon a precise provides the overall system timing and state machine control convergence of its period and phase to the reference trigger for the phase-locked loop circuit, so that the processing, source, by means of an over/under count matching and calculation, and commnnication tasks are executed in a correction technique. deterministic and efficient fashion. The AID conversion circuit receives the output from the 45 It should be appreciated that a high-resolution ultrasonic amplifier and BW filter circuit and consists of an eight acoustic tracking system that depends upon threshold detec channel lO-bit analog-to-digital converter used to monitor tion means has an inherent nncertain trigger dilemma. This channel offsets and magnitudes for range and polarity errors uncertainty arises because of the multi-cycle nature of the and correction. This information is utilized by the calcula transmitted signal's waveform and the associated difficulty tion circuit as input to the TOF software correction algo- 50 detecting the exact temporal location for consecutive analy rithm to detennine the slope of the wavefonn crest. sis periods when the signal's magnitude may vary greatly In the preferred embodiment, the serial communication depending upon the efficiency of the acoustic coupling, the circuit is comprised of two asynchronous serial communi distance between transmitter and receiver, and signal-to cation interfaces that are connected between the calculation noise ratio of the signal processing techniques. If a threshold circuit and host link and radio link circuits of the commu 55 is set near one of the minor crests of the wavefonn during nication interface. The host link provides a 115K bit per the last analysis period, then it is conceivable a slight second (baud) bi-directional communication link to an aux reduction of magnitude of the waveform during the next iliary host computer system through a Serial-to-Universal analysis period may fall slightly below the set threshold and Serial Bus bridge. The preferred device is the CP2101 which actually not be triggered nntil the next larger excursion of is manufactured by Silicon Laboratories. It supports the 60 the wavefonn occurs. This would create a TOF error pro conversion of a fully asynchronous serial data bus protocol, portional to the period of the acoustic waveform or its with buffering and handshaking support, to an integrated intra-pulse interval and have a detrimental affect on the Universal Serial Bus (USB) Function Controller and Trans measurement accuracy. This analog processing described ceiver and internal clock providing USB 2.0 full-speed above establishes trigger thresholds that allow no more than compliancy. An integrated 512 bit EEPROM stores the 65 a single intra-pulse interval of nncertainty, but that is still required USB device descriptors, including the Vendor ID, inadequate for high-resolution measurements. Although a Product ID, Serial Number, Power Descriptor, Release num- technique is known that controls the largest peak profile of A000025 Case: 12-1252 Document: 46 Page: 116 Filed: 08/29/2012 JX-001.0026 US 7,292,151 B2 25 26 the transmitter acoustic signal and claims to provide an correction algorithm. The initial condition that precedes the absolute trigger condition, this procedure is difficult to start of the relative compensation algorithm may be due to reliably tune and control among different transducer types. the resumption of a stable, locked tracking state after recov In the preferred embodiment of the invention, no modu ery from a fault condition and, therefore, requires compu lation of the acoustic signal is required. Rather, the adaptive tation of a set of reference TOFs producing minimum range threshold method is augmented with a TOF software cor error as a starting basis. The algorithm utilizes a wireless rection algorithm that unambiguously detennines the correct synchronization means to determine a reference TOF cal TOF based upon a means to detect the same carrier wave culation between the transponder and reference sensor of the cycle of ultrasonic energy every period. The software cor receiver constellation. By computing the reference range rection algorithm requires multiple, consecutive TOF acqui 10 distance by the product of the reference TOF and speed of sitions as input for the digital over-sampling and averaging sound in air, this reference range may be compared to the algorithm, the calculation of a higher-order numerical dif range calculated from the matrices solutions described ferentiation of the past and current TOF information as input for the predictive algorithm of higher-order Taylor series below. By iteratively and sequential increasing and decreas- based derivatives used for the relative TOF correction, and 15 ing the TOFs by a single intra-pulse time interval and a measurement of the intra-pulse time intervals of consecu applying the input to matrices formulations described below, tive TOF acquisitions as input for the absolute TOF correc all possible combinations of compensation are permutated tion scheme that minimizes the range error based upon and tested, which produces a unique set of TOFs that selective biasing of the TOFs. minimize the error between the calculated range distance The calculation circuit preferably processes multiple, 20 with respect to the reference range distance. This unique set consecutive TOF acquisitions to effectively improve the of initial TOFs serves as the starting basis for the relative timing resolution that proportionally affects position accu compensation algorithm. In the preferred, embodiment, this racy and precision. The digital filter discussed above intro absolute compensation algorithm works most effectively duces quantization errors because of its discrete operation. when (1) the wireless synchronization means is tightly And minor fluctuations in the acoustical coupling produces 25 coupled to the excitation of the acoustic source, (2) the timing jitter or uncertainty in the triggered zero-crossings of synchronizing signal's arrival is timed by the same mecha the acoustic sinusoidal. A Gaussian average or mean value nism that times the arrival of the reference transducer's of multiple TOF is a simple and effective filter strategy. Due acoustic signal, and (3) the coordinate locations of the to the possibility of poor acoustic coupling or misalignment, sensors of the receiver constellation are established to a high and distant transponder location from the processor unit, the 30 degree of accuracy. number of detectable triggered zero-crossings may vary for The calculation circuit preferably employs two software a fixed duration of multi -cycle ultrasonic acoustic burst. The methods of trilateration calculation to estimate transponder averaging algorithm automatically adjusts to this condition position, wherein the particular method used depends upon by only including TOFs whose delta changes fall within the the availability of a synchronizing signal and the accuracy expected range of the nominal intra-pulse interval defined by 35 desired. The first method is based on a relative TOF calcu the transmission properties of the acoustic source. The lation and the speed of sound is treated as a constant nominal intra-pulse interval is detennined and utilized by estimated at ambient indoor room temperature. The second the following compensation schemes. method requires calculation of an additional TOF timestamp The calculation circuit preferably processes a relative between the transponder and reference receiver, but calcu TOF correction algorithm based upon a predictive tuned 40 lates the speed of sound as an unknown every analysis algorithm that requires higher-order numerical differentia period, and thus improves measurement accuracy. The first tion calculation of the past and current TOFs. This compen method eliminates the global system timing variances and sates the TOFs that may have registered one intra-pulse delays due to the multiplicity of signal conditioning circuitry interval earlier or later than the nominally expected time due and eliminates the need for a controlling signal means to the trigger dilemma described above. By fonnulating 45 synchronized at the generation of the transmission of the these derivatives into a truncated 2nd order Taylor series ultrasonic acoustic wave. The second method also employs expansion and weighting the terms contribution, an estimate relative TOF calculation but requires an additional synchro of expected TOF is calculated and compared to the actual nization signal from the processor unit to determine the TOF through an iterative error minimization calculation. A absolute TOF between transponder and reference receiver. minimized error that results in a delta time change indicative 50 Since the absolute TOF is based upon a single chaunel only, of a discrete intra-pulse interval increase or decrease due to its timing latencies can be readily accounted for and easily an early or late TOF, respectively, produces a characteristic corrected. This method computes the speed of sound every value that directs the algorithm to compensate the actual analysis period, provided the synchronization signal is TOF by the intra-pulse interval and restore it to its correct detected, without need for additional hardware temperature value. In the preferred embodiment, this relative compen- 55 processing or requiring more then five (5) receivers, and sation algorithm works most effectively when, (1) the maxi automatically accounts for the system's main accuracy limi mally expected inter-period TOF change is less than the tation of speed of sound in air as defined by Eq.1.1, if discrete intra-pulse interval, (2) the TOF inter-period pro uncorrected, yields a 1.6 mmlm ranging error for every 10 C. cessing is contiguous, (3) the TOF increase or decrease is no temperature shift. If the synchronization signal is not more than a single intra-pulse interval, and (3) the Taylor 60 detected and, therefore, the second method is not resolvable, series tenns are suitably weighted in the prediction algo the last calculated speed of sound can be utilized within the rithm. first method's calculation to minimize error. The calculation circuit preferably processes an absolute TOF correction algorithm at least once initially, when the c=34.6 rn/s+0.5813 rn/s(Tc--25c C.) (1.1) phase-locked loop is stable, but may be perfonned every 65 analysis period depending on computational resources, that The TOF timestamps and speed of sound values are input determines the initial set of TOF values for the relative into linear independent algebraic equations in a matrix A000026 Case: 12-1252 Document: 46 Page: 117 Filed: 08/29/2012 JX-001.0027 US 7,292,151 B2 27 28 formulation to solve for the unknown transponder(s) posi Since each receiver is fixed at a distance Di from the tion, in a form as shown in Eq.2.1, transponder as determined by the receiver constellation geometry and because the acoustic waves propagate spheri cally, by using Pythagorean's theorem the following set of A·X =B (2.1) 5 range equations are defined in Eqs.5.1-5, all al2 al3 a14 (x l-U )2+(y l-V)2+(Z 1-W)2 =D12 (5.1) a2l a23 a22 a24 2 A= (x2-u )2+(Y2-V)2+(Z2- W)2 =D2 (5.2) a3l a32 a33 a34 a4l a42 a43 a44 10 (xru)2+(yrv)2+(zrw)2=D32 (5.3) Xl (X4-U)2+(y4-V)2+(Z4-W)2=Dl (5.4) X2 X= X3 (XS-U)2+(yS-V)2+(ZS-W)2=Ds2 (5.5) X4 15 Equivocally, the four (4) non-reference receivers are pref bl erably located at an incremental distance relative to the reference receiver, so by substitution of the incremental b2 B= distance defined by Eq.6.1, the following set ofrelativistic b3 20 range equations are defined by Eqs.6.2-5, b4 (6.1) To solve for the unknowns X, Eq.2.1 is rearranged as (6.2) shown in Eq.3.1, whereas the inverse of A requires compu tation of the cofactor matrix A C for the adjoint and determi- 25 (6.3) nant calculations for Eq.3.2 and Eq.3.3, respectively, (6.4) (3.1) (XS-U)2+(yS-V)2+(ZS-W)2~(Dl+CLl.TlS)2 (6.5) X=A-l.B=(A'l.B IAI 30 By expanding and rearranging the terms of Eqs.6.2-5, a set All A2l A3l A4l (3.2) of four linear algebraic equations and four unknowns for the Al2 A22 A32 A42 first method algorithm, depicted in the matrix form of (A'l = Al3 A23 A33 A43 Eq.2.1, is defined by Eq.7.1, A14 A24 A34 A44 35 (3.3) Xl -X2 Yl - Y2 Zl - Z2 -cLl.Tl2 (7.1) Xl -X3 Yl - Y3 Zl -Z3 -cLl.Tl3 2 Xl -X4 Yl - Y4 Zl - Z4 -cLl.T14 To setup the coefficient matrix A, the utilization of five (5) 40 Xl -Xs Yl - Ys Zl -Zs -cLl.TlS receivers produces the following set of relative TOF equa tions defined by EqsA.1-4, (cLl.Tl2)2 + Ri - R~ U (4.1) (cLl.Tl3)2 + Ri - R~ W (cLl.T14)2 + Ri - R~ (4.2) 45 (4.3) where Rf = xf + yf + zf for 5 ;:: i ;:: 1 (7.2) (4.4) 50 The receiver locations are fixed within the system's inertial Alternatively, if the second method algorithm is used, the reference frame, while the transponder(s) are mobile with unknown range of the reference receiver Dl can be substi respect to the same reference frame and are defined as tuted by Eq.8.1, follows, Dl ~cToj, Ll.Tol~time of flight (TOF) from S(u,v,w) 55 to S(xjo Yl' Zl) (8.1) S(xo,yo,zo)==S(u,v,w)~unknown transponder loca And, by rearranging terms, it is depicted in the matrix form tion defined by Eq.9.1, (9.1) Xl -X2 Yl - Y2 Zl -Z2 -(Ll.TOlLl.Tl2 + o.SLl.TI2) u Ri - R~ Xl -X3 Yl - Y3 Zl -Z3 -(Ll.TOlLl.Tl3 + o.SLl.TI3) Ri - R~ 2 W Xl -X4 Yl - Y4 Zl -Z4 -(Ll.TOlLl.T14 + o.SLl.Ti4) Ri - R~ c2 Xl -Xs Yl - Ys Zl -Zs -(Ll.TOlLl.TlS + o.SLl.Tls) Ri - R~ A000027 Case: 12-1252 Document: 46 Page: 118 Filed: 08/29/2012 JX-001.0028 US 7,292,151 B2 29 30 Although similar results may be obtained by application of By examination of the matrices element equivalency of more computational efficient processes such as pivotal con Eqs.l1.2-3 and manipulation of terms so that the angles may densation or Crout's decomposition, the application of be found using the inverse tangent function, the following Cramer's rule was used to evaluate the first-order determi rotation equations Eqs.12.1-3 are derived, nant in Eq.3.3 using second-order determinants from Laplace expansion. The final transponder(s) position equa tions are defined by Eqs.1O.1-S. (12.1) (12.2) (10.1) 10 (12.3) (10.2) X3 sin8,sin8y ) Y3' - 8 cos8, - 1 z = at4 X3 (10.3) 15 cosBz_1 == cos()z from previous iteration (12.4) (10.4) (12.5) 8, = atan(~) for 1st iteration (10.5) 20 These calculations are performed through iterative step (10.6) processes which inherit angular approximations of the pre ceding steps until the final desired angular accuracy is (10.7) achieved by assuming the conditions of Eqs.12A-S. There- (10.8) 25 fore the rotation 8z ' roll, is first approximated by Eq.12.S; then the rotation 8x ' pitch, is approximated by Eq.12.1; and then the final rotation 8y , yaw or tum, is approximated by If the first method is used, D, the range of the transponder Eq.12.2. The next approximation of 8z utilizes the previous to the reference receiver from Eq.lOA may be calculated as value of 8z in Eq.12.3 and the similar steps are preferably a redundant confirmation of the Eqs.1O.1-3 calculations, 30 repeated until the desired accuracy is achieved. The tran provided the frame of reference origin and location of the scendental functions may be evaluated through a conven reference receiver are identical or their offsets accounted for. tional look-up table or by a power series expansion. If the second method is used, C, the speed of sound in air, Preferably, the overall analysis period duration is effec from Eq.lOA must be computed every analysis period if its tively trebled until the three (3) transducers' positions are value is anticipated to be used in the first method in the 35 calculated, which reduces the system's frequency response absence of a synchronization signal. and imposes an increased latency effect. Typically, robust The orientation of the transponders can be derived from a absolute orientation processing requires more stringent line similar utilization of the above algorithms for a transponder of-sight operation and is reserved for more sensitive, less configured with a triad of ultrasonic transmitters. The trans dynamic, and reduced ROM movement trajectories, e.g., ducers are preferably arranged in a triangular plane at the 40 balance and sway. Therefore, the latency effect is less transponder of sufficient area for the desired angular reso noticeable upon the real-time performance of the sensory lution. The sequential excitation of each transducer and interfaces. subsequent calculation of position by the aforementioned In the preferred embodiment, the interactive hand-held methods provides suffIcient information to determine orien transponders support a dual axis inertial sensor, which is tation by the inverse kinematic calculations of Eqs.l1.1-4, 45 operably configured to provide tilt (pitch and roll) orienta where the analysis is simplified by assuming the origin of tion in its horizontal mounting plane. The inertial sensor is rotations occurs about Tl and Tl23 represents the initial mounted in the intended operational horizontal plane with relative position matrix from this origin and Tl23 is the respect to the systems inertial frame of reference. Once the transformed or forward kinematic position matrix. sensors signals has been converted to an acceleration value (11.1) 0 X2COS()y + Z2smBy cosB (11.2) X3 y [ Rx(8)Ry(8)T123" 0 s1ll8x(X2S1118y -Z2cos8y) X3 smBxsmBy [ o -cOS8x(X2S1118y - Z2cos8y) -x3cosBxsmBy Xl COS()z + Yl sinBz X2COS()z + Y2sinBz X3 COS()z + Y3 sinBz r (11.3) 1 Rz({)r TI23 == -Xl sinBz + Yl cosBz -x2sinBz + Y2COS()z -x3sinBz + Y3cos Bz r ~ ~ ~ .: Xl = YI = Zl = 0 A Y2 = 0 A Y3 = Z3 = 0 for initial orientation (11.4) A000028 Case: 12-1252 Document: 46 Page: 119 Filed: 08/29/2012 JX-001.0029 US 7,292,151 B2 31 32 that varies between +/-1 g the tilt in degrees is calculated as defined as a small constant and based upon the actual static shown in Eqs.13.1-2, for pitch and roll, respectively. timing variance empirically measured. The smaller this value the more confidence there exists in the systems' ~a sin(A)1 g) (13.1) measurement capability. ~a sin(Ajl g) (13.2) In the preferred embodiment, the product of the numeri cally-derived 1st and 2nd order derivatives of the measured This outside-in ultrasonic tracking implementation, where position scaled by a frequency dependent gain provides a the transponders are mounted on the mobile object, produces computationally practical adaptive dynamic process noise inherent temporal delays due to the finite TOF registration estimate model. The derivative product term increases Qk and calculation delays after the transponder has already 10 proportionally for higher velocity and acceleration compo moved into a different position before the measurement is nents of motion, e.g., quick, abrupt directional changes, complete. This overall latency period is compensated and which effectively increases the gain and, therefore, means minimized through use of a Kalman filter data processing more confidence exists in the measurement rather than the algorithm to estimate the pose of the transponder by opti estimate. This provides faithful, low-latency response to mally and recursively combining past history, new measure 15 high-frequency motions. Conversely, the frequency scaling ments, and a priori models and information. Generally term decreases the predictive "overshoot" characteristic of speaking, the Kalman filter is a digital filter with time lower power, repetitive motion, e.g. slower, cyclic, ROM varying gains that are optimally determined through a sto trajectories, which effectively decreases the gain and, there chastic dynamical model of the motion. The overall goal is fore, means more confidence exists in the estimate rather to minimize filter lag while providing suffIcient smoothing 20 than the measurement. It should be appreciated this filter of the motion data. implementation provides superior tracking fidelity and com An adaptive, multi dynamic model is developed based parable smoothing characteristics as compared to practical upon the kinematic quality of the expected movement tra lengths of finite impulse response running-average filters jectory. The predictive kinematic model for the Kalman filter and various low-orders infinite impulse response filters. It is depicted in matrix form utilizing a truncated 2nd order 25 achieves enough predictive response to compensate for the Taylor series expansion as below in Eqs.14.1-2, inherent TOF and computation latencies, while providing and comparable smoothing properties of other filter types. (14.1) [:L =[~ ~ttl +[:l QK==IKq[ (Zk_1-Zk_3)(Zk_1- 2Zk_2 +Z 1_3)sin(zk_1-zk_3)]1 (16.1) 30 (16.2) (14.2) 0 In the preferred embodiment, a three dimensional (3D) [1 1: : o~:" 11:[.[:[ piecewise cubic curve interpolates a movement trajectory for smoothing and reduced sample storage for greater 35 memory effIciency. Preferably, four (4) sequential discrete The Kalman filter is now described for a single dimen control points of the n-length set of control points, the sion, although it is utilized for prediction and smoothing for sample resolution dependent upon the desired movement all position dimensions. The predictor stages consist of the granularity, and corresponding timestamp are needed to calculation of the state and the error covariance projection calculate in real-time the interpolated position between any equations. The state projector equation, EqlS.l, utilizes a 40 pair of control points. A Catmull-Rom spline algorithm is the discrete time-sampled difference equation of r calculated preferred method in that the path intersects the control points from Eq.1S.2. In other words, the numerically derived and would best approximate a movement that may have velocity and acceleration components of motion are linearly acute directional changes. The Catmull-Rom spline algo combined with the previously a priori position to estimate rithm is defined by Eqs.17.1-3, where the geometry matrix the new position. The corrector stages consist of sequential 45 Gk represents the matrix of three dimensional (3D) control computation of the gain, updated state estimate, and updated points. error covariance equations. The a posteriori state estimate, Eq.1S.4, is based on a linear combination of the weighted measurement residual and the last state estimate. 0 -1 2 -1 (17.1) 50 1 2 0 -5 3 J.l Ck(J.l) = G"i 0 4 -3 J.l2 (15.1) 0 0 -1 J.l3 (15.2) -O.SJ.l + J.l2 - O.SJ.l3 (17.2) 55 (15.3) 1 - 2.SJ.l2 + l.SJ.l3 Ck(J.l)=Gk O.SJ.l + 2J.l2 - l.SJ.l3 (15.4) -O.SJ.l2 + O.SJ.l3 (15.5) 60 (17.3) The new error covariance projector, Eq.1S.2, is it's pre The fl value is normalized and represents the % value viously computed value combined with the current process between the 2nd 3rd control points. To calculate the interpo nd noise covariance, Qk' which is tuned by an example model 65 lated value between the 1st and 2 or the n-l th and nth control derived from the measured motion dynamics shown in points, the value of first control point of the pair and the Eq.16.1. The gain's measurement noise covariance, Rio is value of the last control point pair are doubly entered into the A000029 Case: 12-1252 Document: 46 Page: 120 Filed: 08/29/2012 JX-001.0030 US 7,292,151 B2 33 34 geometry matrix, respectively. The appropriate dflldt is associated drivers. The preferred LED device is a CMD87 determined by the desired rate of playback of movement manufactured by Chicago Miniature Lamp. These LEDs' trajectory. To playback at the same rate as the recorded intensity is controlled by a white LED driver. The preferred session, and assuming fairly constant velocity, a timestamp white LED driver device is a MAX1570 manufactured by should also be saved at each control point registration so that Maxim. The white LED driver provides a maximum 120 rnA the fl calculation is correctly scaled by the delta time constant current source to each LED for optimal uniform interval. The n-Iength set of control points would be manu luminescence. The drive current can be proportionally regu ally registered by the user pressing a switch or automatically lated through external pulse width modulation (PWM) post processed by a sorting method where a control point is means from the processor circuit to modulate its brightness registered at the tangents of the trajectory having sufficient 10 level. Additionally, an electronic switch is connected in magnitude and/or experience sign changes which indicates series to each LED drive to individually control its active discontinuous or non-monotonic movement. state. By simultaneously controlling the PWM duty cycle The major functional interfaces of the transponder unit and active state of each LED, the light strobe can appear to preferably include the sensory interface, transducer inter smoothly migrate along the linear array in spite of its face, processor, and communication interface. The following 15 discontinuous operation. descriptions of the transponder unit are based upon the Preferably, the stimuli interface circuit provides the pri dependence flow represented by FIG. 6. mary aural stimulus by means of a 4 kHz piezo buzzer. The The sensor interface refers to the collective support for the preferred device is SMT-3303-G manufactured by Projects ultrasonic transmitter, heart rate receiver, and accelerometer Unlimited. This electromechanical buzzer requires an exter- circuits. The ultrasonic transmitter circuit is preferably gated 20 nal transistor drive circuit and digital control signal gated at by a pulse-width modulated (PWM) digital signal at nomi a rate near its resonant frequency. The buzzer inputs are nally 0.8% duty cycle of the 40 kHz resonant frequency, e.g., connected to and controlled by PWM means from the a single 250 fls pulse every analysis period, by the processor processor circuit to provide a gross volume adjustment circuit. The radiated ultrasonic signal strength is controlled which is dependent upon the amplitude of the drive signal. by gating a MOSFET transistor switch at a duty cycle which 25 optimally energizes the transducer's series resonant tank Additionally, the stimuli interface circuit provides the circuit for sufficient duration. The resonant circuit's reactive primary tactile stimulus by means of a vibrator motor. The components include an impedance matching inductor, the driver for the vibrator motor enables a 120 rnA DC current transducer's intrinsic capacitance, and a small damping source to excite the motor armature. The preferred driver device is the MAX1748 manufactured by Maxim. The resistive load. At resonance, a electrical damped sinusoidal 30 rotational speed of the motor's armature is controlled by with a potential up to -400 V pk_pk is developed across the transducer to sufficiently drive it at acoustical power levels PWM means from the processor circuit. practical for the system's intended range of operation. The processor circuit preferably receives input from the Enabling a lower duty cycle control through means of a stimuli interface, sensor interface, and the communication software algorithm monitoring the transponders range 35 interface and provides controlling signals therein. The pre would effectively lower the transponders power consump ferred processor circuit is the MC9S08GB60 which is tion and radiate less ultrasonic acoustic energy for close manufactured by Motorola Inc. It is a low-cost, high range operation when signal saturation and clipping is performance 8-bit microcontroller device that integrates the undesirable. Conversely, a higher duty cycle control would specialized hardware circuits into one convenient device. radiate greater ultrasonic energy to compensate for less 40 The software calculation engine circuit operates from an efficient, non-optimal acoustical coupling orientations of the embedded 60 KB FLASH for program memory with in- transponder with respect to the receiver constellation. circuit progranJillable capability and 4 KB RAM for data Optionally, two additional transducers may be driven in memory. The time base circuit is preferably comprised of an unison or sequentially from a different transponder assembly external, high-noise immunity, 4.0 MHz system clock, to support measurement of absolute rotation about a single 45 which multiplies this value by the internal frequency-locked or multiple axes, or provide calculated positional redun loop for a bus clock of 40.0 MHz and single instruction dancy for certain diffIcult line-of-sight applications. execution time of 25 llS. This clock also provides all the The heart rate receiver circuit wirelessly receives a 5 kHz capture and control timing requirements for the other speci heart rate signal from a Polar® transmitter belt. The trans fied circuits. Multiple parallel I/O ports and dedicated asyn mitter, worn around the chest, electrically detects the heart 50 chronous serial communication signals provide digital con beat and starts transmitting a pulse corresponding to each trol for the circuits of the parallel/serial I/O circuit. heart beat. The receiver captures the signal and generates a In the preferred embodiment, the graphic LCD and touch corresponding digital pulse which is received by the timing screen circuit is the primary user input device for database capture-control circuit of the processor interface. A software management for an interactive transponder configuration. algorithm processes the signal with known time-based aver- 55 For example, it may be a 128x64 graphical liquid crystal aging and an adaptive window filter techniques to remove display system (LCD) and associated 4-pin touch screen any extraneous artifact or corruption caused by interfering input device. A preferred LCD device is the 51553 manu sources. factured by Optrex and the preferred touch screen device is The accelerometer circuit consists of a low cost +/-1.5 g the TSG-51 manufactured by Apollo Displays. LCD display dual axis accelerometer that can measure both dynamic, e.g. 60 information, configuration commands, and bitmaps images vibration, and static, e.g. gravity or tilt, acceleration. If the can be loaded through the software calculation engine via a accelerometer is oriented so both its axes are parallel to the parallel memory interface to emulate a graphical user inter earth's surface it can be used as a two axis tilt sensor with face. A touch screen input device is connected to a controller a roll and pitch axis. circuit to decode soft key presses at areas over the graphical The stimuli interface circuit provides the primary visual 65 object. Preferably, the key presses are registered, filtered, sensory interface preferably comprised of a linear array of decoded, and processed by the controller and then trans five (5) bright, white light emitting diodes (LED) and ferred to the software calculation engine via an interrupt A000030 Case: 12-1252 Document: 46 Page: 121 Filed: 08/29/2012 JX-001.0031 US 7,292,151 B2 35 36 driven asynchronous serial communication channel of the device, which further comprises a second visual display for I/O interface. A preferred LCD controller is the UR7HCTS providing visual stimuli to a user in combination with the manufactured by Semtech. first visual display. The timing capture-control circuit provides controlling 6. The system of claim 1, wherein: means for the stimuli interface and portions of the sensor the output device comprises an array of light emitting interface. The stimuli interface is preferably comprised of a devices. five channel 16-bit timer PWM module with programmable 7. The system of claim 1, wherein: interrupt control which provides 250 llS timing resolution to the output device provides audible stimuli to the user. automatically modulate the circuits' drivers through variable 8. The system of claim 1, wherein: duty cycle control. 10 the output device provides tactile stimuli to the user. In the preferred embodiment, the AID conversion circuit 9. A system according to claim 1, wherein the first receives the output from the accelerometer circuit and communication device sends ultrasonic signals received by consists of a two channel lO-bit analog-to-digital converter the processing system for determining movement informa used determine the rotational angle of roll and pitch in the tion for the first communication device. accelerometer deviates from its horizontal plane orientation. 15 10. A system according to claim 1, wherein said system This information is communicated to the signal processor comprises a registration system adapted to be configured via the radio link. remotely by said first communication device. In the preferred embodiment, the radio link circuit is 11. A system according to claim 10, wherein said regis- comprised of a wireless bi -directional communication inter 20 tration system allows the user to record a reference move face (with a receiver and transmitter shown generally at 20 ment trajectory remotely using said first communication and 30) to (1) receive a synchronization signal for control of device. the transponders interoperability, (2) to transfer acquired local sensor data, including, but not limited to, accelerom 12. A system according to claim 1, wherein said first eter, heart rate, battery, user I/O status, to processor unit and communication device is further comprised of: (3) to provide means to configure its local database from 25 a user input device and display and wherein said first command of processor unit. The preferred wireless commu communication device is configured with multiple nication link is based upon the AT86RF211, a highly inte training applications and wherein the user may choose grated, low-power FSK transceiver optimized for license one training application to activate, and wherein said free ISM band operations from 400 MHz to 950 MHz. and user may download additional training applications to manufactured by Atmel. Its key features are described 30 said communication device. above. 13. A system according to claim 12, wherein said user In the preferred embodiment, the switch I/O circuit uses input device may be used to configure options customized a SPST push button switch for user input to control the for the user. system's operational states, start and stop program execu 14. A system according to claim 12, wherein said user tion, and function as feedback input to the program. A 35 input device may be used to authenticate user access and preferred device is the KSS231 SPST pushbutton switch open a user session. manufactured by ITT Industries. 15. A system according to claim 12, wherein said user What is claimed is: input device may be used to calibrate said first communi 1. A system for tracking movement of a user, comprising: cation device to establish a reference pose or reference a first communication device comprising a transmitter for 40 traj ectory. transmitting signals, a receiver for receiving signals 16. A system according to claim 1, wherein said first and an output device, said first communication device communication device is adapted to accept various mechani adapted to be hand-held; cal extensions pieces depending on the application desired. a processing system, remote from the first communication 17. A system according to claim 1, wherein said first device, for wirelessly receiving said transmitted signals 45 communication device transmits accelerometer signals to from said first communication device, said processing said processing system. system adapted to determine movement information for 18. A system according to claim 1, wherein said first said first communication device and sending data sig communication device transmits heart rate signals to said nals to said first communication device for providing processing system. feedback or control data; and 50 19. A system according to claim 1, wherein said first wherein said first communication device receives and communication device is further comprised of: processes said data signals from said processing system an inertial sensor and wherein said first communication and wherein the output device provides sensory stimuli device transmits signals containing orientation infor according to the received data signals. mation to said processing system. 2. A system according to claim 1, wherein said first 55 communication device is a transponder. 20. A system according to claim 1, wherein said first 3. The system of claim 1, wherein the first communication communication device is further comprised of: device further comprises: a nonvolatile memory. a first visual display for providing an interactive interface 21. A system according to claim 20, wherein the first for the user. 60 communication device is adapted to download customized 4. The system of claim 3, further comprising: user programs from the Internet to be uploaded to a remote a display device in communication with the processing system as the application program. system for providing sensory stimuli for the user 22. A system according to claim 20, wherein performance according to the transmitted signals from the first algorithms are stored in said memory. communication device. 65 23. A system according to claim 22, where said perfor 5. The system of claim 4, wherein the display device mance algorithms calculate custom information personal to indicates the movement direction of the first communication a user in real-time. A000031 Case: 12-1252 Document: 46 Page: 122 Filed: 08/29/2012 JX-001.0032 US 7,292,151 B2 37 38 24. A system according to claim 22, wherein said perfor 37. A system according to claim 36, wherein said receiver mance algorithm produces a Motivation Index that repre array is in the form of an S-shaped curve. sents the overall level of enthusiasm or enjoyment for a 38. A system according to claim 36, wherein position particular activity. information is calculated based on time of flight measure 25. A system according to claim 22, wherein said perfor ment of said ultrasonic signals. mance algorithm produces a composite numerical value 39. A system according to claim 38, wherein position derived from a weighted average of statistical performance information can be calculated without interference from indicators and subjective user input including at least one of occluding objects. the following items: relative scoring improvements, confor 40. The system of claim 1, wherein the processing system mity to a range of motion pattern, duration of participation, 10 is adapted to determine position information. high activity access rate, relative skill level improvement, 41. The system of claim 40, wherein said processing daily goal achievement. system is adapted to determine the error between the actual 26. A system according to claim 20, wherein a user's movement information of said first communication device session data can be saved to said memory for later retrieval. and a movement information defined by a reference move- 27. A system according to claim 1, further comprising: 15 ment trajectory. a second communication device, adapted to be hand held, 42. A system according to claim 41, wherein said pro in electrical communication with the first communica cessing system is adapted to send feedback signals to said tion device, with the processing system adapted to first communication device based on said error. determine movement information of the second com- 43. The system according to claim 42, wherein the output munication device relative to the first communication 20 device provides feedback stimuli to the user in response to device. the received feedback signals. 28. A system according to claim 27, wherein said pro 44. A system according to claim 43, wherein said feed cessing system is adapted to determine movement informa back stimuli are aural instructions to the user for guiding the tion for both said first and second communication devices user's movements to conform to said reference movement and to calculate a displacement vector from said movement 25 traj ectory. information. 45. A system according to claim 43, wherein said feed 29. A system according to claim 28, wherein said pro back stimuli are aural cues informing the user of encroach cessing system is adapted to compare said calculated dis ments of threshold conditions. placement vector to a reference vector position and to 46. A system according to claim 43, wherein said output calculate a numerical result. 30 device is an array of light emitting devices adapted to be 30. A system according to claim 29, wherein said pro strobed at an intensity, rate or pattern proportional to said cessing system sends feedback signals to said first commu error. nication device based on said numerical result. 47. The system of claim 1, wherein the processing system 31. A system according to claim 30, wherein a user's is adapted to determine acceleration information of the first communication device. movement efficiency can be determined. 35 48. The system of claim 1, wherein: 32. A system according to claim 27, wherein said pro the first communication device further comprises a sensor cessing system is adapted to determine movement informa for determining tilt information of the first communi tion for both said first and second communication devices cation device; and the first communication device is and wherein a vector is calculated and compared to a desired adapted for transmitting the tilt information to the reference vector to calculate a numerical result and wherein 40 processing system. said processing system sends feedback signals to said first 49. The system of claim 1, wherein: communication device based on said numerical result, said the first communication device comprises an interactive first communication device further comprised of an output interface such that movement of the first communica device for providing feedback stimuli to the user in response tion device controls the movement of an object in a to said received feedback signals. 45 computer generated virtual environment. 33. A system according to claim 27, wherein said pro 50. An apparatus for use in tracking movement of a user, cessing system is adapted to determine movement informa comprising: tion for both said first and second communication devices a transmitter for transmitting signals; a receiver for receiving signals wirelessly from a remote :~e~h:~~i:~e~::' :a~:t;::e:~d e!~~:?c~:n~~~~~~~ 50 processing system; devices are adapted to communicate with each other for wherein the apparatus is hand-held; synchronization purposes. wherein the receiver is adapted to receive feedback or 34. The system of claim 27, wherein the second commu control data signals from the processing system, the nication device comprises: 55 feedback or control data signals derived from processed an output device for providing sensory stimuli to said user information including movement information of the according to said received data signals. apparatus; and 35. A system according to claim 1, wherein said signals wherein the receiver receives the data signals from the transmitted from said first communication device are radio processing system and wherein the apparatus processes frequency signals. 60 the received data signals to provide feedback or control 36. A system according to claim 1, further comprising: information to the user. a receiver array in data communication with said process 51. An apparatus according to claim 50, wherein said ing system for receiving ultrasonic signals from said apparatus is further comprised of: first communication device and wherein said receiver a display for providing an interactive interface for the array sends data to said processing system for use in 65 user. calculating movement information for said first com 52. An apparatus according to claim 50, wherein said munication device. apparatus is further comprised of: A000032 Case: 12-1252 Document: 46 Page: 123 Filed: 08/29/2012 JX-001.0033 US 7,292,151 B2 39 40 an output device for providing sensory stimuli to said user 71. An apparatus according to claim 50, further comprised according to said received data signals. of: 53. An apparatus according to claim 52, wherein said an inertial sensor and wherein apparatus transmits signals output device is an array of light emitting devices. containing orientation infonnation to the processing 54. An apparatus according to claim 52, wherein the system. output device provides audible stimuli to the user. 72. An apparatus according to claim 50, further comprised 55. An apparatus according to claim 50, wherein said of: processing system is adapted to determine the error between a nonvolatile memory. the actual movement infonnation of said apparatus and a 73. An apparatus according to claim 72, wherein said reference movement trajectory. 10 apparatus is adapted to download customized user programs 56. An apparatus according to claim 55, wherein the from the Internet to be uploaded to the processing system as processing system is adapted to send feedback signals to the application program. said apparatus based on said error and wherein said appa 74. An apparatus according to claim 72, wherein perfor ratus is further comprised of an output device for providing mance algorithms are stored in said memory. feedback stimuli to the user in response to said received 15 75. An apparatus according to claim 74, wherein said feedback signals. perfonnance algorithm produces a Motivation Index that 57. An apparatus according to claim 56, wherein said represents the overall level of enthusiasm or enjoyment for feedback stimuli are aural instructions to the user for guiding a particular activity. the user's movements to conform to said reference move 76. An apparatus according to claim 74, wherein said ment trajectory. 20 perfonnance algorithm produces a composite numerical 58. An apparatus according to claim 56, wherein said value derived from a weighted average of statistical perfor feedback stimuli are aural cues infonning the user of mance indicators and subjective user input including at least encroachments of threshold conditions. one of the following items: relative scoring improvements, 59. An apparatus according to claim 56, wherein said conformity to a range of motion pattern, duration of partici- output device is an array oflight emitting devides adapted to 25 pation, high activity access rate, relative skill level improve be strobed at an intensity, rate or pattern proportional to said ment, daily goal achievement. error between the movement of said apparatus compared to 77. An apparatus according to claim 72, wherein a user's said reference movement trajectory. session data can be saved to said memory for later retrieval. 60. An apparatus according to claim 50, wherein said 78. An apparatus according to claim 50 adapted to operate transmitted signals by said transmitter are ultrasonic signals 30 in conjunction with a receiver array in data communication received by the processing system for determining move with the processing system for receiving ultrasonic signals ment infonnation for said apparatus. from apparatus and wherein said receiver array sends data to 61. An apparatus according to claim 50, wherein the the processing system for use in calculating movement processing system is adapted with a registration system information for said first apparatus. adapted to be configured remotely by said apparatus. 35 79. An apparatus according to claim 78, wherein said 62. An apparatus according to claim 61, wherein the receiver array is in the form of an S-shaped curve. registration system allows the user to record a reference 80. An apparatus according to claim 79, wherein move movement trajectory remotely using apparatus. ment information is calculated based on time of flight 63. An apparatus according to claim 50, wherein said apparatus is further comprised of: 40 measurement of said ultrasonic signals. a user input device and display and wherein said appara 81. An apparatus according to claim 50 further comprised tus is configured with multiple training applications, of an output device and wherein said apparatus processes each of which is selectively activated with the user said feedback data from the processing system and provides input device. stimulus from said output device to cue the user to move in 64. An apparatus according to claim 63, wherein said user 45 a predetennined direction to assess the user's ability to input device may be used to authenticate user access and balance. open a user session. 82. An apparatus according to claim 50 further comprised 65. An apparatus according to claim 63, wherein said user of an output device and wherein said apparatus processes input device may be used to calibrate said apparatus to said feedback data from the processing system and provides establish a reference pose or reference trajectory. 50 stimulus from said output device to cue the user based on the 66. An apparatus according to claim 50, further compris movement information of said apparatus in reference to at ing: least one of: a desired range of motion and a desired a remote visual display in communication with the pro location. cessing system for providing visual stimuli for the user. 83. An apparatus according to claim 82, wherein the 67. An apparatus according to claim 50 wherein said 55 desired range of motion be established by placing targets in transmitted signals by said transmitter are radio frequency the real or virtual world at predetennined locations. signals for transmitting information to the remote processing 84. An apparatus according to claim 50, wherein the system. apparatus is a first transponder adapted for communicating 68. An apparatus according to claim 50, wherein said with a second transponder, also hand held by the user. apparatus is adapted to accept various mechanical exten- 60 85. An apparatus for use in tracking movement of a user, sions pieces depending on the application desired. said apparatus being hand-held, comprising: 69. An apparatus according to claim 50, wherein said a transmitter for transmitting signals wirelessly to a apparatus transmits accelerometer signals to the processing remote processing system; system. a receiver for receiving signals from the processing sys 70. An apparatus according to claim 50, wherein said 65 tem, wherein the received signals are feedback signals apparatus transmits heart rate signals to the processing derived from comparing movement infonnation of the system. apparatus with a reference movement infonnation; A000033 Case: 12-1252 Document: 46 Page: 124 Filed: 08/29/2012 JX-001.0034 US 7,292,151 B2 41 42 an output device for providing stimuli to the user, wherein execution to increase range of motion of a predetermined the feedback signals are used to initiate aural stimuli to body part of the user. the user; and 89. An apparatus according to claim 85, wherein said a user-actuated button for providing input to the appara aural stimuli are cues to train the user for a predetermined tus. physical activity. 86. An apparatus according to claim 85, wherein said 90. An apparatus according to claim 85, wherein said transmitted signals by said transmitter are ultrasonic signals aural stimuli provides the user with on-going performance for use in obtaining movement information for said appa information and where movement information of said appa ratus. ratus is collected over a period to time to determine the 87. An apparatus according to claim 85 wherein said aural 10 user's ability to perform a particular movement or activity. stimuli are cues to guide the user's movements to conform 91. An apparatus according to claim 85, wherein said to a desired reference movement information. button is actuated by the user to signal an end of a user movement. 88. An apparatus according to claim 87, wherein said aural signals are cues to the user for proper movement * * * * * A000034 Case: 12-1252 Document: 46 Page: 125 Filed: 08/29/2012 JX-001.0035 UNITED STATES PATENT AND TRADEMARK OFFICE CERTIFICATE OF CORRECTION PATENT NO. : 7,292,151 B2 Page 1 of 1 APPLICATION NO. : 111187373 DATED : November 6, 2007 INVENTOR(S) : Kevin Ferguson and Donald Gronachan It is certified that error appears in the above-identified patent and that said Letters Patent is hereby corrected as shown below: In column 1, line 8, please delete "SUMMER" and insert -- SUMMARY --. In column 7, lines 11-12, please delete "can be can saved" and insert -- can be saved --. In column 39, line 25, please delete "devides" and insert -- devices --. Signed and Sealed this Twenty Second Day of April, 2008 JON W, DUDAS Director ofthe United States Patent and Trademark Office A000035 Case: 12-1252 Document: 46 Page: 126 Filed: 08/29/2012 Case: 12-1252 Document: 46 Page: 127 Filed: 08/29/2012 JX-003 UNITED STATES DEPARTMENT OF COMMERCE United States Patent and Trademark Office June 08, 2010 THIS IS TO CERTIFY THAT ANNEXED HERETO IS A TRUE COpy FROM THE RECORDS OF THIS OFFICE OF: U.S. PATENT: 7,492,268 ISSUE DATE: February 17,2009 By Authority of the Under Secretary of Commerce for Intellectual Property and Director of the United States Patent and Trademark Office N. WILLIAMS Certifying Officer A000036 Case: 12-1252 Document: 46 Page: 128 Filed: 08/29/2012 JX-003.0002 111111111111111111111111111111111111111111111111111111111111111111111111111111111 1111111111111111111111111111111111111111111111111111111111111 USOO7492268B2US007492268B2 (12) United States Patent (10) Patent No.: US 7,492,268 B2 Ferguson et al. (45) Date of Patent: *Feb.17,2009 (54) HUMAN MOVEMENT MEASUREMENT 4,645,458 A 2/1987 Williams SYSTEM 4,695,953 A 9/1987 Blair et al. 4,702,475 A 10/1987 Elstein et al. (75) Inventors: Kevin Ferguson, Dublin, OH (US); 4,751,642 A 6/1988 Silva et al. Donald Gronachan, Holtsville, NY 4,817,950 A 4/1989 Goo (US) 4,912,638 A 3/1990 Pratt 4,925,189 A 5/1990 Braeunig (73) Assignee: Motiva LLC, Dublin, OH (US) 5,148,154 A 9/1992 MacKay et al. 5,184,295 A 2/1993 Mann ( *) Notice: Subject to any disclaimer, the term of this 5,214,615 A 5/1993 Baur patent is extended or adjusted under 35 5,227,985 A 7/1993 DeMenthon U.S.c. 154(b) by 0 days. 5,229,756 A 7/1993 Kosugi et al. 5,239,463 A 8/1993 Blair This patent is subject to a terminal dis claimer. 5,255,211 A 10/1993 Redmond 5,288,078 A 2/1994 Capper et al. (21) Appl. No.: 11/935,578 (22) Filed: Nov. 6, 2007 (Continued) (65) Prior Publication Data FOREIGN PATENT DOCUMENTS US 2008/0061949 Al Mar. 13,2008 WO PCTlUS9617580PCT /uS9617 580 5/1997 Related U.S. Application Data (63) Continuation of application No. 111187,373, filed on OTHER PUBLICATIONS luI. 22, 2005, now Pat. No. 7,292,151. Allard, P., et ai, Three-Dimensional Analysis of Human Movement, (60) Provisional application No. 60/592,092, filed on luI. Human Kinetics (1995) 3,8-14. 29,2004. (Continued) (51) Int. Cl. Primary Examiner-Toan N Pham G08B 23/00 (2006.01) (74) Attorney, Agent, or Firm-Standley Law Group LLP (52) U.S. Cl...... 340/573.1; 340/407.1; 4341114 (58) Field of Classification Search ...... 340/573.1, (57) ABSTRACT 340/573.4,539.12,539.13,539.22,407.1, 340/825.36; 3811315; 4341112, 114 See application file for complete search history. A system for measuring the position of transponders for test ing and training a user to manipulate the position of the (56) References Cited transponders while being guided by interactive and sensory U.S. PATENT DOCUMENTS feedback through a bidirectional communication link to a processing system for the purpose of functional movement 4,337,049 A 611982 Connelly assessment for exercise and physical rehabilitation. 4,375,674 A 311983 Thorton 4,627,620 A 1211986 Yang 4,631,676 A 1211986 Pugh 15 Claims, 10 Drawing Sheets A000037 Case: 12-1252 Document: 46 Page: 129 Filed: 08/29/2012 JX-003.0003 US 7,492,268 B2 Page 2 u.s. PATENT DOCUMENTS 6,308,565 Bl 1012001 French et al. 6,315,673 Bl 1112001 Kopera et al. 5,320,538 A 611994 Baum 6,346,045 B2 212002 Rider 5,347,306 A 911994 Nitta 6,361,507 Bl 3/2002 Foxlin 5,372,365 A 1211994 McTeigue et al. 6,366,272 Bl 4/2002 Rosenberg et al. 5,375,610 A 1211994 LaCourse et al. 6,400,452 Bl 6/2002 Maynard 5,385,519 A 111995 Hsu et al. 6,430,997 Bl 8/2002 French et al. 5,405,152 A 411995 Katanics et al. 6,487,906 Bl 12/2002 Hock 5,423,554 A 611995 Davis 6,515,593 Bl 212003 Stark et al. 5,429,140 A 711995 Burdea 6,545,661 Bl 4/2003 Goschyet al. 5,466,200 A 1111995 Uerich 6,720,876 Bl 4/2004 Burgess 5,469,740 A 1111995 French et al. 6,749,432 B2 6/2004 French et al. 5,474,083 A 1211995 Church et al. 6,765,726 B2 7/2004 French et al. 5,485,402 A * 111996 Smith et al...... 702/160 6,774,885 Bl 8/2004 Even-Zohar 5,495,576 A 211996 Ritchey 6,834,436 B2 12/2004 Townsend et al. 5,516,105 A 511996 Eisenbrey et al. 6,876,496 B2 412005 French et al. 5,524,637 A 611996 Erickson 7,158,118 B2 112007 Liberty 5,577,981 A 1111996 Jarvik 7,236,156 B2 6/2007 Liberty et al. 5,580,249 A 1211996 Jacobsen et al. 7,239,301 B2 7/2007 Liberty et al. 5,584,700 A 1211996 Feldman et al. 7,262,760 B2 8/2007 Liberty 5,587,937 A 1211996 Massie 7,359,121 B2 * 4/2008 French et al...... 359/630 5,591,104 A 111997 Andrus et al. 7,414,611 B2 8/2008 Liberty 5,597,309 A 111997 Riess 200210183961 Al 12/2002 French et al. 5,616,078 A 411997 Oh 2006/0028446 Al 212006 Liberty et al. 5,638,300 A 611997 Johnson 5,641,288 A 611997 Zaenglein, Jr. OTHER PUBLICATIONS 5,645,077 A 711997 Foxlin 5,656,904 A 811997 Lander Brownstein, B., et al, Functional Movement in Orthopedic and Sports 5,659,691 A 811997 Durward et al. Physical Therapy, Churchill Livingstone (1997), 15. 5,702,323 A 1211997 Poulton Brugger, W., et al, Computer-aided tracking of body motions using a 5,703,623 A 1211997 Hall c.c.d.-image sensor, Med. BioI. Eng. & Comput, (Mar. 1978),207- 5,704,837 A 111998 Iwasaki et al. 210. 5,711,304 A 111998 Dower Codella, c., et ai, Interactive Simulation in a Multi-Person Virtual 5,715,834 A 211998 Bergamasco et al. WorldACM (May 3-7,1992),329-334. 5,720,619 A 211998 Fisslinger DeLoura, M., et ai, Game Programming Gems, Charles River Media, 5,759,044 A 611998 Redmond (2000) 200-204. 5,785,630 A 711998 Bobick Greenleaf, W.J., DataGlove, DataSuit, and virtual reality Advanced 5,785,631 A 711998 Heidecke technology for people with disabilities, Proceedings of the Seventh 5,790,076 A 811998 Sypniewski Annual Conference 'Technology and Persons with Disabilities,' 5,790,124 A 811998 Fischer et al. (Mar. 1992).211-214 5,792,031 A 811998 Alton Kasvand, T., et ai, Computers and the Kinesiology of Gait, Comput. 5,812,257 A 911998 Teitel et al. BioI. Med. Pergamon Press (1976) vol. 6 111-120. 5,838,816 A 1111998 Holmberg Kenmochi, A., et ai, A netwotk virtual reality skiing system-system 5,846,086 A 1211998 Bizzi et al. overview and skiing movement estimation, Symbiosis of Human and 5,850,201 A 1211998 Lasko-Harvill Artifact, (Jul. 1995) 423-428. 5,872,438 A 211999 Roston Kraus, A., Matrices for Engineers, Hemisphere Publishing Corp. 5,888,172 A 311999 Andras et al. (1987) 118-120, 124-126. 5,890,995 A 411999 Bobick Lengyel, E., Mathematics for 3D Game Programming & Computer 5,913,727 A 611999 Andoot Graphics, Charles River Media (2004) 76-78, 467-468. 5,929,782 A * 711999 Stark et al. 340/870.01 Medved, v., Towards a virtual reality-assisted movement diagnos 5,963,891 A 1011999 Walker et al. tics-an outline, Robotica (Jan.-Feb. 1994) vol. 12, 55-57. 5,989,157 A 1111999 Walton Mulder, A., Human movement tracking technology, School of 6,004,243 A 1211999 Ewart Kinesiology, Simon Fraser University (Jul. 1994) 1-14. 6,028,593 A 212000 Ro senberg et al. Ruby, D., Biomechanics-how computers extend athletic performance 6,043,873 A 3/2000 Ramer et al. to the body's far limits, Popular Science (Jan. 1982) 58-60. 6,050,822 A 4/2000 Faughnn Sandweiss, J., et al, Biofeedback and Sports Science, Plenum Press 6,050,963 A 4/2000 Johnson et al. New York (1985) 1-201. 6,054,951 A 4/2000 Sypniewski Scarborough, E.L., Enhancement ofAudio Localization Cue Synthe 6,066,075 A * 5/2000 Poulton ...... 482/8 sis by Adding Environmental and Visual Cues, Air Force Inst. Of 6,073,489 A 6/2000 French et al. Tech., Wright-Patterson AFB, OH. School of Engineering (Dec. 6,077,201 A 6/2000 Cheng 1992) 1-4. 6,088,091 A 7/2000 Ramer Smith, J., et ai, Virtual Batting Cage and Human Model, Virtual 6,098,458 A 8/2000 French et al. Human http://www.cs. berkeley.edu/'rcdavisl classes/cs294/, (Jun. 6,100,896 A 8/2000 Strohecker et al. 17,2004)1-5. 6,119,516 A 912000 Hock Zetu, D., et al, Extended range tracking for remote virtual reality 6,132,337 A 1012000 Krupka et al. aided facility management, Department of Mechanical Engineering 6,152,856 A 1112000 Studor The University of Illinois at Chicago, http://alpha.me.uic.edu/dan/ 6,162,191 A 1212000 Foxlin NsfPaper/nsfl.htrnl, (Apr. 19,2004)1-9. 6,164,973 A 1212000 Macri Codamotion: The science of real-time motion capture and analysis, 6,181,343 Bl 112001 Lyons webpages from http://www.chamdyn.comlindex.htrnl. (Apr. 17, 6,183,259 Bl 2/2001 Macri 2004) 1. 6,198,528 Bl 3/2001 Maynard IREX, Virtual Reality Technologies, webpages from http://www. 6,244,987 Bl 6/2001 Ohsuga irexonline.comlhow_it_works.htrn, (Apr. 19,2004) 1-2. A000038 Case: 12-1252 Document: 46 Page: 130 Filed: 08/29/2012 JX-003.0004 US 7,492,268 B2 Page 3 Polhemus, Patriot: The Fast and Affordable Digital Tracker, www. tion.cfm?id~91385.91409&dl+ACM&type~series&i (Jun. 10, polhemus. com, (Feb. 2004) 1-2. 2004) 1-4. Polhemus, Liberty: The Forerunner in Electromagnetic Tracking Europe is Bursting with Virtual Reality Ideas, But Developers Are Technology, www.polhemus.com. (May 2003) 1-2. Critically Strapped for Cash, webpages fromhttps:llwww/lexis.com! Success Story Profile: Innovative Sports Training, Motion Monitor, research/retrieve? _m~66dI7057cl b77fl97aledb9f5fadb (2002) 1-2. 87d&_browseType~Text, (Jan. 1993) 1-2. Reality built for two: a virtual reality tool, Symposium on Interactive 3D Graphics, ACM Press webpages from http://portal.acm.org/cita- * cited by examiner A000039 Case: 12-1252 Document: 46 Page: 131 Filed: 08/29/2012 JX-003.0005 u.s. Patent Feb. 17,2009 Sheet 1 of 10 US 7,492,268 B2 10 Y ------"') FIG-1A -~-ooIO~O 1 ~~: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 '"--I ~ __ ~------y ------) FIG-1B A000040 Case: 12-1252 Document: 46 Page: 132 Filed: 08/29/2012 JX-003.0006 u.s. Patent Feb. 17,2009 Sheet 2 of 10 US 7,492,268 B2 FIG-2A ~ I FIG-2B 1 kg I I \ FIG-2C FIG-2D A000041 Case: 12-1252 Document: 46 Page: 133 Filed: 08/29/2012 JX-003.0007 u.s. Patent Feb. 17,2009 Sheet 3 of 10 US 7,492,268 B2 , t Evaluate Display PIN Entry I Security & Navigation Requirements Controls . Request Q) ./ II) ~I[requlred]--. AuthentiCation}""-- -"") . c -'= c.. [not required] User Session ~ [identification] °C :::l u Q) Authenticatet ____ J U) • User User Session [loaded] t Configure Display ~ Session UI Graphical .. Incons of Name & ID Descriptor ,,/" 1 ,," Display Graphical D Request '" Incons of Workload Program ---" Intensity & Limits, Selection + Measurement User Session Criteria, Stimulus Properties, ect. [setup] rh I I Process I I Q) I Program 1+---_ .... Display ~ II) I C I Request Graphical -'= Request c.. Incons of ,,~ a. Session ---"'" ,,- Parameters :::l ~" -" Setup ,,~ +' Request & Instruction Q) ~ U) Program Text User Session ---" Setup ~ [setup] rh User Session Process I J [setup] rh Session +-----" Process Parameters J Program 1+---_ .... l Parameters i ~ [A or B] ( Deployment Phase FIG-3A A000042 Case: 12-1252 Document: 46 Page: 134 Filed: 08/29/2012 JX-003.0008 u.s. Patent Feb. 17,2009 Sheet 4 of 10 US 7,492,268 B2 ( Deployment Phase) i User Session ------, [save] rh Program [load] Process 1M Deployment - ______1 Requirements Evaluate Secondary 1M Display Graphical Requirement Incon for Detail of Modular Length, Weighted, or [not required] Augmentative Force Resistance [required] Attachments _c/__ ,> 1 / " i / i ,-~+.--~/ i ---...... i i i .. Program Program• [Deployment] [Deployment] i Fasten I Fasten i Primary 1M i i i Secondary 1M i i i i i i i , i , i \.. J --t-, --t--'\ .. J Display \ Graphical Display Incon of Graphical User Body Incon of User Body Location &: [ready] [ rea d]y Strap Holster Location &: Strap Holster ~eChan;sm Mechanism Calibration Phase FIG-3B A000043 Case: 12-1252 Document: 46 Page: 135 Filed: 08/29/2012 JX-003.0009 u.s. Patent Feb. 17,2009 Sheet 5 of 10 US 7,492,268 B2 ( Calibration Phase Process Requirements of 1M Calibration Determine Secondary 1M Requirement Display GraphicalD. Ineon of Primary 1M ~!nOI requiredl~ Calibration Status [required] Display Graphical~ Request 1 se~n;~~a~f 1M ,...... , Primary 1M Pose ---, -'= ~---+ ____ ~~..... ___ ~ ____[_t ....a~tc_h] Position ( Execution Phase FIG-3C A000044 Case: 12-1252 Document: 46 Page: 136 Filed: 08/29/2012 JX-003.0010 u.s. Patent Feb. 17,2009 Sheet 6 of 10 US 7,492,268 B2 Query ~--[Iearn]---...... Learn Modality [normal] Request Primary 1M Pose ---, Modification Program• [Record] User Session ---, I [Setup] I I CI> I Cf) 0 I .s::. I Process + c.. Primary 1M Program I:: 0 Pose [Execution] :,J :::l I 0 I CI>x Process I I.LI Learn I I Terminus I I I [not done] I J --~ [done] Process Program Terminus Program [not done] [Save] [done] •) FIG-3D Data Analysis Phase A000045 Case: 12-1252 Document: 46 Page: 137 Filed: 08/29/2012 JX-003.0011 u.s. Patent Feb. 17,2009 Sheet 7 of 10 US 7,492,268 B2 ------L-~ --r------=---I-~) -:~-:--~=:---] "~------~ I I I I I I I I I I / .- / .- .- --- -""" ...... 10 FIG-4B A000046 Case: 12-1252 Document: 46 Page: 138 Filed: 08/29/2012 JX-003.0012 u.s. Patent Feb. 17,2009 Sheet 8 of 10 US 7,492,268 B2 Remote Position Processor Analog Signal Digital Signal Processor Interface Processor Interface Receiver Processor Constellation r, ~-~, Timebase «access» .s I Sensor & AID .s I Preamplifier I- Conversion en I I M I I L..-____.r ...... -- I ....1 J I r---- I J \ I r- I I J r, I "" ------./ I r------) I I Phase-Locked I I J Loop I I r, I Amplifer & Absolute L...-_...,._---' ~ I r, , 15 I I Software BW Filter I Value "t) --l- I «access» Calculation ~-) Engine «a.:c:~~_...J ( ~ en en Timing CI) o Capture o c & PWM ~ r, Peak Detect I Comparator I r---- r, & _____ J I Sample-Hold ( Para lIel/Seria I «access» I «access» I I/O I I I I I I I r, I ~------Digital Filter Communication Interface I r------.I I «1M status» I I -.I ~~..--=-=-.;--n Link Radio Link L..-U_S_B_2_'O_J..... ------L...-___---' --- FIG-5 20 30 A000047 Case: 12-1252 Document: 46 Page: 139 Filed: 08/29/2012 JX-003.0013 u.s. Patent Feb. 17,2009 Sheet 9 of 10 US 7,492,268 B2 FIG-6A FIG-6E FIG-6e A000048 Case: 12-1252 Document: 46 Page: 140 Filed: 08/29/2012 JX-003.0014 u.s. Patent Feb. 17,2009 Sheet 10 of 10 US 7,492,268 B2 Transponder Stimuli Interface Processor I =jl r.L, 28MHz 1-, Fit n White LED I ~ ... ~ Tlmebase -----, & Driver r.L, e..... Vibrator 8 .c:::::::n Motor «clock»l ~ ¥ r.L, & Driver >- }I. PWM -----, f-, IJ r.L, «access>~ Audio «configure» Annuciator -----' & Driver ~ r===n. (J) r===n. T (J) r.L, CI) r.L, 0 0 Software ,- AID 0 Conversion ¥1"0._ Calculation Sensor Interface ~ Engine ~ =n ()~ r.L, E Ultrasonic Fit lioonngu.. J ~ Transmitter r.L, ¥ Parallel/Serial «access» ~ess»T I/O " J -----, r.L, T ~ -::- r.L, Accelerometer ~() :::l 0> Heart Rate t;:: .:: / Receiver 0 " 0 -""'''" ¥ i" I Dual Axis I I low g I I I I I Communication Interface I Polar compatible ( ,.----' ~ user I/O» ..... ( I -----, T -----, «user I/O» r.L, r.L, Graphic LCD r---- ,- & Switch I/o Touch Screen ( ) -----, «1M msg» r.L, 20, ~'\ n Radio Link FIG-7 30 ( )------900MHz ISM A000049 Case: 12-1252 Document: 46 Page: 141 Filed: 08/29/2012 JX-003.0015 US 7,492,268 B2 1 2 HUMAN MOVEMENT MEASUREMENT selecting the optimal technology for the most robust applica SYSTEM tion development. Irrespective of the intrinsic strengths and weaknesses of the tracking technology employed, ultimately This application is a continuation of U.S. Ser. No. 111187, the user's satisfaction with the system's utilization and effi 373 filed 22 luI. 2005, now U.S. Pat. No. 7,292,151, issued 6 cacy, including the production of reliable, easily understood, Nov. 2007, which claims the benefit of U.S. 60/592,092, filed measurable outcomes, will dictate the overall success of the 29 luI. 2004. Each of these applications is incorporated by device. reference as if fully recited. This invention's system and methods facilitates biome chanical tracking and analysis of functional movement. In the BACKGROUND OF THE ART AND SUMMARY 10 preferred embodiment, this invention is low cost, robust, easy OF THE INVENTION to deploy, noninvasive, unobtrusive, and conveys intuitive and succinct information to the user to execute movement This invention relates to a system and methods for setup properly and provides performance indicators of said move and measuring the position and orientation (pose) of tran ment for feedback purposes. One feature of the present inven- sponders. More specifically, for training the user to manipu 15 tion provides for an interactive tracking system because the late the pose of the transponders through a movement trajec sensor functionality, or referred to herein as active transpon tory, while guided by interactive and sensory feedback ders or transponders, is integrated with local user input con means, for the purposes of functional movement assessment trol, and real-time sensory interfaces on the same device. The for exercise, and physical medicine and rehabilitation. transponder is a wireless communication and monitoring Known are commercial tracking and display systems that 20 device that receives a specific signal and automatically employ either singularly, or a hybrid fusion thereof, mechani responds with a specific reply. In one embodiment, the inven cal, inertial, acoustical or electromagnetic radiation sensors tion provides functional movement assessment based upon to determine a mobile object's position and orientation, the relative measures of limb pose with respect to two posi referred to collectively as pose. tions defined by the transponders. The transponders can oper- The various commercial tracking systems are broadly clas 25 ate independently or work in unison to process and share sified by their relative or absolute position tracking capability, computational tasks and information between the local data in which system the pose of a mobile object is measured bases. This decentralized, distributed processing scheme relative to a fixed coordinate system associated with either allows the configuration and coordination of the training ses combination of receiver(s) or passive or active transmitter(s) sion, and processing and analysis of the measurements to housing mounted on the user. The tracking system's compo 30 occur without requiring expensive auxiliary computer and nents may be tethered with obvious inherent movement display systems to manage the same, and without relying on restrictions, or use wireless communication means to costly software development of complex synthetic environ remotely transmit and process the information and allow for ments for visualization purposes. Also, the user can manage greater mobility and range of movement. the applications and performance databases off-line on a Typically these tracking systems are utilized for biome 35 remote computer system with Internet connectivity to cus chanics and gait analysis, motion capture, or performance tomize and configure the system parameters in advance of animation and require the sensors to be precisely mounted on their session. the joints. Various means of presenting the tracking informa The present invention is designed to provide such system tion in a visual display are employed, such as Heads-Up and methods for high-fidelity tracking or registration of the Display (HUD), that provide occluded or see-through visibil 40 poses of active transponders and engage the user to purposely ity of the physical world, or Fixed-Surface Display (FSD), manipulate the transponders' pose along a prescribed or cho such as computer desktop monitors, depending upon the reographed movement trajectory in order to train and assess simulation and immersive quality required for the applica functional movement capability. In the preferred embodi tion. The application may require various degrees of aural, ment, the system is comprised of two subsystems: (1) a sub- visual, and tactile simulation fidelity and construct direct or 45 system comprised of one or more active transponders, which, composite camera views of the augmented or three dimen in its most sophisticated implementation, responds to peri sional (3D) virtual reality environment to elicit interactive odic requests from another component of the system to radi user locomotion and/or object manipulation to enhance the ate or transmit a signal for purposes of absolute position user's performance and perception therein. The tracked tracking; processes an embedded inertial sensor for relative object may be represented in the virtual environment in vari 50 orientation tracking and absolute tracking refinement; and ous forms, i.e., as a fully articulated anthropoid or depicted as provides an essentially real-time aural, visual, and tactile a less complex graphical primitive. The rendering strategy sensory interfaces to the user, and (2) a subsystem comprised employed depends upon the degree of photo realism required of a centralized position processor system or unit and receiver with consideration to its computational cost and the applica constellation unit, collectively referred to as the processor tion's proprioception requirements. 55 unit, which is essentially a signal processor that synchronizes Tracking technologies possess certain inherent strengths the transponders' periodicity of radiating signal and other and limitations dependent upon technology, human factors, operational states; collectively receives and processes the and environment that need consideration when discussing radiated signal; iteratively calculates the transponders instan their performance metrics. Regardless of differentiating reso taneous pose and convolution, thereof; and continually lution and accuracy performance benchmarks, many imple 60 exchanges this information, and its analysis thereof, with the mentations suffer from varying degrees of static and dynamic transponders and/or auxiliary host computer system in essen errors, including spatial distortion, jitter, stability, latency, or tially real-time via a combined wireless and tethered commu overshoot from prediction algorithms. Some human factors nication means. This real-time bi-directional exchange of include perceptual stability and task performance transpar information allows for proper transponder identification, ency, which are more subjective in nature. And environmental 65 coordination, and the accurate measurement of pose, thereof, issues such as line-of-sight, sensor attachment, range, and and timely actuation ofthe sensory interfaces for optimal user multiple-object recognition, need to be considered when regulated closed-loop control. A000050 Case: 12-1252 Document: 46 Page: 142 Filed: 08/29/2012 JX-003.0016 US 7,492,268 B2 3 4 The transponder is broadly classified by its level of hard a means to create a single movement vector whose end ware and software configuration that define its scope of intel points are defined by the locations of at least two tran ligence, sensory support, and configuration. The degree of sponders, wherein, the expansion and contraction of the intelligence is detennined by its capability to locally access, vector's length is calculated, analyzed, and reported in process, and modify the database. Further, either transponder essentially real-time; classification can be sub-classified by its manipulative a means to create a single movement vector whose end requirements. In one embodiment, where multiple transpon points are defined by the locations of two transponders, ders are used, a principle transponder is consciously and wherein, a representative point along the vector length is deliberately moved along the reference movement trajectory, referenced and its higher-order derivatives are computed while a subordinate transponder serves as an anchor or sec- 10 by mathematical numerical processes, wherein the ondary reference point elsewhere on the locomotion system result is calculated, analyzed, and reported in essentially whose kinematics are not necessarily controlled by the user's real-time; and, volition. a means to correlate said vector's length and at least one An interactive transponder, preferably, has significant other measure consisting of a higher-order derivative, to intelligence; supports relative and absolute tracking capabili- 15 the reference movement trajectory, wherein the result is ties; provides complete sensory stimuli support; provides for calculated, analyzed, and reported in essentially real- functional enhancement through attachment of modular, time. extension pieces; and provides a user display and input sys A registration system for practical functional movement tem to control the training session. In the preferred embodi applications should clearly convey information to the user ment, the interactive transponder is primarily held in the hand 20 regarding his movement quality while he performs the task, to facilitate more complex user input and greater sensory without compromising or distracting from said execution by intimacy. Conversely, in another embodiment, the fixed tran uunecessary head movements or change in eye gaze and sponder has limited intelligence; supports only the absolute normal focus. Poor visualization strategies that distract the pose tracking capability; provides no sensory stimuli support; user are ineffectual for promoting heads-up, immersive inter- and is usually mounted to a fixed site on the limb or trunk. 25 action, and the alphanumerical information it imparts often A combination of transponder deployment strategies may can not be consciously processed fast enough to elicit correc be required depending on the training session's objectives, tive action. This system provides for both a local, standalone such as two interactive transponders grasped by each hand; or sensory interface as a primary feedback aid, or alternatively, alternatively, an interactive transponder, and a fixed transpon an interface to a remote fixed-surface display for greater der attached to the limb or trunk; or lastly, two fixed transpon- 30 visualization and simulation capabilities. The visual stimulus ders attached to the limb(s) and/or trunk. could be modulated to warn of range violations, or provide In one embodiment, this invention proposes to elicit move signals for purposes of movement cadence and directional ment strategies based on the deployment of at least two tran cueing. A principle interactive transponder is typically hand- sponders that define the endpoints of a movement vector held, which is naturally in close proximity to the user's aural whose relative translation and rotation is measured and evalu- 35 and visual sensory field during most upper extremity move ated for the assessment of functional movement capability, ments, or, conversely, the visual stimulus may be viewed including but not limited to, limb range of motion and its through a mirrored or reflective means if not in optimal line control thereof, limb strength conditioning, and overall prop of-sight. A remote fixed-surface display might augment the rioception and hand-eye coordination skills, and overall body immersive quality of the user's experience by providing con movement. This registration system measures a single move- 40 trol of a view camera of a simulated computer environment, ment vector whose endpoints are comprised of an anchor and display of the transponders and/or interactive objects' point, i.e. one that is located in a less dynamic frame of static or dynamic poses within the computer display's skewed reference, e.g., such as the trunk or abdomen, and another through-the-window perspective projection. more distal location fixed on or held by a limb or extremity, In summary, one embodiment of the present invention is e.g., the hand, arm, or leg. As this movement vector is trans- 45 comprised of: lated and rotated through space by the act of the user modi a means for modulating an embedded luminescent display fying the pose of the principle transponder in concert with the organized and oriented into a directional-aiding pattern, reference movement trajectory, the vector's length will by varying its degree of intensity and color, or other expand and contract relative to the proximity of principle physical characteristics, to provide a visual display transponder with respect to the subordinate transponder. The 50 stimulus. This sensory interface is excited at a rate, rep vector's length conveys unique and explicit infonnation etition, or pattern proportional to the pose error of the regarding the user's movement efficiency and biomechanical transponders' movement traj ectory compared to the ref leverage. For example, by attaching a fixed subordinate tran erence movement trajectory; sponder at the hips and a fixed principle transponder on the upper arm, the biomechanics of the act of lifting a box or 55 a means to view said visual display stimulus with the aid of similar object can be elegantly qualified. If the user assumes a mirror(s) or other reflective means; a poor lifting technique, i.e. legs locked with the trunk a means for the real-time projection of sound or speech severely flexed with head down and the anns stretched out commands through an audio device to provide warning, beyond the basis of support, the vector's length would con alarm, instructional, and motivational aid, and/or addi sistently be measured longer than compared to a good lifting 60 tional cueing upon encroachment of static and dynamic technique, i.e., legs bent at knees with the back straight, head limit/boundary conditions defined by the reference gaze up, and arms close to body. Also, the measurement( s) of movement trajectory; higher-order derivatives derived from numerical mathemati- a means for real-time tactile feedback including, but not cal processes of a reference point described by the vector limited to, modulation of the rotational properties of a would provide additional indication of movement control or 65 vibrator motor proportional to the pose error of the tran smoothness. In summary, one embodiment of the present sponders' movement vector compared to the reference invention is comprised of: movement trajectory; A000051 Case: 12-1252 Document: 46 Page: 143 Filed: 08/29/2012 JX-003.0017 US 7,492,268 B2 5 6 a means for combining the excitation of said stimuli pro but may include external media devices, such as USB FLASH portional to the pose error of the transponders' move Key or other portable media means, that may have inter ment vector compared to the reference movement tra operability with other computerized devices. The data struc jectory; and, tures may include: a means to coordinate the real-time, periodic parametric Modulation & Feedback Thresholds/Triggers Properties update and modulation of the stimuli imparted by the the aural, visual, tactile interfaces require threshold settings sensory interfaces within the transponders from a pro which determine their excitation or stimulation characteris cessing unit by means of a wireless communication linle tics. These settings can be derived from previous performance This invention addresses the need for an intuitive, interac data or defaults determined from normative data, or modified tive method to instruct, create, and deliver a movement tra- 10 in real-time, by algorithmic methods including moving aver jectory command without necessarily relying on pre-pro ages, standard deviations, interpolation based upon goal-ori grammed, regimented movement trajectories. The ented objectives, etc. registration system can be configured via remote setup at the Normative Performance-performance data collected principle transponder to pre-record and choreograph a free over a large population of users through controlled studies, form movement trajectory of the principle transponder with 15 that is distilled down into specific user categories based upon the intent of the user mimicking the same said path. This certain demographics that the user may compare and rank impromptu learning modality can expedite the session down hislher results. This data may be initially embedded within time between different users and movement scenarios, and the transponders or position processor non-volatile memory accommodate users' high anthropometric variability in range and may be augmented or modified automatically or by user of movement. In summary, one embodiment of the present 20 volition when counected to the Internet. invention is comprised of: Competitive Ranking-applications which have a pre a means is to provide a movement trajectory learning dominate point goal-oriented purpose would allow access to modality that allows the user to calibrate and create the a global ranking file archive accessed through the Internet or desired endpoints, midpoints, and/or total reference automatically via updated executive files. This ranking file movement trajectory through user programmer entry of 25 would be created through an analysis ofuser participation and an input device resident on the transponder; publishing of hi s/her results through Internet Web-based ser a means to process and save a movement trajectory using a vIces. computationally efficient Catmull-Rom spline algo Downloadable Executive Programs & Configurations rithm or other similar path optimizing algorithms to new software programs, including new features, enhance create control points along key points of the movement 30 ments, bug fixes, adjustments, etc., could be downloaded to trajectory that define the optimally smoothest path inter the transponder through an Internet connection. Graphics secting the control points; images would be stored in compressed or uncompressed a means to provide database management by a processing binary forms, i.e., bitmap, gif, jpeg, etc. This new programs unit via a wireless communication link or, alternatively, could be transferred to any suitable computerized position through user data entry of an input device resident on the 35 processor unit located at a remote facility via the transpon interactive transponder; and, der's wireless link. Therefore, the user's transponder is the a means to access, edit, and store the program and/or data node that establishes the portable network capabilities of the bases to nonvolatile memory operably coupled to the system, not necessarily the computerized position processor. principle transponders for the purpose of automating the creation, delivery, storage, and processing of movement 40 Custom Menu Interfaces-specialized activities may trajectories. Customized user programs and databases require more advanced (or simplified) interfaces dependent would be downloaded from a central repository or rel upon the users' cognitive abilities and interactive specificity. evant website in advance of the training session to the This menu may include interactive queries or solicit informa transponder from the user's home location via the Inter tion regarding the user's daily goals, subjective opinions or net or other convenient locales having networked Inter- 45 overall impression of the activity and ones performance net access, and transported to the systems remote physi- which could be incorporated in the Motivation Index cal location, and uploaded into the system's memory, described below. and executed as the application program. Report Generation Tools and Templates-XML, HTML or This a priori process of remote selection, download, and other authoring language used to create documents on the transfer of programmatic content and database would mini- 50 Web that would provide an interactive browser-based user mize the user's decision making and input during product interface to access additional performance data analysis and utilization by offering only relevant and customized program report generation tools and templates that may not be avail ming material of their choosing targeted for their specific able or offered with the standard product. exercise, fitness, or rehabilitation goals. Performance data Custom Performance Algorithms---certain application could be saved indefinitely in the database's nonvolatile 55 specific performance analysis may require dynamically memory, until an upload process was performed through the linked algorithms that process and calculate non-standard or said network so the database could be transferred to another specialized information, values, units, physical measure location for purposes of, but not limited to, registration, pro ments, statistical results, predictive behaviors, filtering, cessing, archival, and normative performance evaluation, etc. numerical analysis including differentiation and integration, An exemplary list of specific data structures contributing to 60 convolution and correlation, linear algebraic matrices opera or affecting the means for automating the creation, delivery, tions to compute data pose and scaling transformation, and storage, and processing of movement trajectories described proprietary types. One example of a proprietary type is Moti below may be stored within the non-volatile memory of the vation Index, a composite numerical value derived from a transponder or position processor which may use high-den weighted average of statistical performance indicators and sity serial FLASH, although other types of memory may be 65 subjective user input including relative scoring improve used such as SmartMedia, Compact Flash, etc. Additionally, ments, conformity to ROM pattern, lengthy activity access the memory device interface should not be limited to internal, duration, high access rate, relative skill level improvement, A000052 Case: 12-1252 Document: 46 Page: 144 Filed: 08/29/2012 JX-003.0018 US 7,492,268 B2 7 8 daily goal achievement, etc., that could represent the overall environments, and especially when the tracking volume level of enthusiasm and satisfaction, the user has for a par likely contains potentially occluding objects, i.e., an unin ticular activity. volved limbs or clothing, that become potential sources of Range of Motion (ROM) Pattem Generator-the ROM competing, reflected paths. The preferred embodiment of the pattem requires some key control points to be captured along registration system utilizes the time of flight (TOF) measure the desired trajectory and stored in order that the algorithm ment of ultrasonic acoustic waves due to its immunity from can calculate an optimally smooth path, in real-time, during interference from the visible and near-visible electromag the comparative analysis phase. netic spectrum and its superior ability to overcome most ROM Pattem Capture & Replay-the ROM pattem can be multi-path reflections problems by simple gated timing of the can saved to memory in real-time by discrete position 10 initial wave front. Upon command from the processor unit, samples versus time depending upon the resolution desired the transponders produce a few cycles burst of ultrasonic and memory limitations and later played back on the tran energy and the transducers of the receiver constellation unit sponder or remote display for analysis. Activity Specific Attributes-includes Reps/Sets, Dura are stimulated and mechanically resonate accordingly, upon the wave front arrival. The processor unit's analog signal tion, Pause, Heart Rate Limits, intra-activity delay, level, 15 point scalars, energy expenditure, task-oriented triggers, etc., processing circuits transform the mechanical energy into and other parametric data that controls intensity, execution electrical signals that resemble tapered sinusoidal wave rate and scoring criteria for the activity. forms, which another electronic circuit triggers upon using an Instructional Information-textual, graphical, or anima adaptive threshold technique which, in turn, the processor unit detects and calculates TOF timestamps indicating the tion-based instruction, advice, coaching, activity description, 20 diagramed transponder deployment and intra-device connec wave front arrival. In the preferred embodiment, the system tivity, etc. that facilitates the intuitiveness, understanding, and overcomes the ultrasonic technology's intrinsic challenge of usage of the system. The form of instruction may include precisely triggering on same the waveform location and pro music files saved in various formats, including Wave, MP3 or vides consistent unambiguous trigger detection by comple- menting the adaptive threshold technique with a software other current or future audio data compression formats, and 25 video files saved in MPEG or other current or future video timestamp correction algorithm, which includes in part, a data compression formats. digital over-sampling and averaging timestamp algorithm, a Real-time Data Management-proprietary data manage relative timestamp correction scheme utilizing a predictive ment protocols that reside above the communication driver algorithm ofhigher-order Taylor series based derivatives, and layer that manage the real-time, synchronous and asynchro 30 an absolute timestamp correction scheme that minimizes the nous exchange of data between transponder(s) and position range error based upon discrete biasing of timestamps. processor. This would provide an essential real-time sharing Further, in the preferred embodiment, the processor unit of activity data, analysis, and feedback stimulus thresholds, utilizes the absolute and relative trigger timestamps in a or coordination of multiple transponder configurations, or for multi-modal trilateration algorithm for the measurement of a collaboration of same or different user requirements to 35 three-dimensional (3 D) translations and rotations of the tran complete a similar activity objective. sponders. The primary trilateration calculation is derived by This invention addresses the need for adaptability of the an application of Pythagoream theorem involving a point registration system to different movement measurement sce position solution based-upon range measurements from at narios. In one embodiment, it utilizes a versatile, modular least three (3) points, versus the well-known triangulation configuration and mounting of the transponders onto the user. 40 method which uses bearing angles of two cameras of known The efficient deployment ofthe transducers between different pose. Additionally, the system's main accuracy limitation is users' and from task to task requires a universal mounting mostly affected by the temperature variability of outdoor scheme to provide consistent localization and pose of the environments and its influence on the speed of sound in air transponders at the desired measurement sites on user's body. value. This algorithm mitigates this problem by mathemati Also, to compensate for the receivers' finite tracking volume 45 cally computing the speed of sound every analysis period when stationary, the receiver constellation unit may be provided at least five (5) receivers and a transponder synchro- mechanically modified to optimize its tracking properties by nizing means are utilized. Ifthe integrity ofthe synchronizing conveniently repositioning it in closer proximity to the signal is temporarily compromised, the system automatically expected transponders movement trajectories and line-of employs a variation of the trilateration algorithm that uses the sight, thereof. In summary, one embodiment of the present 50 last known speed of sound value. invention is comprised of: In the preferred embodiment, the maximum update rate, a means to quickly and efficiently alter the location of the and hence the major contributor to the latency of the position transponders using a fastening system designed to calculation, is determined by the typical acoustical reverbera quickly attach and dispose various forms of transponder tion, typically between 20 to 100 ms, encountered in an assemblies; 55 indoor environment. Since the transponders are held or fixed a means to augment the physical properties, i.e., weight on the user's body and, therefore, are mobile, the TOF mea and length, of the principle transponder with adjunct surements will experience an additional latency effect. A electro-mechanical components that provide variations Kalman filter is used as a prediction/estimation strategy to in biomechanical leverage for isotonic and isometric minimize and compensate for the latency effect. The predic- utilization; and, 60 tion algorithm uses a higher-order Taylor series based deriva a means to allow the user to manually alter the geometry tives and augmentative inertial sensor data. Its predictive and pose of the receiver constellation unit to facilitate an refinement is dependent upon predefined models of expected optimal tracking location based upon collectively maxi movement conditions. Because functional movement is epi mizing the ultrasonic source's energy received at the sodic, having periods of stillness interspersed with bursts of transducer interface. 65 motion activity, a multi-modal filtering strategy is preferably This invention addresses the practicality and robustness of employed to handle the unpredictable jerkiness at the start of the registration system when used in either indoor or outdoor motion and relatively predictable, smooth motion afterwards. A000053 Case: 12-1252 Document: 46 Page: 145 Filed: 08/29/2012 JX-003.0019 US 7,492,268 B2 9 10 In sunnnary, the preferred embodiment of the present inven FIGS. 2A-2D illustrate example extension pieces for the tion is comprised of: present invention; a means to detect the same carrier wave cycle of ultrasonic FIGS. 3A-3D illustrate one example of process flows for energy using a software correction algorithm requiring the present invention; multiple, consecutive TOF acquisitions as input for the FIGS. 4A and 4B illustrate a sample application of the digital over-sampling and averaging algorithm, the cal present invention; culation of a higher-order numerical differentiation of FIG. 5 illustrates a block diagram of the remote processing the past and current TOF infonnation as input for the system of the present invention; predictive algorithm of higher-order Taylor series based FIGS. 6A-6C illustrate example receiver configurations of derivatives used for the relative TOF correction, and a 10 the present invention; and measurement of the intra-pulse time intervals of con FIG. 7 illustrates a block diagram of the components of one secutive TOF acquisitions as input for the absolute TOF embodiment of the transponder of the present invention. correction scheme that minimizes the range error based upon selective biasing of the TOFs; DETAILED DESCRIPTION OF THE a means to utilize a dual matrix fonnulation of the trilat- 15 EMBODIMENTS eration algorithm, and a calculation strategy thereof, which decision is dependent upon the integrity of the The present invention provides a practical, versatile mea system's connnunication link, synchronization condi surement tool for the assessment of the user's manipulation tion, and the desired measurement accuracy; and, strategy of the transponder 10 or transponders along a refer a means to coordinate the information transfer between 20 ence movement trajectory. Moreover, the system and meth transponders and the processor unit so that their contri ods measure and analyze the kinematics of the relative trans bution to the resultant movement vector calculation can lations and rotations of the limbs or extremities with respect be measured without intra-signal interference. to each other or to a more inertial reference location on or off These goals will be attained by such system and methods body as the transponders are manipulated. This infonnation that are comprised of the user's interaction described by the 25 provides useful insight on biomechanical demands and following steps as set forth as the preferred embodiment: anthropometric factors that influence human movement effi a. Authenticate user access and open user session from a ciency and control. Although measurement perfonnance met local or remote database; rics are important design criteria, it's equally important to b. Setup user training session, i.e., workload limitations, provide intuitive and motivating program instruction and measurement criteria, and audio/visual/tactile stimuli; 30 administration, and to provide comprehensive analysis and integration of the motion data in a fonn that is objective and c. Select training program and configure its options; easily interpreted. This system improves upon the practicality d. Deploy the transponders as instructed to predefined loca and user interactive aspects of setup, deployment, calibration, tions of users locomotion system to create at least one execution, feedback, and data interpretation of a tracking transponder movement vector; 35 system designed for function human movement. e. Calibrate the transponder movement vector to establish Human movement is a response to external enviroumental its reference pose; forces which requires the accurate coordination of the distal f. Create a movement trajectory using learn mode, if segment( s) to compensate for these forces. Skillful coordina required; tion of human movement is dependent upon the cohesive g. Initiate the start of session; 40 interaction of multiple sensory systems, including visual, h. Detennine the instantaneous pose of transponder move vestibular, with the musculoskeletal system. More specifi ment vector relative to its reference pose from a periodic cally, the challenges and goals of cognitive spatial mapping, temporal iteration of this step; (2) minimization of energy expenditure, (3) maintaining sta i. Perfonn qualitative and quantitative statistical analysis of bility, (4) steering and acconnnodation strategies for various accumulated measured poses of the transponder move 45 environments, (5) dynamic equilibrium, (6) active propulsion ment vector relative to the pattern of instantaneous poses and weight support, and (7) core locomotion pattern should defined by the reference movement trajectory; be relationally considered to properly assess human move j. Update the major transponders sensory interfaces to ment. Therefore, it is preferable to engage the interaction of modulate said system parameters in a periodic temporal these sensory systems during a training session to promote iteration of this step; 50 the desired functional movement outcome. Because many k. End the session once program objectives have been movements persist for 400-500 ms, enough time is allowed obtained; for the initiation of the movement and for user correction 1. Analyze the results by interacting with local and/or based upon visual and kinesthetic infonnation acquired dur remote databases; ing the time of the movement. However, the implemented m. Provide numerical, graphical, and/or animated infonna 55 means of visual feedback should be not be distracting or tion indicating desired perfonnance measurements. interfering with the task at hand. In the preferred embodi ment, this system engages the sensory systems with non BRIEF DESCRIPTION OF THE DRAWINGS distracting, intuitive, embedded aural, visual, and tactile stimuli which provide real-time indication of the principle The disclosed embodiments will be better understood 60 transponder pose error with respect to the reference move when reference is made to the accompanying drawings, ment trajectory. wherein identical parts are identified with identical reference In order to conduct a time efficient training session, this numbers and wherein: registration system attempts to minimize the encumbering FIG. lA illustrates one example of a deployment apparatus experimental setup and calibration procedures characteristic of the present invention; 65 of more complex and higher cost motion analysis technology. FIG. IB illustrates one example of hand-held fonn for the These complementary systems serve important academic or transponder of the present invention; clinical oriented research needs or for motion capture for A000054 Case: 12-1252 Document: 46 Page: 146 Filed: 08/29/2012 JX-003.0020 US 7,492,268 B2 11 12 computer animation purposes and strive for highly accurate In one embodiment (FIG. 2B), the tactile type provides measurement of joint motion data in terms of angular dis force feedback functionality by controlling the rotational placement. Therefore, the integrity and reliability of their speed of an embedded vibrator motor in the shaft. Alterna motion data is dependent upon proper sensor setup and cali tively, the visual type (FIG. 2C) may be comprised of a series bration. of light emitting diodes that could be uniformly embedded For instance, single axis goniometer-based systems usually along the length of the handle or transponder and their inten require specially designed harnesses to hold the monitor and sities variably controlled therein. It should be appreciated that are firmly strapped or taped over the joint to avoid relative a simple, economical mirrored or reflective surface placed in motion artifacts. Usually these devices are tethered and their front of the user's visual field could provide sufficient real- fit, weight, and constraining mechanical linkages can impose 10 time indication of the user's subjective conformity to the said limitations on the joint motion and cause discomfort for the movement trajectory while allowing non-distracting viewing user. Most optical or video-based systems require the place of this visual stimulus. For example, a program that requires ment of numerous active or passive markers over landmarks, the user to reposition the principle interactive transponder such as the joints' center of rotation. These systems should through an arc-like movement trajectory in the midsagittal guarantee sufficient environmental illumination and contrast 15 plane through out a range of motion beginning from the waist between markers and background to function optimally. Also, upwards until parallel to shoulder height. As the user per these systems are severely affected by occluded markers that forms the movement, the visual sensory interface could be may disappear for long periods of time due to rotations and proportionally excited if the user moves too quickly, or hesi line-of-sight limitations. Other video-based systems do not tates too long, or produces shaky or erratic episodic motions, use markers but require the assignment of the body's joints 20 or is beyond the prescribed bounds of the movement arc. The manually or through computerized automation during data light stimulus is easily viewed in the mirror and would indi analysis, making real-time analysis arduous and real-time cate corrective action in his or her movement strategy, while feedback virtually impossible. appropriate aural commands may be issued simultaneously to In the preferred embodiment, the system doesn't require encourage the same correction. Regardless of the sensory complicated, time consuming sensor setup and calibration by 25 interface type, its control and excitation properties will be virtue of it minimalist sensor requirements and uncompli determined by some statistical aspect ofthe user's conformity cated sensor mounting. Instead, it requires only the deploy to and progression through the movement trajectory. ment of a sensor on the body (in one embodiment a dual The hand-held transponder may include a modular exten sensor group on a combination oflimb(s) and or trunk) and sion piece with an embedded graphic display device and doesn't enforce stringent movement protocol, but encourages 30 associated input means to allow the user to setup, operate, free-form, unrestrictive movement of the transponders. provide visual feedback, and view performance results of the The transponder's preferred deployment means, include device usage without additional remote display means. More either insertion into a universal strap and holster apparatus specifically, a software-controlled user interface could pro (FIG.IA) that secures on the user's limb, extremity, or trunk, vide certain visual prompts in a menu oriented presentation, including, but not limited to, the hip, ankle, knee, wrist, upper 35 to instruct the user on (1) device setup, i.e., aural, visual, and arm, neck, waist or an augmentative mechanical attachment tactile feedback parameters, types of program start and ter to one or a combination of modular extension pieces shaped mination cues, program intensity based on ratio of amount of into a hand-held form (FIG. IB). A strap ortorx-like clip and repetitions, sets, and rest periods or categorical gradation of holster design provides a firm, yet light weight and comfort challenge, learn mode behavior, etc., (2) scrollable program able mounting location away from areas that clothing and or 40 selection with brief descriptions including objective, desired uninvolved limbs may occlude. measurement, i.e., range of motion, energy, accuracy, speed, The modular extension piece is either an instrumented etc., and instructive information, and (3) alphannmeric and/or sensory type designed to support alternative tactile stimulus graphical display of measured performance data and other device or alternative configurations of aural, visual, and tac biophysical data and its analysis thereof, displayed in stan- tile feedback types, or non-instrumented, weighted extension 45 dard plotted forms including line, bar, and pie charts, etc. It is pieces as shown in FIGS. 2A-2D. All modular extension important to note that the user input process is intuitive and pieces may be of various physical dimensions and intrinsic streamlined so as not to detract from the practicality and user weight, with a captive handle design that preferably requires friendliness of the system. Only relevant applications and its zero grip strength to grasp. Alternatively, the modular exten control thereof will be sequestered from the database and sion piece may be coupled to the transponder through a fixed 50 presented to the user. or flexible, segmented, articulated coupling to accommodate In one embodiment, two or more transponders and exten attachment of additional transponders and/or other modular sion pieces, or combinations thereof, may be assembled at extension pieces. These components would quickly assemble their endpoints with a universal spring coupling. The to each other using a spherical snap joint or twist snap latch, assembled device could be grasped in both hands and bent in or similar mechanism, to provide quick alteration of form and 55 various rotational angles about the spring coupling's axis. function when used for different movement trajectory sce Isotonic strength conditioning programs can be developed narIos. due to the force resistance feedback supplied by the spring. A In one embodiment, the weighted extension attachments multi-transponder assembly in the form of a flexible rod or (FIG. 2A) are offered in fixed gradations of one (1) kilogram staff could provide an indication of balance of upper extrem- increments or other convenient unit of measure and either be 60 ity strength and proprioceptive function dependent upon the indicated as such with a numerical label, quantitative mark, or angular closure rate and rotational imbalance and orientation color-code feature, or combination thereof. For upper extrem deviation from initial starting position. ity evaluation, the weighted extension piece integrated into a Additionally, in the preferred embodiment, the modular zero-grip handle would enhance the improvement of muscu extension pieces have provisions for other attachable appara- lature strength ofthe limb, while not compromising the user's 65 tus (FIG. 2D) that can augment the program's intensity or endurance with a potentially fatiguing hand grasp require difficulty. For example, an eyelet is embedded in the end of ment. the extension piece and is designed to attach an elastomeric A000055 Case: 12-1252 Document: 46 Page: 147 Filed: 08/29/2012 JX-003.0021 US 7,492,268 B2 13 14 band, such as the type manufactured by Theraband®. By During the Deployment Phase (FIG. 3B), and dependent securing the other open stirrup end of the band to the user's upon the program's objectives, a suitable combination of foot, isotonic strength conditioning programs can be devel transponder types will be mounted on the user's body as oped due to the force resistance feedback supplied by the instructed by the program. This example requires the assem elastomeric band. Moving the transponder through a move bly of a hand-held interactive transponder with graphical ment trajectory is now made more restrictive and challenging. display, and a weighted extension piece coupled therein to be grasped by the hand on the same side as the affected shoulder. APPLICATION EXAMPLES Another subordinate transponder 12 is placed into a holster assembly strapped around the lower quadriceps on the same An example training session deploying a dual transponder 10 side. This setup is shown in FIG. 4. group is now described that may be designed to improve the During the Calibration Phase (FIG. 3C), a simple calibra range of motion, strength, and coordination of shoulder tion procedure may be requested to evaluate transponder abduction in a user. The training session would primarily function and specific user range of motion constraints. Typi serve as an exercise aid that provides essential feedback to the cally, this information is determined beforehand and saved in user so that he/she learns to progressively improve the 15 the system's database. Also, practicality of this system is manipulation of the transponder through the reference move claimed for lack of extensive calibration requirements. ment trajectory, while benefiting from increased shoulder Dependent upon the program's options, a user-defined range of motion and strength improvement. movement trajectory may be created prior to program start in lieu of executing the predefined version. The learn mode In advance of the training session, a software application is 20 could be utilized to quickly choreograph free-form move operated from a host computer that provides a utility for ment trajectories and save them into the transponder's non baseline configuration and management of the system's and volatile memory for later execution. The learn mode would be transponder's local databases, and/or access to other remote accessed through the user interface and instruct the manage databases, and for the real-time interface to the data flow ment of the control point assignment by pressing the push between the system's components. The application's naviga 25 button switch at the appropriate junctures of movement dis tion and selections are presented to the user through a typical continuity or, preferably, allowing automated assignment by graphical user interface like Microsoft Windows® XP oper the software. In the preferred embodiment, a computationally ating system. A generalized step-wise procedure requires the efficient Catmull-Rom spline is used to define a three dimen administrator or user to (1) select the desired program and sional (3D) curve that passes through all the control points features from a menu screen list, and (2) to initiate a commu 30 along the movement trajectory path. Ifmanually interceding, nication process that causes the program parameters to be the user is instructed to press the push button once at each transferred to the processor unit through a standard computer major juncture in the movement trajectory, but, preferably, for communication protocol, i.e. serial, USB, ethernet, etc., no more than a few locations, until the desired end of range of whereupon, (3) the information is subsequently processed motion is reached as shown in FIG. 4B. Similarly, the return and transferred into the transponders local memory via a 35 path may be similarly defined or he/she may elect to use the wireless communication link, and, finally, (4) the transpon same forward path in reverse. These control points are regis der's software program accesses this database to manage the tered by the processor unit and transferred and saved to the device utilization and configuration of the local display transponders' memory to serve as the control points for the means. Alternatively, a Compact FLASH-based memory real-time calculation of a Catmull-Rom spline. The Catmull- card, embedded serial FLASH, or a similar nonvolatile 40 Rom spline is calculated in real-time from the desired initial memory device provides the user an additional specialized starting point to provide a continuous set of position points database supporting remote data collection capabilities. This representing the "learned" reference movement trajectory. database would be preprogrammed in advance and the result After the program is selected or the learn mode complete, ant performance data retained, even if the device's power is the user may be instructed to alter the pose ofthe transponders lost, or for extended unsupervised exercise sessions con 45 to satisfY the initial starting conditions of the program. Either ducted remotely from the host computer system or when the one or a combination of sensory interfaces could be excited host computer system is unattached or unavailable. After the by the principle transponder to cause the user to direct or steer session is completed, the user would be queried if the results it towards the desired start point. For instance, the visual are to be saved for later analysis or would automatically be sensory interface could sequentially extinguish or dim its saved, dependent upon device setup. This data could be 50 peripheral light sources to converge to a central light source as retrieved at a later time when the system is once again the principle transponder is positioned closer to the desired attached to a host computer system, and the software utility start point. Alternatively, the aural sensory interface could could be commanded to upload the database. change its tonality and loudness as the start point is Henceforth, the following procedural description refers to approached. Or alternatively, the tactile sensory interface the activity dependencies diagrammed in FIGS. 3A-3D that 55 could be modulated to provide less force feedback as the start the user would encounter while operating the system. point is approached. During the Security Phase (FIG. 3A), the user may be During the Execution Phase (FIG. 3D), the transponders requested to provide a security authentication code for vali are continually manipulated along the reference movement dation, which opens access to hislhers custom programs in the trajectory to the best of the user's skill and fidelity, within the training session. Next, during the Setup Phase (FIG. 3A), the 60 bounds of the user's physical limitation, until an aural, visual, user can configure global options or select the desired pro or tactile response is given that indicates the activity volume gram. The global options may include, but are not limited to, has been successfully completed or a sufficient number of workload intensity, measurement criteria, sensory interface conformity violations or failures have been registered. The properties, and reporting features. A program menu list would processor unit calculates the instantaneous pose coordinates indicate name, ID, and a brief description, or alternatively, be 65 of the transponders every analysis interval and periodically represented by a detailed graphical icon. When the program is communicates this information to the transponders via the selected, other program-specific options can be setup. wireless communication link. As the principle transponder is A000056 Case: 12-1252 Document: 46 Page: 148 Filed: 08/29/2012 JX-003.0022 US 7,492,268 B2 15 16 moved in mimicry to the reference movement trajectory the One testing and training scenario for postural stability conformity error between the actual and reference movement would be to measure frequency and amplitude of body sway trajectory is calculated periodically in real-time to determine in three dimension (3D) space while feet remain in a fixed the characteristics of feedback quality to be elicited by the position. This task can be performed in both a double or single sensory interfaces for the user's closed -loop control to correct leg stance to test for bilateral symmetry relating to balance. hislher manipulation strategy. For example, the conformity Another modification of the test would be to perform each test error may be calculated from statistical processes based upon with eyes both open and closed to help determine the contri the standard deviation of the least mean squared (LSM) prin bution of the visual component to overall balance ability. ciple transponder's position error compared to the reference Tracking body sway while creating the illusion of motion movement trajectory, or based upon, or combination thereof, 10 through proper visual cueing on a display means would be a threshold magnitude of some multi-order numerical differ another test to help determine the reliance on specific sensory entiation of said movement to indicate a "smoothness" qual- components of balance. Delivering repetition of protocols ity of translation and rotation along the movement trajectory with increasing difficult oscillation thresholds with biofeed path. back of successes and failures of such predetermined goals is Alternatively, a host computer system could provide an 15 one way to train to improve balance. auxiliary processing and display means to allow another soft The transponder can deliver aural, visual, and tactile ware program to access the transponder's calculated posi stimuli to queue the individual to the degree of frequency and tional data through an application programmer's interface amplitude of body oscillations. The aural and tactile compo and use this data to alter the pose of a graphical primitive in nents provide the only means of feedback when the testing proportion to the motions of the transponders within the con 20 and training are performed with eyes closed or the visual field text of computer generated virtual enviroument. The dynamic is compromised. Examples include, but are not limited to, (1) control of objects in the computer generated virtual environ an audio signal increasing and decreasing in volume propor ment could be used to augment the local sensory interfaces of tional to the amplitude of body sway, (2) a vibration action the transponders through an interactive, goal-oriented video proportional to frequency of body oscillations, and (3) a light game modality. The video game challenges could be 25 source illuminated when both frequency and amplitude goals increased over time based upon some scoring criteria of suc are achieved. Multiple transponders can be used to evaluate cessful manipulation of the principally controlled on-screen and reinforce proper balance posture by communicating posi graphical object with respect to cueing derived from other tion information of certain body segments in relationship to secondary static or dynamically moving objects. It is impor others. An example would be the comparison of position of tant to note that only primitive forms of video game chal 30 the head with respect to the hips while generating a vibration lenges would be considered, to take into account the user's action if an excessive forward lean of the head as compared to cognitive awareness and physical limitations, and the eco the hips is recognized. nomics of software development for photo realistic virtual Another test for balance would be to test ones Limits of environments and animation. Also, this auxiliary computer Stability (LOS). This test refers to ones ability to effectively display means would offer an alternative visualization 35 operate within their sway envelope. The sway envelope or method of interactive and immersive video feedback aid to LOS is defined as the maximal angle a person's body can enhance the application presentation. achieve from vertical without losing balance. An individual Additional examples of how the present invention may be with healthy balance is capable ofleaning (swaying) within a applied are described as follows: known sway envelope and recover back to a centered position 40 without the need for a secondary maneuver such as a step, Balance excessive bend at the torso or arm swinging. LOS for bilateral The body has the ability to maintain balance under both stance in normal adults is 8 degrees anterior, 4 degrees pos static and dynamic conditions. In static conditions, such as in terior and 8 degrees laterally in both directions. standing, the body strives to efficiently maintain posture (of The present invention described can be used as a testing ten referred to as postural stability) with minimal movement. 45 and training device for balance control during movement or In dynamic conditions such as in walking or sports play, the perturbations within a desired sway envelope. Through body strives to maintain balance while performing in an ever proper visual queuing represented on the display means that changing environment. The ability to maintain balance is a defines a normal sway envelope, the amount ofbody displace- complex process that depends on three major sensory com ment can be measured from vertical stance. ponents. The sensory systems include visual, vestibular and 50 The transponder can deliver aural, visual, and tactile proprioception. For example, we rely on our visual system stimuli to queue the individual as to when he or she has (eyes) to tell us ifthe environment around us is moving or still; achieved the desired range oftheir sway envelope, then assess we rely on our vestibular system (iuner ears) to tell us ifwe are the individual's ability to return back to a vertical stance. upright or leaning, standing still or moving; and we rely on Examples include, but are not limited to, (1) a vibration action our proprioceptive system (feet and joints) to tell us if the 55 when the user varies (meanders) from the desired movement surface we are standing on is uneven or moving. If balance path, (2) an array oflights change intensity and pattern as the problems develop, they can cause profound disruptions in individual successfully approaches the intended target, (3) an your daily life. In addition to increased risk for falls, balance audio signal is generated when the individual has maintained disorders can shorten attention span, disrupt normal sleep a stable position with respect to proper visual queuing repre patterns, cause excessive fatigue, increase dependence on 60 sented on the display means for a selected period of time. others and reduce quality of life. It is not uncommon for Multiple transponders can be used to evaluate and reinforce individuals with a history of balance problems to regain their proper balance posture by communicating position informa balance control through accurate diagnosis followed by spe tion of certain body segments in relationship to others. An cific medical treatment and/or rehabilitation exercises. example would be the comparison of position ofthe head with The present invention described can be used as a testing 65 respect to the hips while generating a vibration action if an and training device for balance improvement under both excessive forward lean of the head as compared to the hips is static and dynamic conditions. recognized. A000057 Case: 12-1252 Document: 46 Page: 149 Filed: 08/29/2012 JX-003.0023 US 7,492,268 B2 17 18 Dynamic balance can be evaluated while having the indi vidual's movement trajectory varied from the intended two vidual perform coordinated movements which specifically dimensional (2D) reference movement trajectory by devia challenge the various components of balance in a dynamic tion from the planar path into the uninvolved spatial dimen nature. Such movements include, but are not limited to jump sion. An array oflight sources could increase illumination in ing, hopping, and walking. These movements can be per intensity and repetition as the ROM goal was approached and formed with eyes both open and closed, during interaction an audio tone could signal the individual they have achieved with static or dynamic visual queuing on the display means. the desired pause time at the proper ROM. The ability to perform these dynamic balance tasks with Multiple transponders can be deployed to determine the comparisons to others of similar sex, age or disability can be contribution of each joint or anatomical structure where more assessed. Example measurements may include, but are not 10 limited to, (1) amount of body sway in three dimension (3D) then one joint is involved in the ROM movement (example; space, (2) time to complete specific task, and (3) effects of shoulder and scapular in overhead reaching). The vector sum fatigue on balance ability. of each transponder movement in a specific axis can be added Balance training in both static and dynamic conditions can together to determine the total ROM. The ROM of one j oint in be easily achieved by providing specific visual queuing on the 15 a two joint motion can be subtracted from the total ROM to display means, which challenge the individual to perform determine the contribution of a single joint in a two joint repetitive and progressively more difficult balance drills. Per movement. formance reports can be generated to establish a baseline, isolate specific strengths and weaknesses within the specific Human Performance Testing and Training sensory and motor control aspects of balance, and document 20 There are many devices that test the strength and speed of progression and improvements. isolated joint movements, for example, the leg extension and The transponder can deliver aural, visual, and tactile bicep curl. This information has value in testing both healthy stimuli to queue the individual as to when he or she has individuals, athletes and individuals whose strength and achieved the desired balance task. By example, a vibration speed capabilities may be compromised by injury, disease, action is produced proportional to the frequency of a body 25 poor conditioning or simply age. Recently in the field of segment oscillation after the user lands from a hop test and human performance, it has been recognized that the analysis attempts to stabilize and maintain proper postural balance. of the mobility of the isolated joint, although providing some When the individual finally stabilizes and achieves correct value, does not offer enough information to determine how postural balance, an audio signal indicates the task has suc the body will perform during functional movements. Func cessfully completed. Multiple transponders can be used to 30 tional movements are defined as movements equal to those evaluate and reinforce proper balance posture by communi encountered on the athletic field, in the work environment or cating position information of certain body segments in rela while performing activities of daily living. Functional move tionship to others. An example would be the comparison of ments involve the movement and coordination of multiple position of the head with respect to the hips while generating joints and muscle groups acting together to perform a more a vibration action if an excessive forward lean of the head as 35 complex task then a single, isolated joint movement. compared to the hips is recognized. The present invention described can be used as a testing Range of Motion (ROM) and training device for functional movement improvement. The present invention described can be used as a testing By tracking various registration points on the body with and training device to determine the range of motion within a 40 respect to each other or to an off-body registration point, joint. Range of Motion is the normal distance and direction performance measurements of functional movements can be through which a joint can move. Limited ROM is a relative assessed, such as jumping, cutting, turning, bounding, hop term indicating that a specific joint or body part cannot move ping, shuttling, etc. through its normal and full ROM. Motion may be limited by The present invention described can be used as a testing a mechanical problem within the joint that prevents it from 45 and training device for individuals involved in physical reha moving beyond a certain point, by swelling of tissue around bilitation, general fitness or sports performance enhancement the joint, by spasticity of the muscles, or by pain. Diseases to improve their functional movement abilities. Proper visual that prevent a joint from fully extending, over time may queuing can be represented on the display means to instruct produce contracture deformities, causing permanent inability and motivate individuals to perform specific functional move- to extend the joint beyond a certain fixed position. 50 ments. The present invention described can be used to test the The transponder can deliver aural, visual, and tactile feed starting point and end point which an individual is capable of back of proper movement execution. Examples include, but moving a body part, typically a limb and associatedjoint(s). are not limited to, (1) an audio signal alerting the user that the Comparisons to age and sex based normative data can be desired performance stance is incorrect, (2) the light sources made. Proper visual queuing can be represented on the dis- 55 illuminate when the desired speed is achieved in a first step play means to instruct and motivate the individual through the quickness drill, (3) a vibration action to indicate the limits of proper testing procedure. tracking range, (4) a vibration action proportional to the mag The present invention described can be used as a testing nitude of a biophysical measurement during the interaction and training device for individuals involved in physical reha with visual queues represented on the display means, (5) a bilitation or general fitness to improve ROM. Proper visual 60 vibration action when the body or limb position does not queuing can be represented on a display means to motivate correlate well to the desired body or limb position of the individuals to extend their range of motion beyond their cur visual queuing represented on the display means, (6) an audio rent capabilities. signal indicating start, stop and pause periods or other con The transponder can deliver aural, visual, and tactile feed trolling events, (7) an audio signal indicating proper body back that alerts the individual to successes or failures in 65 alignment or posture has been compromised, and (8) an audio proper execution of each repetition. An example of tactile signal indicating the relationship of desired target heart rate to feedback would be the transponders are vibrated if the indi- a desired threshold. A000058 Case: 12-1252 Document: 46 Page: 150 Filed: 08/29/2012 JX-003.0024 US 7,492,268 B2 19 20 Hardware Description trilateration requires the precise resolution of the common In the preferred embodiment, the processor unit is com intersection of multiple spheres circumscribed by the dis prised principally of a constellation of five (5) ultrasonic tance between each transmitter and receiver transducer. Each transducers and signal processing circuitry, thereof, and a sphere has an inexact radius due to system noise and mea signal processor that interfaces to this receiver group, per surement resolution. Therefore, the intersection becomes a forms the pose calculations, and interfaces to the transpon volume instead of a point and the size of the volume is ders and host computer databases. The following interface dependent upon the radii ofthe intersecting spheres as well as descriptions for the processor unit are based upon the depen the distance between the spheres' centers. As the radii get dency flow represented by FIG. 5. larger with respect to the distance between the centers, i.e., The sensors 14 preferably used for the receiver constella- 10 the transmitter is farther down range, the spheres begin to tion unit are cylindrically-shaped ultrasonic transducers, for appear more and more tangential to one another and the example, the model US40KR-0l 40 kHz PVDF ultrasonic intersection volume increases, although not necessarily sym receivers manufactured by Measurement Specialties Inc., metrical in all dimensions. Therefore, to minimize position which provide adequate acoustic pressure sensitivity and uncertainty, the receiver transducers should be separated exhibit 360 degree onmidirectional broad beam response 15 from each other as much as practical proportions allow with along the horizontal plane. The onmidirectional characteris respect to the confines of the tracking field volume as the tic, albeit in one plane only, is very desirable to minimize above said geometric examples provide. line-of-sight occlusion. Because of its low resonance Q value, This receiver constellation unit can be repositioned with the rising and decay times are much faster than conventional respect to the tracking field by a simple mechanical adjust ceramic transmitters. This reduces its power requirements 20 ment as shown in the preceding figures. The mechanical since less burst drive duration is needed to achieve sufficient adjustment raises and lowers and changes the length and pivot triggering thresholds at the receiver. This transducer type is axis of the cantilever arm which is fixed to a ground base also utilized similarly in the transponders to provide the support. potential for the most optimal acoustic coupling. Because the receiver constellation unit operates a distance The receiver constellation unit is preferably mounted on a 25 from the processor unit, each receiver preferably has an asso fixed support base, and has a pivoting and/or swiveling ciated pre-amplifier circuit to convert the high-input imped mechanical linkage which provides an adjustable mechanism ance piezoelectric signal into a low-level voltage proportional for configuration of the receiver constellation unit's inertial to the acoustic signal energy impinging the transducers suf frame of reference relative to the tracking field. In the pre ficient in order to accurately transmit the signals to the pro- ferred embodiment, it is strategically positioned and oriented 30 cessor unit. In one embodiment, a high-input impedance AC in proximity to the tracking field in order (1) to minimize amplifier design with 30 dB gain can be utilized. The pre- line-of-sight degradation with respect to the expected tran ferred operational amplifier is the OPA373 manufactured by sponder orientation, (2) to optimize registration resolution Texas Instruments. It was chosen for its low 1 pA input bias with respect to field volume size, and (3) to satisfy the math current, high 6 MHz GBW, and low-voltage single supply ematical restrictions of performing trilateration calculations 35 operation. The amplifier is configured as a non-inverting type based upon the solution of simultaneous linear equations. It with the high-pass cutoff frequency set at 1 kHz. The overall should be noted that the trilateration matrices may be solved circuitry is preferably enclosed in a metal shield to minimize if the matrices have a rank of five, and are non-singular, i.e., electromagnetic noise coupling into the highly sensitive the matrix determinant is non-zero. In the preferred embodi amplifier inputs. In addition, a local, regulated power supply ment, the geometric parameters and their coordinate location 40 is included to allow for a wide range of input voltage supply of the receiver constellation must insure linear independence and provide sufficient power supply rejection to compensate of the colunms of the matrices and to avoid the matrices from for the noise susceptibility of remote power distribution. All becoming singular. the pre-amplifier circuits' power and signal connections pref One example geometrical permutation of the receiver con erably originate from the processor unit. stellation unit that satisfies these rules is shown in FIG. 6A. It 45 The processor unit subsystem preferably consists of an occupies a volume of approximately 8 cu. ft. and essentially analog signal processing interface that provides (1) additional fixes the transducers in a way that defines two primary voltage amplification and filtering of base band signal from orthogonal, bisecting planes defined by three non-collinear the preamplifiers, (2) absolute value function, (3) peak detec points each. Another preferred implementation that occupies tion function, and (4) analog-to-digital comparator function nearly the same volume is shown in FIG. 6B and is charac 50 to provide support for an adaptive threshold means. The adap terized by its S-shaped curve and tilted with respect to the tive threshold technique provides robust triggering of the horizontal plane. Another preferred implementation that most proximal ultrasonic source at a precise temporal point occupies nearly the same volume is shown in FIG. 6C and is along the traversing sinusoidal waveform of the electrical characterized by its helical or logarithmic spiral shape ori signal. Essentially, a new threshold signal is recalculated each ented perpendicular to the horizontal plane. Further, as indi 55 analysis period based upon a small percentage reduction of cated in the preceding figures, the transducers vertical axes the last peak waveform detected. Therefore, the tracking are oriented 90° with respect to the typical vertical axis ori range is not necessarily restricted due to an arbitrarily high entation of the transponder's transmitter to improve acoustic threshold setting and the noise immunity is improved as the coupling in the vertical plane, a consideration for overhead, threshold tracks the waveform envelope and not transient upper extremity tracking. Although this causes some reduc 60 disturbances. An alternative automatic gain control strategy tion in the lateral registration bounds, the compromise pro for the amplification function is unnecessary since the trigger vides a more symmetric field about the middle or primary threshold will adjust to the signal level instead. In the pre location of tracking interest. ferred embodiment, the threshold faithfully tracks the peak to In the preferred embodiment, the overall size of the minimize integer period phase errors, so the amplifier's gain receiver constellation unit is predicated on a phenomenon 65 is set to prevent signal saturation from occurring when the referred to as Geometric Dilution of Precision (GDOP). The receiver constellation unit and transponders are in closest solution of a unique three-dimensional location based upon proximity during normal use. A000059 Case: 12-1252 Document: 46 Page: 151 Filed: 08/29/2012 JX-003.0025 US 7,492,268 B2 21 22 In one embodiment, an amplifier and BW (band width) In the preferred embodiment, a comparator circuit receives filter circuit receives the output from the sensor and pream the output from the peak detect and sample-hold circuit to plifier circuit and provides additional amplification and low convert the analog signal to digital form for high-speed trig pass filtering to condition it for reliable threshold triggering gering operation of the processor. The preferred device is the and input to other analog signal processing circuitry. A dual MAX941 which is manufactured by Maxim. A percentage of amplifier configuration may be used to provide an additional the peak threshold is used to set the inverting input. When the gain of 40 dB, AC coupling to remove DC offsets of the non-inverting voltage exceeds the inverting voltage, the com preamplifier outputs and long cable losses, and low-pass filter parator's output will trip and produce a high-true logic pulse to reject noise beyond the interest signal's bandwidth. The that triggers the processor. A latch control input allows the 10 processor to disable the comparator action to prevent unnec first stage amplifier may be configured as a non-inverting type essary triggering during the reverberation phase and to pre with a gain of 20 dB. The low-impedance DC input signal is vent potentially disruptive noisy output chattering near effectively blocked by the coupling capacitor in series at its threshold crossover beyond its hysteretic immunity. The per non-inverting input with a high-pass frequency cutoff set at centage of threshold level is predetermined through the scal- 20 kHz. This gain stage feeds a second amplifier configured as nd 15 ing resistors to be set low enough to trigger on the rising edge low-pass, 2 order Butterworth MFB filter. This filter type of the signal's first crest at the furthest range of transponder provides smooth pass band response and reduced sensitivity operation, but high enough above the intrinsic system noise to component tolerances. The second stage low-pass fre level and external noise caused by reverberation and other quency cutoff is set at 80 kHz with a pass band gain of20 dB. ultrasonic sources. Once the first crest is registered, subse- An absolute value circuit receives the output of the ampli 20 quent crests may be triggered at their zero-crossing represent fier and BW filter circuit and converts the bipolar signal into ing the most precise timing registration by momentarily dis a unipolar form for magnitude detection. A dual amplifier abling the sample-hold circuit. Because ofthe longer duration configuration may be used to provide highly accurate full trigger receptivity window, early multiple reflections are wave rectification of the millivolt-level signal. The first stage mitigated by transducer placement at least 3.5 cm away from amplifier feedback switches to control the distribution of 25 adjacent planar surfaces, so the reflected acoustic energy input current between the two signal paths dependent upon doesn't produce a canceling effect of the direct acoustic the input signal polarity. For a positive input voltage the input energy of the later crests. Once a sufficient number of crests current will be positive which forward biases Dl and reverse have been registered, then the triggering window is blanked st biases D2. This configures the 1 stage as an inverter driving for the remainder of the analysis period by latching the com- nd the inverting input resistor of the 2 stage, which is also 30 parator's value. configured as an inverter because its non-inverting input is In the preferred embodiment, a digital signal processing held at virtual ground due to the non-conducting path ofD2. interface is connected to the analog signal processing inter This effectively creates a combined circuit of two cascaded face to transform the analog trigger processing into digital inverters for an overall gain of + 1. For a negative input signal position information. its input current is negative which forward biases D2 and 35 The digital filter circuit receives output from the compara- st reverse biases Dl. This configures the 1 stage as an inverter tor circuit and preferably consists of a digital low-pass filter nd driving the non-inverting input ofthe 2 stage which changes implemented in a complex programmable logic device the sign of the circuit gain. In this mode, the input current is (CPLD) that serves to precondition the comparator circuit's nd shared between two paths to the input of the 2 stage, where digital outputs. The preferred device is an AT1504ASVL _2/3 of the input current flows around the 1st feedback stage 40 CPLD which is manufactured by Atmel. Base band system nd and -1f3 flows in the opposite path around the 2 stage feed noise or other glitches potentially occurring in the analog back path for a net gain of -1. signal processor interface, but prior to the actually arrival of In the preferred embodiment, a peak detect and sample the ultrasonic signal, could cause a threshold disruption that hold circuit receives the output of the absolute value circuit registers a "runt" pulse as a false trigger condition. The "runt" and registers a peak value that is required to set a magnitude 45 pulse would be misinterpreted as the actual TOF trigger and threshold precisely at some percentage of full-scale of the cause serious error in the position calculation. AnANDINOR peak. A dual amplifier configuration may be used to provide one-hot state machine design may be used to ignore level the highest ratio of high output slew rate to low droop. The transitions that are not stable for at least Ih system clock first stage is typically in negative saturation until the input frequencyx8 states, so only transitions of 4 flS or greater are voltage rises and exceeds the peak previously stored on the 50 passed through. The system clock delays introduced by the sample capacitor at the inverting input. Now the amplifier acts digital filter's synchronous state machine affect all channels as a unity gain buffer and the input voltage charges the sample the same and are, therefore, effectively eliminated by the capacitor which faithfully tracks the rising voltage. Once the inherent dependency on relative measurement. input voltage diminishes in magnitude, the first blocking In the preferred embodiment, the processor and digital diode reverse biases and the sample capacitor holds an accu 55 filter circuits receive the output from the analog processor and rate replica of the highest voltage attained with minimal provide controlling signals therein. The preferred processor droop because of the low input bias current of the amplifier circuit is the MC9S08GB60 which is manufactured by and elimination ofleakage altogether in the second blocking Motorola Inc. It is a low-cost, high-performance 8-bit micro diode by bootstrapping its cathode at the same potential pro controller device that provides all the aforementioned hard- vided by the low-impedance buffer of the second output 60 ware circuits integrated into one convenient device. The cal stage. An electronic switch and bleed resistor allow the volt culation circuit is abstracted from embedded 60 KB FLASH age across the sample capacitor to be reset by the processor for program memory with in -circuit programmable capability during power up and after the triggering event is recorded so and 4 KB RAM for data memory. The time base circuit is the adaptive threshold value can be refreshed each cycle. A 1st preferably comprised of an external, high-noise immunity, order Butterworth filter may be used at the input to smooth 65 4.0 MHz system clock, which multiplies this by the internal false in-band transients that could disrupt the peak accuracy frequency-locked loop for a bus clock of 40.0 MHz and single detection. instruction execution time of251lS. This clock also provides A000060 Case: 12-1252 Document: 46 Page: 152 Filed: 08/29/2012 JX-003.0026 US 7,492,268 B2 23 24 all the capture and control timing functionality for the other power output, etc. or get infonnation about parameters such specified circuits. Multiple parallel I/O ports and dedicated as battery, PLL lock state, etc. In nonnal mode, any data asynchronous serial communication signals provide for the entering its input channel is immediately radiated or any digital control of the analog signal processing and communi desired signal collected by the aerial is demodulated and cation interfaces, respectively. transferred to the microprocessor as reshaped register bit The timing capture-control circuit receives the output from information. In wake-up mode, the device periodically scans the digital filter circuit representing the arrival of the TOF for an expected message sequence and broadcasts an interrupt triggers to detennine the relative TOF propagation of the if a correct message is detected. ultrasonic acoustic wave as it passes through the receiver In the preferred embodiment, at least three (3) consecutive constellation unit. More specifically, it is comprised of a five 10 TOF timestamps are registered for each receiver during the channel l6-bit timer input capture module with program acquisition phase. Preferably, the transponder's transducer mable interrupt control that provides edge detection and 50 11 s emits a multi-cycle ultrasonic acoustic burst of at least ten timing precision to automatically register the TOF triggers cycles in duration so that sufficient energization of the timestamps asynchronously without using inefficient and less receiver transducer is realized and at least three crests of the accurate software polling means. 15 waveform can be properly registered. At low signal levels The phase-locked loop circuit receives the output from the when ultrasonic acoustic coupling is poor, this requirement timing capture-control circuit and is preferably comprised of may fail and an invalid tracking status is asserted. Preferably, a three channel, l6-bit timer compare module is implemented the reference receiver transducer of the receiver constellation as an all-digital phase locked loop (ADPLL), which synchro unit is positioned in closest proximity to the acoustic signal nizes the capture window and blanking functions with respect 20 source so that it is the first transducer to be affected by the to the reference input channel. It is comprised primarily of a initial wave front. This reference receiver provides the overall free-running l6-bit timer configured to periodically interrupt system timing and state machine control for the phase-locked the processor dependent upon a precise convergence of its loop circuit, so that the processing, calculation, and commu period and phase to the reference trigger source, by means of nication tasks are executed in a deterministic and efficient an over/under count matching and correction technique. 25 fashion. The AID conversion circuit receives the output from the It should be appreciated that a high-resolution ultrasonic amplifier and BW filter circuit and consists of an eight chan acoustic tracking system that depends upon threshold detec nell O-bit analog-to-digital converter used to monitor channel tion means has an inherent uncertain trigger dilemma. This offsets and magnitudes for range and polarity errors and uncertainty arises because of the multi-cycle nature of the correction. This infonnation is utilized by the calculation 30 transmitted signal's wavefonn and the associated difficulty circuit as input to the TOF software correction algorithm to detecting the exact temporal location for consecutive analysis determine the slope of the wavefonn crest. periods when the signal's magnitude may vary greatly In the preferred embodiment, the serial communication depending upon the efficiency of the acoustic coupling, the circuit is comprised of two asynchronous serial communica distance between transmitter and receiver, and signal-to tion interfaces that are connected between the calculation 35 noise ratio of the signal processing techniques. If a threshold circuit and host link and radio link circuits of the communi is set near one of the minor crests of the wavefonn during the cation interface. The host link provides a l15K bit per second last analysis period, then it is conceivable a slight reduction of (baud) bi -directional communication link to an auxiliary host magnitude of the wavefonn during the next analysis period computer system through a Serial-to-Universal Serial Bus may fall slightly below the set threshold and actually not be bridge. The preferred device is the CP210l which is manu 40 triggered until the next larger excursion of the wavefonn factured by Silicon Laboratories. It supports the conversion of occurs. This would create a TOF error proportional to the a fully asynchronous serial data bus protocol, with buffering period of the acoustic wavefonn or its intra-pulse interval and and handshaking support, to an integrated Universal Serial have a detrimental affect on the measurement accuracy. This Bus (USB) Function Controller and Transceiver and internal analog processing described above establishes trigger thresh- clock providing USB 2.0 full-speed compliancy. An inte 45 olds that allow no more than a single intra-pulse interval of grated 512 bit EEPROM stores the required USB device uncertainty, but that is still inadequate for high-resolution descriptors, including the Vendor ID, Product ID, Serial measurements. Although a technique is known that controls Number, Power Descriptor, Release number and Product the largest peak profile of the transmitter acoustic signal and Description strings. A host computer may enumerate and claims to provide an absolute trigger condition, this proce- access this device utilizing the manufacturer's virtual COM 50 dure is difficult to reliably tune and control among different port device drivers using a USB channel. transducer types. In the preferred embodiment, the radio link circuit is com In the preferred embodiment of the invention, no modula prised of a wireless bi-directional communication interface to tion of the acoustic signal is required. Rather, the adaptive preferably (1) broadcast a synchronization signal to control threshold method is augmented with a TOF software correc- the transponders interoperability, (2) to receive other tran 55 tion algorithm that unambiguously determines the correct sponder sensor data, including, but not limited to, accelerom TOF based upon a means to detect the same carrier wave eter, heart rate, battery, user I/O status, (3) to provide control cycle of ultrasonic energy every period. The software correc messages for the transponders' sensory interfaces, and (4) to tion algorithm requires multiple, consecutive TOF acquisi provide means to configure transponders' local databases. tions as input for the digital over-sampling and averaging The preferred wireless communication link is based upon the 60 algorithm, the calculation of a higher-order numerical differ AT86RF2ll, a highly integrated, low-power FSK transceiver entiation of the past and current TOF information as input for optimized for license-free ISM band operations from 400 the predictive algorithm of higher-order Taylor series based MHz to 950 MHz. and manufactured by Atmel. It supports derivatives used for the relative TOF correction, and a mea data rates up to 64 kbps with data clock recovery and no surement of the intra-pulse time intervals of consecutive TOF Manchester Encoding required. The device has a three wire 65 acquisitions as input for the absolute TOF correction scheme microprocessor interface that allows access of read/write reg that minimizes the range error based upon selective biasing of isters to setup the frequency selection, transmission mode, the TOFs. A000061 Case: 12-1252 Document: 46 Page: 153 Filed: 08/29/2012 JX-003.0027 US 7,492,268 B2 25 26 The calculation circuit preferably processes multiple, con ferred, embodiment, this absolute compensation algorithm secutive TOF acquisitions to effectively improve the timing works most effectively when (1) the wireless synchronization resolution that proportionally affects position accuracy and means is tightly coupled to the excitation of the acoustic precision. The digital filter discussed above introduces quan source, (2) the synchronizing signal's arrival is timed by the tization errors because of its discrete operation. And minor same mechanism that times the arrival of the reference trans fluctuations in the acoustical coupling produces timing jitter ducer's acoustic signal, and (3) the coordinate locations ofthe or uncertainty in the triggered zero-crossings of the acoustic sinusoidal. A Gaussian average or mean value of multiple sensors of the receiver constellation are established to a high TOF is a simple and effective filter strategy. Due to the pos degree of accuracy. sibility of poor acoustic coupling or misalignment, and dis- 10 The calculation circuit preferably employs two software tant transponder location from the processor unit, the nnmber methods of trilateration calculation to estimate transponder of detectable triggered zero-crossings may vary for a fixed position, wherein the particular method used depends upon duration of multi-cycle ultrasonic acoustic burst. The averag the availability of a synchronizing signal and the accuracy ing algorithm automatically adjusts to this condition by only desired. The first method is based on a relative TOF calcula including TOFs whose delta changes fall within the expected 15 tion and the speed of sound is treated as a constant estimated range of the nominal intra-pulse interval defined by the trans mission properties of the acoustic source. The nominal intra at ambient indoor room temperature. The second method pulse interval is determined and utilized by the following requires calculation of an additional TOF timestamp between compensation schemes. the transponder and reference receiver, but calculates the The calculation circuit preferably processes a relative TOF 20 speed of sound as an unknown every analysis period, and thus correction algorithm based upon a predictive tuned algorithm improves measurement accuracy. The first method eliminates that requires higher-order numerical differentiation calcula the global system timing variances and delays due to the tion of the past and current TOFs. This compensates the TOFs multiplicity of signal conditioning circuitry and eliminates that may have registered one intra-pulse interval earlier or the need for a controlling signal means synchronized at the later than the nominally expected time due to the trigger 25 generation of the transmission of the ultrasonic acoustic dilemma described above. By formulating these derivatives wave. The second method also employs relative TOF calcu into a truncated 2nd order Taylor series expansion and weight ing the terms contribution, an estimate of expected TOF is lation but requires an additional synchronization signal from calculated and compared to the actual TOF through an itera the processor unit to determine the absolute TOF between tive error minimization calculation. A minimized error that 30 transponder and reference receiver. Since the absolute TOF is results in a delta time change indicative of a discrete intra based upon a single channel only, its timing latencies can be pulse interval increase or decrease due to an early or late TO F, readily accounted for and easily corrected. This method com respectively, produces a characteristic value that directs the putes the speed of sound every analysis period, provided the algorithm to compensate the actual TOF by the intra-pulse synchronization signal is detected, without need for addi interval and restore it to its correct value. In the preferred 35 tional hardware temperature processing or requiring more embodiment, this relative compensation algorithm works then five (5) receivers, and automatically accounts for the most effectively when, (1) the maximally expected inter system's main accuracy limitation of speed of sound in air as period TOF change is less than the discrete intra-pulse inter val, (2) the TOF inter-period processing is contiguous, (3) the defined by Eq. 1.1, if uncorrected, yields a 1.6 mmlmranging 0 TOF increase or decrease is no more than a single intra-pulse 40 error for every 1 C. temperature shift. Ifthe synchronization interval, and (3) the Taylor series terms are suitably weighted signal is not detected and, therefore, the second method is not in the prediction algorithm. resolvable, the last calculated speed of sound can be utilized The calculation circuit preferably processes an absolute within the first method's calculation to minimize error. TOF correction algorithm at least once initially, when the c=34.6 mis+0.5813 m/s(Tc-25C C.) phase-locked loop is stable, but may be performed every 45 (1.1) analysis period depending on computational resources, that The TOF timestamps and speed of sound values are input determines the initial set ofTOF values for the relative cor into linear independent algebraic equations in a matrix for rection algorithm. The initial condition that precedes the start mulation to solve for the unknown transponder(s) position, in of the relative compensation algorithm may be due to the 50 a form as shown in Eq. 2.1, resnmption of a stable, locked tracking state after recovery from a fault condition and, therefore, requires computation of a set of reference TOFs producing minimum range error as a all al2 a13 al4 Xl b (2.1) starting basis. The algorithm utilizes a wireless synchroniza l a2l a22 a23 a24 X2 b2 tion means to determine a reference TOF calculation between A·X =B A= X= B= b the transponder and reference sensor of the receiver constel 55 a31 a32 a33 a34 X3 3 lation. By computing the reference range distance by the G41 a42 G43 G44 X4 b4 product of the reference TOF and speed of sound in air, this reference range may be compared to the range calculated from the matrices solutions described below. By iteratively To solve for the unknowns X, Eq. 2.1 is rearranged as and sequential increasing and decreasing the TO F s by a single 60 shown in Eq. 3.1, whereas the inverse of A requires compu C intra-pulse time interval and applying the input to matrices tation of the cofactor matrix A for the adjoint and determi formulations described below, all possible combinations of nant calculations for Eq. 3.2 and Eq. 3.3, respectively, compensation are permutated and tested, which produces a unique set ofTOFs that minimize the error between the cal (3.1) culated range distance with respect to the reference range 65 distance. This unique set of initial TOFs serves as the starting basis for the relative compensation algorithm. In the pre- A000062 Case: 12-1252 Document: 46 Page: 154 Filed: 08/29/2012 JX-003.0028 US 7,492,268 B2 27 28 -continued Xl -X2 Zl - Z2 -c!:J.T12 U (7.1) All A2l A3l A4l (3.2) Yl - Y2 Zl -Z3 -c!:J.Tl3 Al2 A22 A32 A42 2 Xl -X3 Yl - Y3 (A'l = Xl -X4 Yl - Y4 Zl - Z4 -CLl.T14 W Al3 A23 A33 A43 Xl -XS Yl - Ys Zl -ZS -CLl.TlS Dl A14 A24 A34 A44 (cLl.T12)2 + Ri - R~ (cLl.Tl3)2 + Ri - (3.3) 10 R~ (cLl.T14)2 + Ri - R~ To setup the coefficient matrix A, the utilization of five (5) (cLl.Tld + Ri - receivers produces the following set of relative TOF equa R~ tions defined by Eqs. 4.1-4, 15 Ll.T12 =Tr Tl (4.1) where R?=x/+y?+z? for 5~i~1 (7.2) Alternatively, if the second method algorithm is used, the Ll. T13=TrTl (4.2) unknown range ofthe reference receiver D 1 can be substituted by Eq. 8.1, Ll.T14=T4-Tl (4.3) 20 Dl =cTOl> Ll.Tol~>time of flight (TOF) from S(u,v,w,) to S(XVYi,Zl) (8.1) Ll.TlS=Ts-Tl (404) And, by rearranging terms, it is depicted in the matrix form The receiver locations are fixed within the system's inertial defined by Eq. 9.1, reference frame, while the transponder(s) are mobile with 25 respect to the same reference frame and are defined as fol lows, a. R2 (9.1) S(Xi'Yi,zJfor 5~i~1 =>fixed receiver locations Xl -X2 Yl - Y2 Zl -Z2 -( Ll.TOl Ll.T 12 + 0.5Ll.Tf2) u 1 Xl -X3 Yl - Y3 Zl -Z3 -(Ll.TOlLl.Tl3 + 0.5Ll.Tf3) R21 30 2 S(xo,Yo,zo)=S(u,v,w)~>unknown transponder location W Xl -X4 Yl - Y4 Zl -Z4 -(Ll.TOlLl.T14 + 0.5Ll.Tf4) R21 c2 Since each receiver is fixed at a distance Di from the tran Xl -Xs Yl - Ys Zl -Zs -(Ll.TOlLl.TlS + 0.5Ll.Tfs) R21 sponder as determined by the receiver constellation geometry and because the acoustic waves propagate spherically, by 35 Although similar results may be obtained by application of using Pythagorean's theorem the following set of range equa more computational efficient processes such as pivotal con tions are defined in Eqs. 5.1-5, densation or Crout's decomposition, the application of Cram (x l-U)2 +(Yl-V)2 +(z 1-w)2~D12 (5.1) er's rule was used to evaluate the first-order determinant in Eq. 3.3 using second-order determinants from Laplace 40 (xru)2+(yrv)2+(zrw)2~D22 (5.2) expansion. The final transponder(s) position equations are defined by Eqs. 10.1-8. (xru)2+(yrv)2+(Z3-W)2~D32 (5.3) IA21 (10.1) (X4-U)2+(y4-V)2+(Z4-W)2~Dl (504) 45 u=w (xs-u )2+(yS-V)2+(ZS- w)2~Ds 2 (5.5) IA21 (l0.2) V=- IAI Equivocally, the four (4) non-reference receivers are pref erably located at an incremental distance relative to the ref (l0.3) erence receiver, so by substitution of the incremental distance 50 defined by Eq. 6.1, the following set of relativistic range (lOA) equations are defined by Eqs. 6.2-5, Di=Dl+C~Tll for 5~i~2 (6.1) 55 (X2-u)2+(y2-V)2+(zrw)2~(Dl+cLl.Td2 (6.2) (10.5) (10.6) (xru)2+(yrv)2+(zrw)2~(Dl+cLl.T13)2 (6.3) (10.7) (X4-U)2 +(y4-V)2 +(Z4- w)2~(Dl +CLl.T14)2 (604) 60 (10.8) (xs-u )2+(yS-V)2+(ZS- w)2~(Dl +cLl.TlS )2 (6.5) If the first method is used, D, the range of the transponder By expanding and rearranging the terms ofEqs. 6.2-5, a set to the reference receiver from Eq. 10.4 may be calculated as a of four linear algebraic equations and four unknowns for the 65 redundant confirmation of the Eqs. 10.1-3 calculations, pro first method algorithm, depicted in the matrix form ofEq. 2.1, vided the frame of reference origin and location of the refer is defined by Eq. 7.1, ence receiver are identical or their offsets accounted for. Ifthe A000063 Case: 12-1252 Document: 46 Page: 155 Filed: 08/29/2012 JX-003.0029 US 7,492,268 B2 29 30 second method is used, C, the speed of sound in air, from Eq. functions may be evaluated through a conventional look-up 10.4 must be computed every analysis period if its value is table or by a power series expansion. anticipated to be used in the first method in the absence of a Preferably, the overall analysis period duration is effec synchronization signal. tively trebled until the three (3) transducers' positions are The orientation of the transponders can be derived from a calculated, which reduces the system's frequency response similar utilization of the above algorithms for a transponder and imposes an increased latency effect. Typically, robust configured with a triad of ultrasonic transmitters. The trans absolute orientation processing requires more stringent line ducers are preferably arranged in a triangular plane at the of-sight operation and is reserved for more sensitive, less transponder of sufficient area for the desired angular resolu dynamic, and reduced ROM movement trajectories, e.g., bal- tion. The sequential excitation of each transducer and subse 10 ance and sway. Therefore, the latency effect is less noticeable quent calculation of position by the aforementioned methods upon the real-time perfonnance of the sensory interfaces. provides sufficient information to detennine orientation by In the preferred embodiment, the interactive hand-held the inverse kinematic calculations of Eqs. 11.1-4, where the transponders support a dual axis inertial sensor, which is analysis is simplified by assuming the origin of rotations operably configured to provide tilt (pitch and roll) orientation occurs about Tl and Tl23 represents the initial relative posi 15 in its horizontal mounting plane. The inertial sensor is tion matrix from this origin and T\23 is the transfonned or mounted in the intended operational horizontal plane with forward kinematic position matrix. respect to the systems inertial frame of reference. Once the sensors signals has been converted to an acceleration value (11.1) that varies between +/-1 g the tilt in degrees is calculated as 20 shown in Eqs. 13.1-2, for pitch and roll, respectively. ~a sin(A)l g) (13.1) X2COS()y + Z2sinBy X3cos8y [(11.2) sin8Ax2sin8y - z2cos8y) x3sinBxsinBy ~a sin(A)l g) (13.2) -cos8x(x2sin8y - z2cos8y) -x3cosOxsinBy 25 This outside-in ultrasonic tracking implementation, where (11.3) the transponders are mounted on the mobile object, produces inherent temporal delays due to the finite TOF registration Xl cos(h + Yl sinB2 X2 cos(h + Y2 sinB2 X3 eose2 + Y3 sinB2 [ and calculation delays after the transponder has already - Xl sinB2 + Yl cos(h - x2 sinB2 + Y2 eosB2 - x3 sinB2 + Y3 eose2 [ moved into a different position before the measurement is z; Z2 Z, 30 complete. This overall latency period is compensated and minimized through use of a Kalman filter data processing algorithm to estimate the pose of the transponder by opti (1104) mally and recursively combining past history, new measure ments, and a priori models and information. Generally speak- By examination of the matrices element equivalency of 35 ing, the Kalman filter is a digital filter with time-varying gains Eqs. 11.2-3 and manipulation of terms so that the angles may that are optimally determined through a stochastic dynamical be found using the inverse tangent function, the following model of the motion. The overall goal is to minimize filter lag rotation equations Eqs. 12.1-3 are derived, while providing sufficient smoothing of the motion data. An adaptive, multi dynamic model is developed based (12.1) 40 upon the kinematic quality of the expected movement trajec tory. The predictive kinematic model for the Kalman filter is depicted in matrix fonn utilizing a truncated 2nd order Taylor (12.2) series expansion as below in Eqs. 14.1-2, By = atan , -z, , . ) (cos8x(x3cos82 + Y3 sm8,) 45 , x3sin81 sin8y ) (12.3) (14.1) Y3 - -c-O--;S8;-23----'- ( =[~ ~ttl +[:l (}z = atan " [:L x3 (14.2) 50 COS 8 _ =cos 8 from previous iteration (1204) 2 1 2 [1°[: :' O~[[l[:[ (12.5) The Kalman filter is now described for a single dimension, 82 = atan(~) for 1st iteration 55 although it is utilized for prediction and smoothing for all position dimensions. The predictor stages consist of the cal culation of the state and the error covariance projection equa These calculations are perfonned through iterative step tions. The state projector equation, Eq 15.1, utilizes a discrete processes which inherit angular approximations of the pre time-sampled difference equation of r calculated from Eq. ceding steps until the final desired angular accuracy is 60 15.2. In other words, the numerically derived velocity and achieved by assuming the conditions of Eqs. 12.4-5. There acceleration components of motion are linearly combined fore the rotation 8z ' roll, is first approximated by Eq. 12.5; with the previously a priori position to estimate the new then the rotation 8x ' pitch, is approximated by Eq. 12.1; and position. The corrector stages consist of sequential computa then the final rotation 8y , yaw or tum, is approximated by Eq. tion of the gain, updated state estimate, and updated error 12.2. The next approximation of8z utilizes the previous value 65 covariance equations. The a posteriori state estimate, Eq. of8z in Eq. 12.3 and the similar steps are preferably repeated 15.4, is based on a linear combination of the weighted mea until the desired accuracy is achieved. The transcendental surement residual and the last state estimate. A000064 Case: 12-1252 Document: 46 Page: 156 Filed: 08/29/2012 JX-003.0030 US 7,492,268 B2 31 32 (15.1) (17.1) (15.2) o -1 2 -1 11 1 2 0 -5 3 J.l Ck(J.l) = Gk 2 0 1 4 -3 J.l2 o 0 -1 1 J.l3 Fk (15.3) Kk =--- (Pk + Rk ) -O.SJ.l + J.l2 - O.SJ.l3 (17.2) 1 - 2.SJ.l2 + l.SJ.l3 10 Ck(J.l)=Gk (15.4) O.SJ.l + 2J.l2 - I.SJ.l3 -O.SJ.l2 + O.SJ.l3 (15.5) The new error covariance projector, Eq. 15.2, is it's previ- 15 ously computed value combined with the current process (17.3) noise covariance, Qk' which is tuned by an example model The fl value is normalized and represents the % value nd derived from the measured motion dynamics shown in Eq. between the 2 3rd control points. To calculate the interpo nd 16.1. The gain's measurement noise covariance, Rk , is lated value between the 1st and 2 orthen-l th and nth control defined as a small constant and based upon the actual static 20 points, the value of first control point of the pair and the value timing variance empirically measured. The smaller this value of the last control point pair are doubly entered into the the more confidence there exists in the systems' measurement geometry matrix, respectively. The appropriate dflldt is deter capability. mined by the desired rate ofplayback ofmovement trajectory. In the preferred embodiment, the product of the numeri 25 To playback at the same rate as the recorded session, and cally-derived 1st and 2nd order derivatives of the measured assuming fairly constant velocity, a timestamp should also be position scaled by a frequency dependent gain provides a saved at each control point registration so that the fl calcula computationally practical adaptive dynamic process noise tion is correctly scaled by the delta time interval. The n-length estimate model. The derivative product term increases Qk set of control points would be manually registered by the user proportionally for higher velocity and acceleration compo 30 pressing a switch or automatically post processed by a sorting nents of motion, e.g., quick, abrupt directional changes, method where a control point is registered at the tangents of which effectively increases the gain and, therefore, means the trajectory having sufficient magnitude and/or experience more confidence exists in the measurement rather than the sign changes which indicates discontinuous or non-mono- estimate. This provides faithful, low-latency response to tonic movement. high-frequency motions. Conversely, the frequency scaling 35 The major functional interfaces of the transponder unit term decreases the predictive "overshoot" characteristic of preferably include the sensory interface, transducer interface, lower power, repetitive motion, e.g. slower, cyclic, ROM processor, and communication interface. The following trajectories, which effectively decreases the gain and, there descriptions of the transponder unit are based upon the depen fore, means more confidence exists in the estimate rather than 40 dence flow represented by FIG. 6. the measurement. It should be appreciated this filter imple The sensor interface refers to the collective support for the mentation provides superior tracking fidelity and comparable ultrasonic transmitter, heart rate receiver, and accelerometer smoothing characteristics as compared to practical lengths of circuits. The ultrasonic transmitter circuit is preferably gated finite impulse response running-average filters and various by a pulse-width modulated (PWM) digital signal at nomi low-orders infinite impulse response filters. It achieves 45 nally 0.8% duty cycle of the 40 kHz resonant frequency, e.g., enough predictive response to compensate for the inherent a single 250 flS pulse every analysis period, by the processor TOF and computation latencies, while providing and compa circuit. The radiated ultrasonic signal strength is controlled rable smoothing properties of other filter types. by gating a MOSFET transistor switch at a duty cycle which optimally energizes the transducer's series resonant tank cir (16.1) 50 cuit for sufficient duration. The resonant circuit's reactive components include an impedance matching inductor, the (16.2) transducer's intrinsic capacitance, and a small damping resis In the preferred embodiment, a three dimensional (3D) tive load. At resonance, a electrical damped sinusoidal with a potential up to -400 V _ is developed across the transducer piecewise cubic curve interpolates a movement trajectory for pk pk 55 to sufficiently drive it at acoustical power levels practical for smoothing and reduced sample storage for greater memory the system's intended range of operation. Enabling a lower efficiency. Preferably, four (4) sequential discrete control duty cycle control through means of a software algorithm points of the n-length set of control points, the sample reso monitoring the transponders range would effectively lower lution dependent upon the desired movement granularity, and the transponders power consumption and radiate less ultra- corresponding timestamp are needed to calculate in real-time 60 sonic acoustic energy for close range operation when signal the interpolated position between any pair of control points. A saturation and clipping is undesirable. Conversely, a higher Catmull-Rom spline algorithm is the preferred method in that duty cycle control would radiate greater ultrasonic energy to the path intersects the control points and would best approxi compensate for less efficient, non-optimal acoustical cou mate a movement that may have acute directional changes. pling orientations of the transponder with respect to the The Catmull-Rom spline algorithm is defined by Eqs. 17.1-3, 65 receiver constellation. Optionally, two additional transducers where the geometry matrix G k represents the matrix of three may be driven in unison or sequentially from a different dimensional (3D) control points. transponder assembly to support measurement of absolute A000065 Case: 12-1252 Document: 46 Page: 157 Filed: 08/29/2012 JX-003.0031 US 7,492,268 B2 33 34 rotation about a single or multiple axes, or provide calculated clock also provides all the capture and control timing require positional redundancy for certain difficult line-of-sight appli ments for the other specified circuits. Multiple parallel I/O cations. ports and dedicated asynchronous serial communication sig The heart rate receiver circuit wirelessly receives a 5 kHz nals provide digital control for the circuits of the parallel/ heart rate signal from a POLAR® transmitter belt. The trans serial I/O circuit. mitter, wom around the chest, electrically detects the heart In the preferred embodiment, the graphic LCD and touch beat and starts transmitting a pulse corresponding to each screen circuit is the primary user input device for database heart beat. The receiver captures the signal and generates a management for an interactive transponder configuration. For corresponding digital pulse which is received by the timing example, it may be a 128x64 graphical liquid crystal display capture-control circuit of the processor interface. A software 10 system (LCD) and associated 4-pin touch screen input device. algorithm processes the signal with known time-based aver A preferred LCD device is the 51553 manufactured by Optrex aging and an adaptive window filter techniques to remove any and the preferred touch screen device is the TSG-51 manu extraneous artifact or corruption caused by interfering factured by Apollo Displays. LCD display information, con sources. figuration commands, and bitmaps images can be loaded The accelerometer circuit consists of a low cost +/-1.5 g 15 through the software calculation engine via a parallel dual axis accelerometer that can measure both dynamic, e.g. memory interface to emulate a graphical user interface. A vibration, and static, e.g. gravity or tilt, acceleration. If the touch screen input device is connected to a controller circuit accelerometer is oriented so both its axes are parallel to the to decode soft key presses at areas over the graphical object. earth's surface it can be used as a two axis tilt sensor with a Preferably, the key presses are registered, filtered, decoded, roll and pitch axis. 20 and processed by the controller and then transferred to the The stimuli interface circuit provides the primary visual software calculation engine via an interrupt driven asynchro sensory interface preferably comprised of a linear array of nous serial communication channel of the I/O interface. A five (5) bright, white light emitting diodes (LED) and asso preferred LCD controller is the UR7HCTS manufactured by ciated drivers. The preferred LED device is a CMD87 manu Semtech. factured by Chicago Miniature Lamp. These LEDs' intensity 25 The timing capture-control circuit provides controlling is controlled by a white LED driver. The preferred white LED means for the stimuli interface and portions of the sensor driver device is a MAX1570 manufactured by Maxim. The interface. The stimuli interface is preferably comprised of a white LED driver provides a maximum 120 rnA constant five channel 16-bit timer PWM module with programmable current source to each LED for optimal uniform lumines interrupt control which provides 250 llS timing resolution to cence. The drive current can be proportionally regulated 30 automatically modulate the circuits' drivers through variable through external pulse width modulation (PWM) means from duty cycle control. the processor circuit to modulate its brightness level. Addi In the preferred embodiment, the AID conversion circuit tionally, an electronic switch is connected in series to each receives the output from the accelerometer circuit and con LED drive to individually control its active state. By simul sists of a two channell O-bit analog-to-digital converter used taneously controlling the PWM duty cycle and active state of 35 determine the rotational angle of roll and pitch in the accel each LED, the light strobe can appear to smoothly migrate erometer deviates from its horizontal plane orientation. This along the linear array in spite of its discontinuous operation. information is communicated to the signal processor via the Preferably, the stimuli interface circuit provides the pri radio link. mary aural stimulus by means of a 4 kHz piezo buzzer. The In the preferred embodiment, the radio link circuit is com preferred device is SMT-3303-G manufactured by Projects 40 prised of a wireless bi-directional communication interface Unlimited. This electro-mechanical buzzer requires an exter (with a receiver and transmitter shown generally at 20 and 30) nal transistor drive circuit and digital control signal gated at a to (1) receive a synchronization signal for control of the rate near its resonant frequency. The buzzer inputs are con transponders interoperability, (2) to transfer acquired local nected to and controlled by PWM means from the processor sensor data, including, but not limited to, accelerometer, heart circuit to provide a gross volume adjustment which is depen- 45 rate, battery, user I/O status, to processor unit and (3) to dent upon the amplitude of the drive signal. provide means to configure its local database from command Additionally, the stimuli interface circuit provides the pri of processor unit. The preferred wireless communication link mary tactile stimulus by means of a vibrator motor. The driver is based upon the AT86RF211, a highly integrated, low for the vibrator motor enables a 120 rnA DC current source to power FSK transceiver optimized for license-free ISM band excite the motor armature. The preferred driver device is the 50 operations from 400 MHz to 950 MHz. and manufactured by MAX1748 manufactured by Maxim. The rotational speed of Atmel. Its key features are described above. the motor's armature is controlled by PWM means from the In the preferred embodiment, the switch I/O circuit uses a processor circuit. SPST push button switch for user input to control the sys The processor circuit preferably receives input from the tem's operational states, start and stop program execution, stimuli interface, sensor interface, and the communication 55 and function as feedback input to the program. A preferred interface and provides controlling signals therein. The pre device is the KSS231 SPST pushbutton switch manufactured ferred processor circuit is the MC9S08GB60 which is manu by ITT Industries. factured by Motorola Inc. It is a low-cost, high-performance What is claimed is: 8-bit microcontroller device that integrates the specialized 1. A system for tracking position of a user, comprising: hardware circuits into one convenient device. The software 60 a first communication device, adapted for being attached calculation engine circuit operates from an embedded 60 KB to, or held by, the user, comprising: FLASH for program memory with in-circuit programmable a transmitter for transmitting signals; capability and 4 KB RAM for data memory. The time base a receiver for receiving signals; and circuit is preferably comprised of an external, high-noise an output device; and immunity, 4.0 MHz system clock, which multiplies this value 65 a processing system, remote from the first communication by the internal frequency-locked loop for a bus clock of 40.0 device, for wirelessly receiving the signals transmitted MHz and single instruction execution time of 25 llS. This by the transmitter, determining position information for A000066 Case: 12-1252 Document: 46 Page: 158 Filed: 08/29/2012 JX-003.0032 US 7,492,268 B2 35 36 the first communication device and sending data signals 9. The method of claim 8, wherein: to the receiver to provide feedback or control data to the the system further comprises a user input device adapted user; for communication with the processing system through wherein the first communication device receives and pro the transmitter; and cesses the received data signals and generates sensory the step of exchanging data signals further comprises send ing user input data signals to the processing system. stimuli for the user, based on the received data signals 10. An apparatus for use in conjunction with a remote and delivered through the output device. processing system for tracking position of a user, comprising: 2. The system of claim 1, further comprising: a transmitter for transmitting position information signals a second communication device, in electrical communica- 10 to the remote processing system; tion with the first communication device and in wireless a receiver for receiving feedback or control data signals communication with the processing system, and adapted wirelessly from the remote processing system, the data for being attached to, or held by, the user; signals derived from processed signals from the trans wherein the processing system is remote from the second mitter; and an output unit for delivering sensory stimuli to the user, communication device and is adapted for detennining 15 based upon the data signals. the position of the second communication device rela 11. The apparatus of claim 10, wherein: tive to at least one of the first communication device and the data signals are derived by comparing the position the processing system. information transmitted to reference position infonna 3. The system of claim 1, wherein: 20 tion. the first communication device comprises a user input 12. The apparatus of claim 10, further comprising: device adapted for communication with the processing a first and a second communication device, each adapted system through the transmitter. for being attached to, or held by, the user; and 4. The system of claim 3, wherein: each of the communication devices is in communicative the user input device is adapted for calibrating the first 25 contact with the processing system directly or through communication device to establish a reference position. the other communication device. 13. The apparatus of claim 12, further comprising: 5. The system of claim 1, wherein: a user input device deployed on at least one of the commu the processing system is adapted to determine acceleration nication devices and adapted for communication with information of the first communication device. 30 the processing system through the transmitter. 6. The system of claim 1, wherein: 14. The apparatus of claim 10, further comprising: the sensory stimuli are at least one of: aural, visual or an interactive interface such that movement of the appara tactile. tus controls the movement of an object in a computer 7. The system of claim 1, wherein: generated virtual environment. 35 15. A system for tracking movement of a user, comprising: the processing system determines posItion infonnation a first communication device, adapted for being attached without interference from occluding objects. to, or held by, the user, comprising: 8. A method for tracking position of a user, comprising the a transmitter for transmitting signals; steps of: a receiver for receiving signals; and providing a system according to claim 1; 40 an output device; and establishing a wireless communication between the first a processing system, remote from the first communication communication device and the processing system; device, for wirelessly receiving the signals transmitted exchanging data signals between the first communication by the transmitter, detennining position information for device and the processing system, the first communica the first communication device and sending data signals tion device sending position information data signals to 45 to the receiver to provide feedback or control data to the the processing system and the processing system send user; ing feedback or control data signals to the first commu wherein the first communication device receives and pro nication device; and cesses the received data signals and generates sensory stimuli for the user, based on the received data signals providing sensory stimuli to the user through the output 50 and delivered through the output device. device, the sensory stimuli based upon the received feed back or control data signals. * * * * * A000067