||||III IIII USOO5596494A United States Patent (19) 11 Patent Number: 5,596,494 Kuo (45) Date of Patent: Jan
Total Page:16
File Type:pdf, Size:1020Kb
||||III IIII USOO5596494A United States Patent (19) 11 Patent Number: 5,596,494 Kuo (45) Date of Patent: Jan. 21, 1997 54 METHOD AND APPARATUS FOR Remote Sensing and Image Interpretation, 3rd. Ed. 1994, T. ACQUIRING DIGITAL MAPS Lillesand, R. Kiefer. 76 Inventor: Shihjong Kuo, 1433 N.W. 189th Way "Aerial Image Databases for Pipeline Rights-of-Way Man #2, Beaverton, Oreg.97006 agement', Photogrammetric Engineering and Remote Sens ing, vol. 60, No. 3 pp. 347-353, Mar. 1994, M. Jadkowski (21) Appl. No.: 338,423 et al. 22 Filed: Nov. 14, 1994 Primary Examiner-Michael Zanelli (5ll Int. Cl." ............................... G06F 17/40 Attorney, Agent, or Firm-Pugh Associates; C. Emmett 52 U.S. Cl. ........................ 364/420; 364/449.1; 348/144 Pugh 58 Field of Search ..................................... 364/420, 449; 342/357,457; 348/117, 144-147 (57) ABSTRACT (56) References Cited A method and an apparatus are disclosed that, when used in an airborne environment, acquire spatial imagery, that are U.S. PATENT DOCUMENTS instantaneous pictures of one or more terrestrial scenes, and 3,500,733 3/1970 Kreuz et al. ............................. 95/12.5 the absolute geophysical coordinates (Latitude and Longi 3,950,097 4/1976 Göpfert et al. ............................. 356.2 tude) of ground objects simultaneously. Any number of 4,020,344 4/1977 Kerschbaum ......... ... 250/330 series of terrestrial images and the absolute geophysical 4,143,400 3/1979 Heckman, Jr. et al. .................. 358/95 coordinate information are recorded digitally. The absolute 4,199,869 4/1980 Van Steenwyk .......................... 33/302 geophysical coordinate information is displayed in real 4,313,678 2/1982 Colvocoresses ............................ 356/2 time. The terrestrial imagery and the absolute geophysical 4,922,254 5/1990 Schuessler et al. ....................... 342.25 5,019,828 5/1991 Schoolman ........ ... 342A457 coordinate coordinates are displayed and queried in near 5,045,937 9/1991 Myrick .......... ... 358/09 real-time. Thus, a new type of digital maps, comprising a 5,101,268 3/1992 Ohba ......................................... 358/88 two-dimensional terrestrial image and the absolute geo 5,104,217 4/1992 Pleitner et al. ............................. 356/2 physical coordinates of any terrestrial objects in the image, 5,140,532 8/1992 Beckwith et al. ... 395/101 is obtained using solely digitally-recorded spatial data on an 5,172,233 12/1992 Yoshihara et al. ...................... 358/222 original, unprocessed mission medium, or any secondary 5,251,037 10/1993 Busenberg ........ 358/21328 media. Near real-time images and accurate longitude and 5,267,042 11/1993 Tsuchiya et al. ..... 358/209 latitude coordinates can be displayed and queried before the 5,467,271 ll/1995 Abel et al. .............................. 364f420 aircraft lands. OTHER PUBLICATIONS Manual of Photogrammetry, 4th Ed. 1980, Published by the American Society of Photogrammetry. 23 Claims, 19 Drawing Sheets GPS ANTENNA RADO LINK 34 37 ATTITUDE SENSING UNIT 32 TERRESTRAL FRAME NSTRUMEN SCENE MAGING COMPUTER DEVCE 31 33 INPUT DEVICE DSPLAY CONSOE 36 35 U.S. Patent Jan. 21, 1997 Sheet 1 of 19 5,596,494 GPS ANTENNA RADIO LINK 34 37 ATTITUDE SENSING UNIT 32 FRAME TERRESTRIAL NSTRUMENT SCENE MAGING DEVICE COMPUTER 31 33 DISPLAY INPUT DEVICE CONSOE 36 35 F.G. 1 U.S. Patent Jan. 21, 1997 Sheet 2 of 19 5,596,494 F.G. 3 U.S. Patent Jan. 21, 1997 Sheet 3 of 19 5,596,494 RADIO LINK 37 --- - ORIENTATION GPS 1 PPS i SENSOR 66 RECEIVER TIMING SYNCHRONIZER 50 GPS 52 DATA LEVEL SENSOR rtuDE 67 i ATTTUDE EVENT 32 AD, "DATA TMING MEMORY IMAGE DATA, FRAME TERRESTRIAL DATA,MAGE BUFFER ATITUDE DATA, SCENE MAGING & GPS DATA, DEVICE CONTROLCONTROL TIMING DATA 31 SIGNAL DIGITAL RECORDER 60 FRMWAREA REFERENCN INPUT DEVICE 59 GPROCESS 36 58 33 POSITION OF EXPOSURE STATION, GROUND POSITION OF IMAGE CENTER DISPLAY CONSOLE FIG. 4 35 U.S. Patent Jan. 21, 1997 Sheet 4 of 19 5,596,494 IMAGE DATA, ATTITUDE DATA, GPS DATA, TIMING DATA GEO FIRMWARE B REFERENCNG DIGITAL PROCESS RECORDER, 70 58 60 2D IMAGE, INPUT DEVICE POSITION OF EXPOSURE STATION, 36 GROUND POSITION OF USER-QUERIED MAGE FEATURE DISPLAY CONSOLE 35 F.G. 5 U.S. Patent Jan. 21, 1997 Sheet 5 of 19 5,596,494 -N N NIT OK)—YES EVENT LOOP 80 1 84 MISSION COMPLETE EXCEPTION SYSTEM SHUTDOWN F.G. 6 U.S. Patent Jan. 21, 1997 Sheet 6 of 19 5,596,494 PROGRAM NIT NIT ox-YES ( USER OUERY LOOP 3 N N USER QUT EXCEPTION PROGRAM SHUTDOWN 3 FIG. 7 U.S. Patent Jan. 21, 1997 Sheet 7 of 19 5,596,494 TAR OF SYSTEM INT -N MEMORY PROGRAM BUFFER AND GPS INT ADC INT MEMORY CHECK CAMERA CONTROL INT NIT DATA 8. MISSION PROFILE FIG. 8 U.S. Patent Jan. 21, 1997 Sheet 8 of 19 5,596,494 START SEQUENCE MISSION SET UP PARAMETER WAD DATA VERIFED2 PARAMETER V ACOUSIT 140 ON VALID QUT PARAMETER 156 WRITE-STATE CHANGED WRITE STATE OR PARAMETERS HANGED2 ACORUSTION SEQUENCE 160 FIG. 9 U.S. Patent Jan. 21, 1997 Sheet 9 of 19 5,596,494 COMMENCE MARKTIME, UPDATE INITIATE DISPLAY ACOUISTION EXPOSURE CONSOLE 184 COMMENCE NEXT ACOUSTION MARKTIME, TRANSFER STATUS ATITUDE DATA EXCEPTION, CHECK, 186 USER OUT GENERATE TRIGGER EVALUATE TRIGGER STATUS, WAT UPDATE CONSOL TRANSFER TRANSFER GPS DATA GPS DATA, FALSE 190 ATTITUDE DATA WRITE IMAGE DATA, GPS DATA, WRITE- GEO GEO-REFERENCEDATTITUDE DATA, DATA, Tu-get- REFERENCNGPROCESS ANCILLARY DATA 58 TO RECORDER 192 F.G. 10 U.S. Patent Jan. 21, 1997 Sheet 10 of 19 5,596,494 DISPLACEMENT VECTOR BETWEEN ANTENNA PHASE LASTN CENTER AND EXPOSURE AIRCRAFT STATION POSITION & UCT TIME MARK CALCULATE TRAJECTORY GEO-PHYSICAL PARAMETERS OF COORONATE OF ANTENNA PHASE EXPOSURE STATION LASTN CENTER AIRCRAFT VELOCITY UCT TIME MARK OF F.G. 1 1A EXPOSURE ORIENTATION SUBTRACT SENSOR DIFFERENCE OUTPUT BETWEEN RUE ORFTANGE NORTH AND FROMADC MAGNETIC NORTH FIG. 11B DIF. BETWEEN TRUE NORTH AND MAGNETIC NORTH LASTN CACULATE UCT TIME PARAMETERS ROLL, PITCH, DRIFT MARK, ROLL, OFTRAJECTORY OF ANGLES AT TIME OF PITCH, DRIFT AT TITUDE EXPOSURE ANGLES VARIABLES UCT TIME FIG. 11C MARK OF EXPOSURE U.S. Patent Jan. 21, 1997 Sheet 11 of 19 5,596,494 7 PSTNST ROTATE ROTATE COORONATE COORDINATE EXPOSURE SYSTEM SYSTEM STATION PTCH ROLL ANGLE ANGLEAT AT TIME OF TIME OF EXPOSURE EXPOSURE TRANSATE ROTATE SCALE, COORDINATE COORDNATE TRANSLATE SYSTEM TO SYSTEM COORONATE GROUND PLANE DRIFT FOCAL ANGLE + LENGTH + HEADING HEIGHT OF ABOVE AIRCRAFT GROUND ABSOLUTE GEO-PHYSICAL COORDNATE OF GROUND FEATURE FIG. 12 U.S. Patent Jan. 21, 1997 Sheet 12 of 19 5,596,494 242 Fig 3B 244 f h/cost U.S. Patent Jan. 21, 1997 Sheet 13 of 19 5,596,494 X' -z' plone O4's principal plone imoge plone U.S. Patent Jan. 21, 1997 Sheet 14 of 19 5,596,494 FILE IMAGE SEC. DATAFILE FILE HEADER SEC. DATA FILE1 -- CAMERA MODEL GPS MODEL SENSORMODEL TIMEISYNC MODEL DATA SYNC FRAME GRABBER NAME MODE VERSION RADiO LINKMODEL DATE PXELDIMENSION MAX RESOLUTION FORMAT LENS MODEL FRMWARE DATA FOCAL LENGTH SEC. F STOP CALIBRATION DATA ANTENNA HW PARAMETER DISPLACEMENT SEC. MAGNETIC DECLINATION EXPOSURE MISSION PARAMETER DURATION SEC. EXPOSURE TIME BASE DESTINATION NUM OF GPS FX START TIME GPS/ATTITUDE NUM OF SAMPLES END TIME SEC. ANTENNA POSITION 1 FRAME COUNT TIME OF FIX LINE INDEX GPS VELOCITY LINE START DERVED DATA PDOP DATE SEC. DATUM STATUS TYPE ROLL 8. PITCH EXPOSURE ORFT STATION TME OF SAMPLE LAST STATION ANTENNA POSITION 2 IMAGE CENTER POSITION IMAGE CORNERS F.G. 14 POSTION ENDAP UNIT U.S. Patent Jan. 21, 1997 Sheet 15 of 19 5,596,494 PROGRAM TO USER VIDEO DISPLAY MEDIA CHECK QUERY MEMORY CHECK r OOP YES YES NO NO NO F.G. 15 U.S. Patent Jan. 21, 1997 Sheet 16 of 19 5,596,494 RETRIEVE DATA DISPLAY SHOW 320 IMAGE CURSOR 324 NEW IMAGE UPDATE DISPLAY, UPDATE QUERY EXCEPTION, ECODE USE USER OUT N INPUT > GEO 332 REFERENCNG PROCESS 58 -- UPDATE ------ QUERY RESULT 328 326 FIG. 16 U.S. Patent Jan. 21, 1997 Sheet 17 of 19 5,596,494 CURSOR we x=324, y=267, I=89, Lot 38,88981 N, LOn 77, Ol O2 W Fig. 7 U.S. Patent Jan. 21, 1997 Sheet 18 of 19 5,596,494 GPS ANTENNA 34 GPS 1 PPS TIMING RECEIVER SYNCHRONIZER GPS 52 50 DATA ATITUDE UNIT 32 ATTITUDE ATTTUDE SYSTEM DATA TIMING i t TERREST FRAME | NSF MEMORY iMAGE DATA, RAL MAGING DATA, BUFFER 8, AT TITUDE DATA, SCENE Vice CONTROL cool GPS DATA, SIGNAL TMING DATA OPERATOR INPUT GEO DIGITAL NPU FIRMWAREA REFERENCING RECORDER DEVICE PROCESS SO 36 58 POSITION OF EXPOSURE STATION, GROUND POSITION OF IMAGE CENTER DSPLAY CONSOLE FIG. 18 33. U.S. Patent Jan. 21, 1997 Sheet 19 of 19 5,596,494 RADIO LINK GPS ANTENNA 37 34 GPS 1 PPS TIMING RECEIVER SYNCHRONIZER 50 GPS 52 ATTTUDE DATA SENSING UNIT AT TITUDE 32 ATITUDE SYSTEM DATA TIMING IMAGE IMAGE DATA, TERRESTRAL MAGINGRAM DAA BUFFERMEMORY & - AT TITUDE DATA, SCENE DeVice CONTROL control GPS DATA, SIGNAL TIMING DATA OPERATOR INPUT GEO DIGITAL INPUT FRMWAREA REFERENCENG DEVICE RECORDER 59 PROCESS 36 58 60 POSITION OF EXPOSURE STATION, GROUND POSITION OF MAGE CENTER DISPLAY CONSOLE 35 F.G. 19 33 5,596,494 1. 2 METHOD AND APPARATUS FOR recorded by an aerial camera does not tell the absolute ACQUIRING DIGITAL MAPS geophysical coordinates, i.e., the latitude and the lon gitude, of ground objects on the film image directly. Chapter VII of Manual of Photogrammetry describes the elements, practices, and cost of planning and execu TECHNICAL FIELD tion of aerial photography. Careful planning is always The present invention relates to photogrammetry, aerial necessary, but it does not eliminate all sources of errors photography, and map generation.