Nature Inspired Basic Research at Eglin 6 October 2014 Ric Wehling & Dr. Jennifer Talley Research Biological Scientist RWWI Integrity Service Excellence Air Force Research Laboratory The Need for Agile Autonomy • Report on Technology Horizons highlights a class of problems RWW has been addressing for years: how to build systems capable of autonomous behavior? • The need is for ‘smart’ systems that can navigate, identify and track ‘targets,’ and perform the mission without having to rely on external aids (GPS, a human in the loop, etc.) • We are looking to Nature for inspiration for the sensor suites and information processing to enable agile autonomous systems DISTRIBUTION A. Approved for public release; distribution unlimited. 2 Guided Flying Machines Have Common Core Instrumentation ATTITUDE, ATTITUDE RATE Strain gauges, hair cells Gyroscopes Halteres distributed across body Compound Eyes, Ocelli Antennae: Acoustic sensors VISION Gravity sensors Olfactory sensors Seeker Compound Eyes Geomagnetic sensors Air speed sensors etc. DISTRIBUTION A. Approved for public release; distribution unlimited. 3 Biologically-Inspired Unmanned Autonomous System: GN&C NOVEL Acoustic sensors BIO-INSPIRED Ocelli SENSORS AND Olfactory Sensors PROCESSING Mechanosensors Antennae NAVIGATION, WFOV STATE IMAGE SEEKER VECTOR CONTROL PRO- GUIDANCE AUTOPILOT SENSOR ESTIMATOR AUTHORITY CESSOR LAW IMU: Optic flow Mode Sensing INERTIAL SENSOR Power Materials Structures Airframe Signatures Understand and apply the basic principles; not emulate flying insects DISTRIBUTION A. Approved for public release; distribution unlimited. 4 In-house Instrumentation Development • Electro Retinography Rig • Stargate (512 UV, blue, green interdigitated 16 x 32 deg fov LED arrays) • Automated goniometer • Flight path characterization • Histology • Environmental monitoring for providing realistic environments during measurements • UV-vis spectral / polarization camera 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 5 In-house Instrumentation • Available associated capabilities – KHILS panoramic Biodome (RGB 120 degree by 240 degree presented imagery, designed to train humans) – Scanning Electron Microscopes 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 6 New In-house Activities • Building capability – Magnetosensing – Antennal investigations – Hex grid array geometry – Compressive sensing (bioprincipic) 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 7 Comparative Approach Same Sensors Same Behavior Different Sensors • Predatory air strikes • Compound Eyes • Fast • Ocelli Robber Fly • Agile • Antenna • Halteres Dragonfly • Compound Eyes (& Damselfly) • Ocelli • Antenna Seeking understanding of sensory integration • Compound Eyes Owlfly and the production of • Antenna behavior. DISTRIBUTION A. Approved for public release; distribution unlimited. 8 Understand Visual Sensors and the Production of Behavior • Spectral Measurements of compound eyes and ocelli. – Equalization of intensity across the spectrum – Stimulate from 300 to 600 nm – Record electroretinogram • Field of view measurements of compound eyes and ocelli. – Using LEDs of 3 different wavelengths – Sample 16 points in space around the insect – Record electroretinogram • Flicker fusion frequency of compound eyes and ocelli. – Using LEDs of 3 different wavelengths at one point in space – Record electroretinogram • High speed recordings of free flight. – Confined to 2 x 1 x 1 meters – Indoors vs outdoors • Head rotation recordings to oscillating horizon. 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 9 Optics Train for Spectral Measurements (PIE) 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 10 Spectral Intensity Calibration Shorter wavelengths have low signal to noise ratio because the near infrared signal saturates the spectrometer Noise from spectrometer Unfiltered xenon lamp spectrum shows different intensities at different wavelengths. DISTRIBUTION A. Approved for public release; distribution unlimited. 11 Spectral Intensity Calibration Structure from xenon lamp Noise from spectrometer Combination of monochromator and photon intensity equalizer (PIE) produce a consistent intensity across the spectrum. DISTRIBUTION A. Approved for public release; distribution unlimited. 12 Spectral Measurements (Owlfly Ascaloptynx appendiculata) DISTRIBUTION A. Approved for public release; distribution unlimited. 13 Pyranometer & Pyrheliometer Data from Sunlight 10 9 8 cloud Eppley Laboratory 7 Precision Spectral Pyranometer (PSP) 6 5 psp295 psp695 4 Output(mV) nip (295 filter) 3 sunset 2 Eppley Laboratory Normal sunrise Incidence Pyrheliometer (NIP) 1 0 0:00:00 4:48:00 9:36:00 14:24:00 19:12:00 24:00:00 28:48:00 33:36:00 38:24:00 -1 Hours 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 14 LED Panel Calibration to Sunlight Measured during sunniest part of the day on rooftop sun Diffuse between diffuse = global – direct * cosine (solar zenith angle) 295 and 695 nm = 198 W/m2 Pyranometer Pyrheliometer measures global measures direct d2 d3 d1 Total energy from LED panel = (d2*d3)÷d12÷2π*diffuse Total LED panel power required to match full sunlight = 3.9 W/m2 DISTRIBUTION A. Approved for public release; distribution unlimited. 15 Scaling the Spectral Content of LEDs to Sunlight Dr. Dennis Norton AFRL/RW 374 nm peak 467 nm peak 517 nm peak 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 16 LED panel output controlled by two parameters Dr. Arunava Banerjee UF Varying PWM [0, 255] Varying I [0, 255] 0.25 0.25 0.2 0.2 0.15 0.15 0.1 0.1 millivolts millivolts millivolts millivolts 0.05 0.05 0 0 0 50 100 150 200 250 300 0 50 100 150 200 250 300 -0.05 PWM I 퐼 푃푊푀푈푉 푃푊푀퐺푅 푃푊푀퐵퐿 훼 ∗ * f , , 255 255 255 255 I = ceiling current control PWM = pulse width modulation control level DISTRIBUTION A. Approved for public release; distribution unlimited. 17 Pyranometer data taken of 8 hours of LED panel operation. Dr. Arunava Banerjee UF Pyranometer output output (mV) Pyranometer output (mV) Pyranometer Time (seconds) Time (seconds) 24.5 0.00003 Effect of ambient 24 0.000025 0.00002 temperature from HVAC 23.5 0.000015 temperature cycling on pyranometer 23 output 0.00001 PSP 295 no LEDs on 22.5 0.000005 Temperature (degrees C) (degrees Temperature 22 (mV) output Pyranometer 0 9:07:12 10:19:12 11:31:12 12:43:12 13:55:12 Time (HH:MM:SS) DISTRIBUTION A. Approved for public release; distribution unlimited. 18 Compute the resting value of each LED panel display Dr. Arunava Banerjee UF a + b (1 – exp (t/Ƭ)) Ƭ = 0 Pyranometer output output (mV) Pyranometer output (mV) Pyranometer Time (seconds) Time (seconds) Calculate scalar change for each intensity 푃푊푀 푃푊푀 푃푊푀 0.138 ∗ 푈푉+ 0.035 * 퐺푅 + 0.051 * 퐵퐿 255 255 255 PWMUV = 7 PWMGR = 109 PWMBL = 38 DISTRIBUTION A. Approved for public release; distribution unlimited. 19 LED Panel Measurements Stimulation All UV G B 8 seconds DISTRIBUTION A. Approved for public release; distribution unlimited. 20 High Speed Video Owlfly Flight DISTRIBUTION A. Approved for public release; distribution unlimited. 21 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 22 Tracking software progress David Forester Kaitlin Fair Michael (David) Richards DISTRIBUTION A. Approved for public release; distribution unlimited. 23 Head Rotation to Oscillating Horizon Jessica Thompson Celina Calma Panel field of view = 45 x 22.5 degrees Ischnura ramburii 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 24 KHILS Projector (half) Dome Jessica Thompson Celina Calma Field of view = 240 x 120 degrees Ischnura ramburii DISTRIBUTION A. Approved for public release; distribution unlimited. 25 KHILS Projector Spectral Characteristics Blue filter sky Green filter “White” ground Red filter “Black” DISTRIBUTION A. Approved for public release; distribution unlimited. 26 Damselfly Movie in KHILS Projector through City Scene Jessica Thompson Celina Calma DISTRIBUTION A. Approved for public release; distribution unlimited. 27 Damselfly Movie in KHILS Projector to Moving Horizon Jessica Thompson Celina Calma 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 28 Analysis of Head Rotation Jessica Thompson Celina Calma 8° 48° 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 29 LED panel versus WFOV stimulus and window stimulus • LED panel (16 by 32 deg at design distance) may not be wide field enough to stimulate horizon following. • LED panel cannot behaviorally attract insects unlike natural sunlight through a window. • Phenomena under investigation. DISTRIBUTION A. Approved for public release; distribution unlimited. 30 Histology Bridget Lyons Laura Nelson Whole brain section Retinal Tissue Retinal Tissue Ocellar Nerve Optic Lobe Top Row: Janus Green stained sample of entire Sarcophagid sample. Two sections of retinal tissue sections in sarcophagus fly. Bottom Row: Sarcophagid in whole mount procedure, using different fluorescing cubes for tissue identification 7 November 2014 DISTRIBUTION A. Approved for public release; distribution unlimited. 31 Compressive Sensing Motivation Kaitlin Fair DSP: Collect Compress Reconstruct Analyze/Act Sparsely CS: Reconstruct Analyze/Act Sample Sparsely Sample Analyze/Act Biology: Sufficient Information* Training Learning Predict *ML: