Acoustics and Electro- Optics Team
Wojciech Andreas Qing Li Ariel Marrero Czerwonka Graber
Talmor Meir Saiyam Shah Walter Seme Outline • Introduction/Background ! Objectives ! Review of Available Technologies • Methodology ! Passive Acoustics " Hardware " Software ! Infrared (IR) • Results ! Analysis during two satellite passes ! Ambient noise ! Samples of Acoustic Signatures ! IR day and night comparison ! Visible light vs. IR light at night • Summary/Conclusion ! Acoustic and IR applicability ! Data fusion ! Ship classification Objectives • To determine the applicability of acoustics and electro-optic technologies in detecting, classifying and tracking small vessels in the Hudson Estuary.
• Review available detection technology for maritime security application.
• Conduct research supporting development of passive acoustic detection, classification and tracking method: ! Acoustic reconnaissance of estuary ! Collection of acoustic signatures from different boats ! Estimate the intensity of interference from ambient noises
• Fusion of satellite acoustic and electro-optic data
• Describe the limitations of acoustics and electro-optic technologies in the maritime security domain. Background: Range of Technologies for Port Security ! Sound Navigation And Ranging (SONAR) " Passive " Active ! Infrared (IR) Detection ! Light Detection And Ranging (LIDAR) ! Radio Detection And Ranging (RADAR) ! Video Cameras ! Light-Emitting Diode (LED) Sound Navigation And Ranging (SONAR)
A system that uses transmitted and reflected sound wave to detect objects .
Active VS. Passive Passive Sonar • Sonar that uses only underwater listening equipment, with no transmission of location-revealing pulses.
• How it works? • Passive sonar listens for the sounds coming from other ships and marine objects.
• Ex. When a submarine uses passive sonar, it is able to obtain information about other ships and submarines without revealing its own position. Skilled sonar operators can determine such things as ship speed, number of propellers and even the exact kind of ship just by listening to the sounds.
Analyzing Data with Spectrogram SENTRi™ Maritime Acoustic Surveillance System ! Applications: "Military (aircraft, torpedoes, mines) "Vessel activity (threats detection) "Diver and UUV detection
!Benefits: " Minimal power requirement. " No impact on marine life " Doesn’t reveal position “stealth” SENTRi™ acts as a guardian, listening 24-7 " Clandestine surveillance. for the presence of such signatures and on reception sends configurable alerts by SMS or Email to the shore side user. !Drawbacks: Applications: " No range information. Two or • Warning of craft in unauthorized areas more systems are required for • Detection and monitoring of pirate target localization. activities " Mini-submersibles and divers • Monitoring of drug trafficking with closed circuit system are • very quiet with respect to ambient Detection of illegal immigration activities noise. • Monitoring of wreck sites Active SONAR ! Active SONAR creates a pulse of sound (ping) and then listens for reflections (echo) of the pulse.
! Active SONAR uses a sound transmitter and a receiver. o When the two are in the same place it is a monostatic operation. o When the transmitter and receiver are separated it is a bistatic operation. o When more transmitters (or more receivers) are used, again spatially separated, it is a multistatic operation. Active Sonar cont… Pros Cons
" Can actively look for targets " Varying impacts to marine life " Range information " Does reveal position “no " Closed circuit systems cannot stealth” elude " Requires more power than " Can detect multiple targets passive SONAR (vessel threats, mines, " Hard to deal with surface torpedoes, divers, UUVs) reflections Radio Detection and !An object detection system that uses electromagnetic waves to identify the range, altitude, direction, or speed of both moving and fixed objects !The range of the object is determined by measuring the time delay of emission and reflection of radar signal !The velocity of the object is measured by applying the Doppler principle Radar Application in Maritime Security
NATO The Honeywell Radar Video Radar Band Surveillance (RVS) system A waterside radar system that provides situational awareness of the waterways in the Vessel general areas surrounding the Port of Miami. Detection Marine Radar – Long range detection of vessels
Automatic Identification System IEEE Radar (AIS) – Transponder based Band detection
Ground Radar (Short Range) - up to 300 m.
River Sonde Ground Radar (Long Range) –up to 5 km
Video Analytics 0GHz HF Sea Sonde –behind buildings and other Source: "Some Radar application and frequency band." Radar Basics. Web. 29 Jun 2010.
- Could be used as Tripwire. - Could be used to track the vessel and its distance. - Could be used to highlight more sensitive areas.
Pros Cons " Very Small ( ~ 2 mm) " Extreme voltage sensitivity " 35,000 to 50,000 hrs life time " Temperature dependence " Very Small ( ~ 1 µsec ) turn on/off " Blue hazard: time " Emits in all directions Video Cameras - Closed-circuit television (CCTV) is the use of video cameras to transmit a signal to a specific place, it is displayed on monitors.
Pros: • We can adopt cameras to be used is almost any environment • Video Camera can be used night and day 24/7. • A picture is worth a thousand words. • It’s a well know and developed technology Security Camera Cons: Can not see well underwater Dependant on electricity LiDAR: Light Detection And ‘Active Sensor’ that uses light pulses to determine distance to object. - 150,000 pulses /second - up to 5 returns/ pulse
NOAA Coastal Services Center. 2008. “Lidar 101: An Introduction Lidar Technology, Data, and Applications.” Charleston, SC: NOAA Coastal Services Center HDL-64E LIDAR- by
64 laser 3D (360x26.8 degree)
Pros: Cons: -high resolution [15 cm] -affected by radar pulses -3D world -Enormous amount of data -Can be applied for night -Environmental constraints - no returns from water Infrared – “Below Red light” • Part of electromagnetic spectrum • Thermal radiation is generated by vibration and rotation of atoms and molecules • At terrestrial temperatures, an object radiates at longer wavelengths of the infrared band
Orlove, Gary & Madding, Robert. Answers to Common Questions People Ask about Infrared Thermography. Infrared Training Center, 2003. pg 1-8 IR Application • Surveillance and remote detection of small vessels • Can be utilized in other ways: ! Search and rescue ! Check for defective equipment
Electro- Optics in Maritime Security by Dr. Barry Bunin Pros Cons • Great for finding • Not good for facial and detecting recognition objects in darkness • Expensive or bad weather • Can be countered: • Easy to apply –Blinding • objects emit –Decoys thermal radiation –Concealment without being (Shrouding) illuminated New York City – urban estuarine environment
" Ambient Noise " wind-induced surface noise " rainfall on the surface " urban noise from immediate surroundings. Available Resources: Harbor conditions NYHOPS- New York Harbor Observation and Prediction System
Provides meteorological and oceanographic conditions both in real- time and forecasted out to 48 hours.
water level, temperature, salinity, currents, NOAA winds (~12 km resolution), Wave height, period and direction Available Resources: Atmospheric
COAMPSCondition- coupled Ocean Weather Underground Atmospheric Mesoscale Prediction System
Generates three-dimensional 24 For validation purposes only hr weather forecasts on a ~ 1 km resolution grid covering the metropolitan New York City region. Methodology Acoustic Methodology - Hydrophone
Hydrophone •A microphone used underwater •Recording or listening to sound
Omni-directional hydrophone •A pressure sensor designed to sense acoustic pressure •Equally in every direction Acoustic Methodology – System Deployment
July 12th, 2010 14:36GMT • Underwater computer --approximately 300 ft away from Griffith building •Four omni-directional hydrophones were released from the Savitsky and positioned near the underwater computer area. •Exact locations were acquired with a special calibration procedure presented at the next slide Acoustic Methodology – System Calibration
15:15:32GMT-18:26:00 GMT •A transmitter radiates reference wide band sound •Savitsky runs two full loops around the Acoustic system and recorded GPS position at any points. •Calculation allows accurate funding hydrophone positions. Acoustic Methodology – Test Path for Savitsky
North-South
Test Path #1 Fast Loop 19:08:24 GMT 11kn/6.5kn
Test Path #2 West-East Medium Loop 19:49:40 GMT Test Path #3 9.6kn/5.0kn Normal speed @7.2kn 20:02:15 GMT
Test Path #4 Fast speed @9.3kn 20:10:49 GMT Acoustic Methodology – Maritime Security Lab Architecture
Source: Sutin, A. B., B (2009). Acoustic research for port protection at the Stevens Maritime Security Laboratory. 3rd International Conference and Exhibition on Underwater Acoustic Measurements: Technologies & Results. Nafplion, Greece. Acoustic Methodology- • A visual representation that shows how the spectral density of a signal varies with time.
Acoustic Signal (time domain)
Section into contiguous time frames. “windows”
Fast Fourier Time Frequency Transform domain domain
Signal Level (dB)
Signal spectrum characteristics Freq.(Hz) Time(sec) Acoustic Methodology - Cross Correlation • A measure of similarity of two waveforms as a function of a time-lag applied to one of them. • Application in detection and direction finding of targets Time delay between 2 Linear Cross Correlation Intensity hydrophones North
South
Time of Events Acoustic Methodology - Cross Correlation Circular Cross Correlation- Two diagrams are based upon target’s location ; the bending of the curve indicates changing bearing to the target
North
Imaginary target East Real target Imaginary target Real target West
South Hydrophone #0, #2 Hydrophone #1, #3 Acoustic Methodology- DEMON Processing
• DEMON - Detection of Envelope Modulation on Noise • Detects slowly varying amplitude of boat noise • A valuable tool in classifying aspects of a particular ship • The characteristics of the DEMON signal depends on ship’s performance (speed/propulsion type/tonnage)
Raw Spectrum Averaged Spectrum DEMON Spectrum Acoustic Methodology- Log Index to Retrieve the Data Duration:7/12/2010 Salinity/Water Temp/ Approx. 950 boats – 7/15/2010 Wind Velocity/ Direction/Rain were recorded Objectives • Use of Infrared Cameras for detection, classification, and tracking of small vessels • Determine advantages and limitations of infrared imaging • “How small is too small?” Infrared Imaging Introduction • Electro-optics - Discipline of electronics and optics used together ! Infrared imaging is one area in electro-optics • Infrared is part of Electromagnetic spectrum ! “Below Red light” • At terrestrial temperatures, an object will radiate at longer wavelengths of the infrared band Infrared Spectrum Infrared Equipment LWIR Camera ! Indigo System’s Merlin Uncooled Microbolometer ! 7000 - 14000 nm wavelength ! Long wave is generated by thermal radiation
NIR/SWIR Camera ! SU320KTSX-1.7RT ! 900 - 1700 nm wavelength ! Does not detect thermal radiation ! Detects infrared light in specified wavelength Set up CCTV • 6th floor Babbio Center patio
NIR/SWIR
LWIR Set up •Recorded during day and night •Satellite passes •Kept cameras at a fixed, stationary point
Date (MM/DD/YY) Time Began (GMT) Time Finished (GMT) 07/12/10 (Monday) 17:32:58 03:03:51 (07/13/10) 07/13/10 10:12:48 16:24:14 (Tuesday) 07/15/10 (Friday) 19:25:46 01:59:04 (07/16/10) 07/27/10 02:13:29 02:35:46 (Tuesday) Results Samples of Results and Analysis
Time Time Date Label Data Start End
TerraSAR-X 7/13/2010 11:07:51 11:13:27 11:13:21 Satelite Passes
Cosmo 7/13/2010 22:43:40 22:49:16 22:46:57 Our 7/13/2010 16:30:55 16:35:11 Rain Ambient Noise Objectives 7/14/2010 21:06:56 21:11:12 strong Wind 7/13/2010 1:39:02 small boat
7/13/2010 1:49:23 fast boat Vessel Samples 7/13/2010 15:07:20 DEP- large ship 7/13/2010 18:19:35 speed boat Satellite Pass On 7/13/2010 11:13:21
TerraSAR-X Pass on 7/13/2010 11:13:21 (GMT) Data Log: TerraSAR-X July13th, 2010, 11:06-11:13 GMT
Weather System Action Observation on Date Time(GMT) Time(ET) Visible Description Conditions Taking Display
7/13/2010 11:08:31 7:08:31 Ferry N to S
Sav is turning Salinity:13.24 to North to fix Ferry moving from S 7/13/2010 11:10:02 7:10:02 psu; its location to N Water Temp: (current drift) 74.32K; strong noise from 7/13/2010 11:10:10 7:10:10 Wind Velocity: Sav 5.65 kn 7/13/2010 11:10:54 7:10:54 Wind Ferry from N to S Direction: 136.1 Satellite 7/13/2010 11:13:06 7:13:06 PASS. Sav drifting near array
Ferry moving North to South. Ferry moving South to North.
Savitsky shifting its position to North. Acoustic Reconnaissance: TerraSAR-X
Linear Cross Correlation Savitsky is Dead- In-the-Water and Ferry from North then begins its to South testing moving in SE direction. Ferry from South to Savitsky moving North north to readjust its location. High Intensity coming Additional from the boat’s Spectrogram noise from engine South (entry to Harbor) Acoustic Reconnaissance cont… Circular Cross Correlation
Savitsky moving north to readjust its location.
Ferry moving from South to North
Infrared Images During Satellite Pass 11:13:21 Satellite Pass On 7/13/2010 22:46:57
Cosmo Sky-Med-2 on 7/13/2010 22:46:57 GMT Data Log: Cosmo Sky-Med-2
On July 13th, 2010, 22:45-22:53 GMT
Weather System Action Observation on Date Time(GMT) Time(ET) Conditions Taking Visible Description Display Salinity: 12.8 psu; A sail boat very near 7/13/2010 22:45:01 18:45:01 Water Temp: Manhattan from 75.33K; North to South.
Wind Velocity: 5.09 Cosmo Sky- 7/13/2010 22:46:57 18:46:57 kn Med-2 Satellite Wind pass
Sailboat moving from North to South. Acoustic Reconnaissance: Cosmo Sky- NY Waterway from South- Med-2Linear Cross Correlation North which stopped in
Noise from North
Sailboat with sails down, from North to South.
Spectrogram Spectrogram Acoustic Reconnaissance cont…
Circular Cross Correlation
Sailboat moving from North to South. Ambient Noise Ambient Noise Analysis for different environmental conditions: Strong rain, July 13, 16:34
30 dB Strong wind (14 Kn.), July 14 21:10 Small Boat 7/13/2010 01:39:02 GMT
Small Boat 7/13/2010 01:39:02 GMT
DEMON Spectrogram Fast Boat 7/13/2010 01:49:23 GMT Speed Boat 07/13/2010 18:19:35 GMT Speed Boat 07/13/2010 18:19:35 GMT DEP 07/13/2010 15:07:20 GMT
DEMON Spectra Electro-Optic Imaging Infrared Imaging Infrared Observations •Water looks like coffee •Very limited range underwater
NIR/SWIR •Determine direction of current •Affected by poor weather
LWIR •Vessel outline stands out •No external light source needed, passive detection •Less affected by weather than NIR/SWIR Infrared vs. Visible at Night •Easier to detect vessels by using infrared cameras
•NIR/SWIR has glare from running lights and NYC
•So does visible light
•LWIR does not have glare
•Wake can be used for tracking Infrared Results • Good detection tool for day and night • Not useful for underwater imaging • Possible to determine features of vessels ! Ex: engine, sails, hot surfaces, and wakes • LWIR + lens = estimated $50k • Applications ! Surveillance ! remote detection ! Track vessels by wake ! Search and rescue ! Check for defective equipment How small is too small? • Ambiguous question – depends on equipment • People and row boats can be found • But vessels can be made to be invisible to thermal cameras ! Concealment ! Blinding Summary !Acoustic and IR applicability vs. limitations !Data fusion !Ship classification !Ambient noise Questions?