Image Acquisition and Processing of Remotely Sensed Data

Image Acquisition and Processing of Remotely Sensed Data Iowa State University Senior Design Project Final Report

Dec 08-01 Team Members Julian Currie Luis Alberto Garcia Amardeep Singh Jawandha Matt Ulrich

Client Matt Nelson ISU Space Systems and Controls Lab

Faculty Advisor Dr. John P. Basart

DISCLAIMER: This document was developed as part of the requirements of an electrical and computer engineering course at Iowa State University, Ames, Iowa. The document does not constitute a professional engineering design or a professional land surveying document. Although the information is intended to be accurate, the associated students, faculty, and Iowa State University make no claims, promises, or guarantees about the accuracy, completeness, quality, or adequacy of the information. Document users shall ensure that any such use does not violate any laws with regard to professional licensing and certification requirements. Such use includes any work resulting from this student-prepared document that is required to be under the responsible charge of a licensed engineer or surveyor. This document is copyrighted by the students who produced the document and the associated faculty advisors. No part may be reproduced without the written permission of the senior design course coordinator.

Last Updated: November 18, 2008

1 Table of Contents

List of Figures and Tables

List of Definitions

1 Project Requirements

1.1 Problem Background

1.2 Current Problem

1.3 Proposed Solution

1.4 Market Survey

1.5 System Description

1.6 Concept Sketch

2 Proposed Approach

2.1 Summary

2.2 Operating Environment

2.2.1 Hardware

2.2.2 Software

2.3 Intended Users

2.4 System Requirements

2.4.1 Functional Requirements

2.4.2 Non-Functional Requirements

2.5 Resource Requirements

2.5.1 Estimated Project Cost

2.5.2 Estimated Labor Cost

2.5.2.1 Work Breakdown and Cost

2.5.3 Scheduling

2.5.3.1 Gantt chart

2.6 Risks and Risk Management

2 2.7 Deliverables

3 Design Considerations

3.1 Theory of Operation

3.2 Pendulum Model of HABET system

3.2.1 Rate gyro

3.2.2 Accelerometer

3.2.3 Sampling Rate

3.2.4 Data Storage Space

4 Implementation

4.1 Hardware

4.1.1 Rate Gyro

4.1.2 Accelerometer

4.1.3 Microcontroller

4.1.4 Data Logging System

4.1.5 Power Budget

4.2 Software

4.2.1 Inertial Measurement System

4.2.1.1 Overview

4.2.1.2 ICSP and JTAG ICE

4.2.1.3 Analog to Digital Conversion

4.2.1.4 MMA7260Q Accelerometer

4.2.1.5 MLX90609 Gyro

4.2.2.1 Overview

4.2.2.2 Customizing the Logomatic and SD Card

4.2.2.3 Universal Asynchronous Receiver/Transmitter

5 Testing/Verification of IMU System 3 5.1 Rate gyro

5.1.1 2nd Order Temperature Calibration

5.1.2 Null-point Calibration

5.1.3 EMI Shielding

5.1.4 Output Verification using Test Platform Encoder

5.1.4.1 In Flight Condition Simulation

5.2 Accelerometer

5.2.1 Zero-point Calibration

5.2.2 EMI Shielding

5.2.3 Tilt Angle Measurements

5.2.4 Output Verification using Test Platform Encoder

5.3 Microcontroller ADC

5.3.1 Output Verification using the Logomatic

6 Conclusion

6.1 Earned Value Analysis

Project Review

References

Project Agreement

4 List of Definitions

FAA: Federal Aviation Administration. FOV: Field-of-View GPS: Global Positioning System HABET: High Altitude Balloon Experiments in Technology ImAP RSD: Image Acquisition and Processing of Remotely Sensed Data. HDS: Horizon Detection System. IMU: Inertial Measurement Unit FAT: File Allocation Table SSCL: Space Systems and Controls Lab ISGC: The Iowa Space Grant Consortium. PCB: Printed Circuit Board. PITCH: is rotation around the lateral or transverse axis. ROLL: is rotation around the longitudinal axis. WAYPOINTS: are sets of coordinates that identify a point in physical space; they usually include longitude and latitude, and sometimes altitude. YAW: is rotation about the vertical axis.

Figure 1: Roll, pitch, and yaw are exemplified. 1 Project Requirements

5 1.1 Problem Background

Currently, it is difficult to determine monitor the health of large areas of crops and to accurately predict their potential yields. This is because it is either cost prohibitive or labor intensive. Current land survey techniques either require manual inspection of the crops or the use of airplanes to conduct aerial photography of desired locations. The ImAP RSD project proposes a more efficient solution whereby an automated system is used to acquire aerial images. 1.2 Current Problem

The ISU SSCL lab requires an Inertial Measurement Unit and data logging system for the ImAP RSD project. 1.3 Proposed Solution

An IMU and data logging system are constructed. The IMU will be integrated into the ImAP RSD system by a later team which will be mounted as a payload to a high altitude weather balloon. The IMU will detect, record, and transmit six degrees of movement as well as temperature information. The final ImAP RSD system will conduct flight missions to capture images of crops at predetermined waypoints that are targeted by flight prediction software in the ISU SSCL. Acquired images will be transferred to a plant pathology team to interpret the data. 1.4 Market Survey

Complete IMUs are available in the market and cover a wide range of prices. For example, sparkfun.com offers the SEN-00839 IMU with 2 degrees of freedom for $99.95 and microstrain.com offers the Inertia- Link-2400-SK1 IMU for $2795.00. Additionally, military grade IMUs are available with even greater price tags. Buying an IMU for the ImAP RSD project is a viable solution, but doing so would defeat the purpose of a student-run senior design project.

Market research has been conducted to determine suitable system components for the IMU. Details regarding selected components can be found in the Design Report corresponding to this project. 1.5 System Description

The IMU is one of three data acquisition subsystems planned for the ImAP RSD system. Additionally, there will be a GPS unit and a horizon detection system. The GPS unit determines longitude and latitude coordinates. The horizon detection system consists of an imaging system and a thermopile system. The imaging system determines the horizon in real-time based on images acquired from the sides of the payload, while the thermopile system detects horizon based on a comparison of sky and ground temperatures. Data from these three subsystems will be used by a primary processor to control a camera pointing system. The camera will be automatically moved to aim at and acquire images of desired crops as the payload experiences constant movement during its flight mission. 1.6 Concept Sketch

6 Figure 2: Conceptual Sketch of ImAP system.

Figure 2 shows a concept sketch of the overall ImAP project. The system is shown swaying in flight with dashed lines connecting to examples of targeted crops. The subsystems discussed previously are outlined in the block diagram.

2 Proposed Approach 2.1 Operating Environment/User Interface

The IMU will operate at a range of altitudes and temperatures from 20,000 to 30,000 feet and from –40 to 85 Centigrade, respectively. 2.1.1 Hardware

The IMU operates in a Faraday cage, which consists of an acrylic enclosure lined with aluminum tape. The enclosure allows for access to an SD card with stored data, a RCA power connection, a serial port, and a LED indicating it is powered on.

2.1.2 Software

7 The code is programmed in the C language and operates on the Atmel ATMega128 microprocessor. The code converts analog data from the sensors to digital data and transmits it to a data logger as well as the central CPU through the serial port. 2.2 System Requirements 2.2.1 Functional Requirements

FR01: IMU shall measure balloon oscillation frequency and angular rotation rate to 1.215 degree per second.

FR02: IMU shall measure linear acceleration to 0.01g for each of the three principle axes.

FR03: Data logging system shall log at a 100HZ+ rate with 10 bit or greater precision.

FR04: IMU shall receive power from a 11.1 V nominal lithium-ion battery

FR05: IMU shall function for a minimum of 2 hours using a 4 Amp-hour battery pack

FR06: IMU shall operate over a temperature range of -25˚ C to +85˚ C 2.2.2 Non-Functional Requirements

NR01: IMU may measure temperature and voltage levels during flight. 2.3 Resource Requirements 2.3.1 Estimated Project Cost

Spring 2008 Budget

Item Cost

SD Card Reader $ 3.67

1 GB SD Card $ 14.99

MLX9069 Gyro $ 59.95

MMA7261QT Accelerometer $19.95

Two Break Away Headers - Straight $ 5.00

Break Away Female Headers $ 1.50

Logomatic V1 board $59.95

8 Subtotal $ 180.01

Student labor $10/Hr $ 4680.00

Total $ 4845.01

Table 1: Spring 2008 Budget

Fall 2008 Budget

Item Cost

Printed Circuit Board $ 33.00

2 MLX90609 gyros $ 119.90

Atmel Mega 128 Processor $ 9.15

Subtotal $ 162.05

Student labor $10/Hr $ 5020.00

Total $ 5182.05

Table 2: Fall 2008 Budget

2.3.2 Estimated Labor Cost 2.3.2.1 Work Breakdown

Spring 2008 Planned Work Breakdown per Team Member

Personnel Gyro and Microcontroller Gyro and Microcontroller and Operational Documentation, Total Accelerometer and Flash Accelerometer Flash Memory Manual planning & Hours Research Memory testing Testing/Programming organization Research

Luis 20 10 20 18 20 30 118

Julian 10 20 10 35 20 20 115

Matt 25 8 20 15 15 30 113

9 Amardeep 20 10 20 20 25 20 115

Total 75 48 70 88 80 100 461

Table 3: Work Breakdown Spring 2008

Fall 2008 Planned Work Breakdown per Team Member

Personnel IMU Circuit Board Gyro and Accelerometer System Integration Operational Documentation, Total Design & Testing for Calibration Manual planning & Hours Data Acquisition organization

Luis 30 25 25 25 20 125

Julian 50 7 35 20 20 132

Matt 30 35 15 20 20 120

Amardeep 40 25 10 25 25 125

Total 150 92 85 90 85 502

Table 4: Work Breakdown Fall 2008

2.3.3 Scheduling

The following chart was constructed during the Spring of 2008 and lays out the planned project schedule:

10 Figure 3: Proposed Task Schedule

2.3.3.1 Gnatt Chart

The following Gantt charts were constructed in the Spring of 2008 and specifically lay out an estimated timeline for each proposed task:

11 Figure 4: Proposed Spring 2008 Gnatt Chart

12 Figure 4: Proposed Fall 2008 Gnatt Chart

2.4 Risks and Risk Management

One of the risks involved with our IMU system involves the ambiguity of weather conditions. The flight test will be scheduled for a clear, calm day, but sudden weather variations are common in the Midwest. Even a short wind gust could drastically alter the orientation of the ImAP system, thus changing any discernible pattern in the data logged in the data logger. Any projectile or bird that hits the ImAP system could cause the balloon to burst, resulting in a badly damaged or destroyed system. Extreme temperature variations are another possibility. Some components are able to withstand a wider variation in temperature and others are not. High temperatures could increase the heat in resistors, thus decreasing efficiency. Battery failure and loose components due to faulty soldering are also possible, but these contingencies should be accounted for in our design.

13 2.5 Deliverables 2.5.1 Spring 2008 Deliverables

The following will be delivered for spring 2008:

 A comprehensive project plan

 A thorough bounded design report

2.5.2 Fall 2008 Deliverables

The following will be delivered for fall 2008:

 An operational IMU

 An accurate data logger

 Poster

 IRP presentation

3 Design Considerations 3.1 Theory of Operation

Once released into flight, it is known from past data that the HABET system ascends upward at an approximate rate of 15ft/s throughout the duration of the flight . In addition, the HABET system experiences both linear accelerations along the 3 principle axis, and angular rotations along the pitch, roll, and yaw axis. The goal of this project is assemble a 6DOF IMU that will use the linear acceleration, and angular rotations to determine the payload attitude. Our construction will include 3 rate gyros and 3 accelerometers to accomplish this task. The output of the rate gyros will be integrated to obtain inclination angles. The accelerometer outputs will give inclination angles. These inclination angles can be combined with the rate gyro theta values in an optimal manner to give us the payload attitude. Below is a simple cartoon showing this idea.

14 3.2 Pendulum Model of HABET system

The math model we have created treats the HABET system as a simple 2-D rigid pendulum. A source of experimental deviation from our model lies with the rigid assumption; in flight, the payload will not experience simple oscillatory behavior; this might be due to wind poofs distorting the cord length and complicating the motion. Below is a cartoon depicting a 2-D pendulum, and the equation of motion associated with it. Data concerning the length (l) of the cord attaching the balloon to the payload, and the mass (m) of the payload will vary from flight to flight, but are easily identifiable in advance. The strategy is to model this system on Simulink to determine what magnitude of parameters we should expect to see.

Fig . 2-D model of the HABET system, and its associated equation of motion. g=gravity, l=cord 15 length, k=damping constant, m=mass of payload. 3.2.1 Rate Gyro

Below is the rotational rate, theta_dot. The y axis is given in units of radians, and the x axis contains units of seconds. Our model correctly predicts the damping motion observed during flights. The output also indicates that the maximum rotational rate we will need to measure is below 75degress per sec. From the FFT, we see that we need to sample greater than 90Hz.

F F T o f G y r o s c o p e R o t a t i o n a l V e l o c i t y C o m p o n e n t 1 0 0 0

6 0 0 e d u t i

n 5 0 0 g a

M 4 0 0

0 - 1 - 0 . 8 - 0 . 6 - 0 . 4 - 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 N o r m a l i z e d F r e q u e n c y

Rotational rates and sampling rate obtained from math model meet functional requirements. Rate gyro used in this project, MLX90906, measures 300deg/sec, which satisfies both functional requirements and math model.

3.2.2 Accelerometer

Below is a cartoon depicting the normal and tangential accelerations experienced by a swinging pendulum.

2 leˆn 16

leˆt 2 Acceleration  leˆt  l eˆn

The figure below shows the accelerations experienced in flight. As we can see, they do not exceed 2g.

17 Below is an FFT of suggesting a sampling rate greater than F F T o f N o r m a l A c c e l e r o m e t e r C o m p o n e n t 80HZ. 4 5

d 2 5 u t i n g

a 2 0 M

0 - 1 - 0 . 8 - 0 . 6 - 0 . 4 - 0 . 2 0 0 . 2 0 . 4 0 . 6 0 . 8 1 N o r m a l i z e d F r e q u e n c y Acceleration and sampling rate obtained from math model agree with our functional requirements. The accelerometer used in this project, MMA7260Q, measures ±2g’s, which satisfies both functional requirements and math model.

3.2.3 Sampling Rate

To obtain the required sampling rate, we performed an FFT on the output of the component models. We isolated the highest frequency component and multiplied it by 2 to achieve the Nyquist rate. The FFT gave us a value of 45Hz, so we are sampling at ~100Hz.

3.2.4 Data Storage Space

Below is the calculation used to determine the storage space needed on the SD card to record all of the data throughout the flight.

18 1Baud=symbols/sec=10 bits/s

Using 19200 Baud: 19200 bits * (10/s*120 seconds) = ~23 MB of storage space.

4 Implementation 4.1 Hardware 4.1.1 Rate Gyro The rate gyro measures the angular velocity about the axis the object is rotating. We are using 3 single axis rate gyros which would measure yaw pitch and roll. The rate gyro is a MEMS device which calculates the angular rate by sensing Coriolis force and change in capacitance. In our project we are using MLX90609 which measures plus or minus 300 degrees/s 4.1.2 Accelerometer The accelerometer measures acceleration in specific direction depending on the device. We are using a 3 axis accelerometer which measures acceleration in the X, Y and Z axis. The accelerometer is a MEMS device which calculates the acceleration by sensing change in capacitance. In our project we are using MMA7260Q which has four selectable sensitivities. This project will use the 2g option.

4.1.3 Microcontroller The microcontroller is used to do the analog to digital conversion and transmit data at a specific sampling rate. We are using ATmega128 for our project

4.1.4 Data Logging System We need a data logging system to do post flight analysis.

4.1.5 Power Budget The power budget table for the IMU system can be found in the appendix. 4.2 Software The code will be written in AVR Studio using the C programming language. BCD conversions will allow each digit to be sent as an 8 bit character over the USART 4.2.1 Inertial Measurement System

4.2.1.1 Overview The Atmel Mega 128-16AU will be programmed using the C programming language. All software development will take place in AVR Studio 4, an Integrated Development Environment created by the Atmel Corporation. 4.2.1.2 ISP and JTAG ICE

19 Communication between the microcontroller and a PC will take place using an ISP (In-System Programmer). The JTAG ICE (Joint Test Action Group In-Circuit Emulator) is the protocol used for on-chip debugging of the microcontroller via AVR studio. Figure (1) shows how the ISP connects from the microcontroller to the PC.

4.2.1.3 Analog to Digital Conversion The ADXL330 and the MLX90609 output analog signals, so A/D conversion is needed to store meaningful data. Analog to digital conversion will take place in the Atmel Mega 128.

4.2.1.4 MMA7260Q Accelerometer The resolution of the accelerometer was obtained as follows:

 Range is 2g or 4g total

 Need to measure accuracy within .01g

 Sensitivity: .3V/g

 Vref=2.56V for microprocessor

 10 bit digital Encoding

o ADC range:

. Total voltage swing=7.2g*.3V/g=2.16V

. (2.16 /2.56)*1024=864 counts

. Resolution:7.2g/864 =.00833 g/count

4.2.1.5 MLX90609 Gyro The resolution of the gyro was obtained as follows:

 Range is 150° / sec

 Sensitivity: 13.33mV /(° / sec)

 Vref=2.56 for microprocessor

 10 bit digital encoding:

o ADC range:

. Total voltage swing=300 ° / sec *.01333V /(° / sec) =4V

. (4/5)*1024=819.2 counts 20 . Resolution: (1000 ° / sec )/819.2=1.2207 ° / sec/ count

4.2.2.1 Overview IMU data will be recorded using a 1GB SD card. The Card will interface with the Atmel Mega 128 via Sparkfun’s Logomatic V1.0 Universal Data Logging Device. The Board comes with a substantial amount of FAT code that is free to use and alter for this project.

4.2.2.2 Customizing the Logomatic and SD Card The logomatic board comes equipped with a microprocessor that converts incoming data into the correct format that the SD card can utilize. The fact that all the FAT code is included on the microprocessors saves the group an enormous amount of time. Learning the intricacies of the FAT system is a task that would have prevented us from reaching our end goal. Powering up the Logomatic with the SD card for the first time causes a configuration file to be written to the SD card. This file allows the board to read serial data directly from the Atmel Mega 128’s USART or from its own 10 bit ADC on the microprocessor. The file has the following configurable options:

1. Mode=0 for the Automatic UART logging mode,1 for the Triggered UART mode, or 2 for ADC mode 2. ASCII = Y or N (specifies whether the unit will log in ASCII format or binary format respectively, only used with mode 2) 3. Baud =1-8 (each number represents a standard baud rate as listed in the datasheet) 4. Frequency (sampling rate for the Logomatic’s ADC) 5. Trigger Character (can be any character, for mode 1) 6. Text Frame (# of characters to be logged after trigger character for mode 1) 7. Operational ADC lines = Y or N (can initialize as many as needed) 8. Safety On = Y or N (sets the frequency caps for the ADC mode)

Since our system will run in mode 1, option #2, #4, #7, and #8 will not be utilized. The baud rate will be set to 115200 Baud. The text frame will be set to allow all 7 ADC inputs to be sampled virtually at the same time. The trigger character will help us discriminate between sampling intervals.

4.2.2.3 Utilizing the UART The Logging system will be operating in the Triggered UART Mode. The triggered UART mode allows a set number of characters to be logged after a trigger character is set. Each frame is delimited with a carriage return and a line feed character. The baud rate can be set to match that of the Atmel Mega 128, which is essential for effective communication. Figure (2) shows a state machine of the Logging device.

4.2.3 SPI Interface

21 The Atmel Mega 128 will send data through an SPI Interface to the SD card (?) and the onboard computer. Configuration and operation of the SPI serial interface peripherals will utilize the appropriate software routines. Testing/Verification of IMU System 5 Rate Gyro

5.1.1 2nd Order Temperature Calibration

A temperature sweep was conducted in an air-tight air chamber. The change of voltage values from the rate gyro’s was then measured. Please see appendix for additional details. 5.1.2 Null-point Calibration

The null-point calibration is done pre-flight when we install the IMU in the payload. It calculates the zero point as the reference for rotation. See appendix for additional details. 5.1.3 EMI Shielding

The payload has a transmitter on board and it transmits signals back and forth to the ground station. Thus the parts need to be shielded from the Electromagnetic Interference (EMI) or Magnetic radiations. When we induced the system with EMI radiation the rate gyro output had significant error. Thus we decided to shield our case with aluminum tape. After shielding the system, we repeated the system and saw no errors induced with EMI radiation. The IMU casing can be found as a CAD drawing in the appendix, it is covered with aluminum tape to shield it from the EMI. 5.1.4 Output Verification using Test Platform Encoder

We did static testing using the test platform at the component level by integrating the output once to get the angular velocity. We then compared this value to the test platform. 5.2 Accelerometer

5.2.1 Zero-point Calibration

Same as the rate gyro, please see the appendix for more information. 5.2.2 EMI Shielding

Same as the rate gyro, please see the appendix for more information. 5.2.3 Tilt Angle Measurements 22 We did static testing using the test platform at the component level. We used the test platform encoder as an absolute angle reference and compared the tilt angle value the accelerometers were giving us. Please see the appendix for additional details. 5.3 Microcontroller ADC

The sensors will go through an 8 channel, 10-bit ADC and be converted to a 10 bit binary number. This number will be sent through the Atmel’s UART in Binary Coded Decimal Format, which will yield a 4 digit number ranging from 0-1023 The ADC clock frequency is 125KHz. 5.3.1 Output Verification using the Logomatic 6 Conclusion

6.1 Earned Value Analysis

Spring 2008

Budgeted Actual Tasks Hours Hours BCWS BCWP ACWP

IMU Research 75 72 $750.00 $750.00 $720.00

MCU Research 48 42 $480.00 $455.00 $420.00

Sensor Testing 70 75 $700.00 $684.00 $750.00

Programming/SW Debugging 88 760 $880.00 $810.00 $760.00

Documentation 100 109 $1,000.00 $984.00 $1,090.00

Subtotal $3,810.00 $3,683.00 $3,740.00

Fall 2008

Budgeted Actual Tasks Hours Hours BCWS BCWP ACWP

IMU Design 150 173 $1,500.00 $1,430.00 $1,730.00

Testing/Data Acquisition 92 123 $920.00 $850.00 $1,230.00

Sensor Calibration85 46 $850.00 $810.00 $460.00 23 System Integration 90 104 $900.00 $850.00 $1,040.00

Operation Manual 85 60 $850.00 $815.00 $820.00

Subtotal $5,020.00 $4,755.00 $5,280.00

Total $8,830.00 $8,438.00 $9,020.00

Behind Schedule Variance BCWP-BCWS -$392 Schedule

Cost Variance BCWP-ACWP -$582 Over Budget

Cost Performance Index BCWP/ACWP 0.935476718

Schedule Performance Index BCWP/BCWS 0.955605889

The group ended up spending more time on our task than anticipated which accounts for us being behind schedule and over budget. References

Dynamics of Flight, Stability and Control; B. Etkin, L. Reid. John Wiley and Sons, 1996 Aurzkai et al. ImAP Fall 2007 Currie et al. ImAP Spring 2008, Design review

25 Appendix

Rate Gyro Testing/Calibration V  V  (T  T )*a  ωb  noise • Calibration: tot 0 0

V To calculate the null point 0 , the rate gyro was put in a temperature chamber and the temperature was set such that T = To. In addition to that there was no angular rate induced therefore we got the null point. The null point calibration will be done before the payload is airborne each time.

• EMI effects: Electromagnetic interference degrades or obstructs the performance of the circuit and therefore we shielded the box with aluminum tape.

• Output verification/Component testing using test platform:

Encoder test platform à  , we differentiate this and compare values from the rate gyro

Rate gyro à angular rate

 Temperature Drift: Rate gyros are susceptible to temperature drift and therefore we did temperature sweep and obtained the scale factor.

Accelerometer Testing/Calibration

• Calibration:

• EMI Shielding: Electromagnetic interference degrades or obstructs the performance of the circuit. V • Tilt measurement using test platform: V  V  ( *1g *sin ) out offset g

Vout = Output of Accelerometer

26 Voffset = 0g offset of Accelerometer

1g = Earths Gravity

 = Angle of tilt

Test Platform: The following simulation is to verify we can use it test the rate gyro.

27 V V   arcsin( out offset  V g

28 Signature Page

______Matt Nelson; SSCL, Client Date

______Dr. John P. Basart, Faculty Advisor Date

______Luis Alberto Garcia, Project Team Leader Date

______Julian Currie, Project Communications Coordinator Date

______Matt Ulrich, Project Team Member Date

______Amardeep Singh Jawanda, Project Team Member Date