SwarmSwarm Intelligence:Intelligence: aa NewNew C2C2 ParadigmParadigm withwith anan ApplicationApplication toto thethe ControlControl ofof SwarmsSwarms ofof UAVsUAVs

PaoloPaolo GaudianoGaudiano

IcosystemIcosystem CorporationCorporation Cambridge,Cambridge, MA,MA, USAUSA www.icosystem.comwww.icosystem.com OverviewOverview

• UAVs: Definition and Examples

• Complex Systems and

• Agent-Based Modeling

• ABM for the control of UAV Swarms

• Conclusions and future work

2 UAV:UAV: DefinitionDefinition

A powered, aerial vehicle that does not carry a human operator, uses aerodynamic forces to provide vehicle lift, can fly autonomously or be piloted remotely, can be expendable or recoverable, and can carry a lethal or non-lethal payload.

Source: DoD UAV Roadmap 2002

3 ManyMany TypesTypes ofof UAVUAV

4 ControllingControlling MultipleMultiple UAVsUAVs

Problem Statement:

• Current UAVs require at least one operator per UAV

• Technological advances make multi-UAV missions a near-term reality

Need control strategies that allow one operator to monitor/control multiple UAVs

5 UAVUAV SwarmsSwarms asas ComplexComplex SystemsSystems A system is complex when:

1. It consists of a large number of elements 2. Significant interactions exist between elements 3. System exhibits emergent behavior: cannot predict system behavior from analysis of individual elements

Traditional “reductionist” approaches cannot cope with complex systems

6 TheThe IcosystemIcosystem GameGame

7 TheThe IcosystemIcosystem GameGame

8 TheThe BadBad NewsNews

• Cannot predict emergent behavior from individual rules, even for such a “simple” complex system

• Individual participants are unaware of overall system behavior

• Small changes in rules lead to dramatically different emergent behaviors

9 TheThe GoodGood NewsNews

• It is possible to manipulate the behavior of a complex system by changing the rules that control individual elements

• We have developed a methodology to predict emergent behavior in complex systems using bottom-up simulation

Agent-Based Modeling!

10 SampleSample ComplexComplex SystemsSystems

11 ControllingControlling EmergentEmergent BehaviorBehavior

• How can we control ? • How do we define individual behaviors and interactions to produce desired emergent patterns?

“Here is where we think the problem is...

12 AgentAgent--basedbased modelingmodeling

• Shift viewpoint from system (centralized) to individual elements (de- centralized)

• Each agent follows local rules

• Behavior depends on interactions with other agents

• Overall system behavior emerges from local interactions

13 Example:Example: FlowFlow SimulationsSimulations • Traditional approach: mathematical description at macroscopic level. • Example: fire diffusion in airplane cabin

14 LimitationsLimitations ofof TraditionalTraditional ApproachesApproaches

• Previous simulation requires extensive computation • Any modification (e.g., number of seats, load, initial conditions) requires new computation

Compare to agent-based approach

15 AgentAgent--basedbased FlowFlow SimulationsSimulations

• The Game • • Traffic

16 SwarmSwarm ControlControl ofof UAVsUAVs Supported by Air Force Research Labs SBIR

• Create Agent-Based Model of UAV swarm

• Test various swarm control strategies for two mission types: • Search (area coverage) • Search, track and hit targets (SEAD)

• Measure performance systematically under various scenarios and conditions

17 TheThe UAVUAV AgentAgent--BasedBased ModelModel

• Rectangular search area • 3-D motion: thrust, pitch, yaw control • GPS for localization • Probabilistic ground/target sensor • Circular collision sensor • Pheromone emitter & probabilistic sensor • Communications (noisy) to central control • Stationary or moving targets

18 Simulation:Simulation: AreaArea Coverage/SearchCoverage/Search

ParameterParameter SettingsSettings PanelPanel

33-D-D ViewView AreaArea PanelPanel CoverageCoverage GridGrid

19 NavigationNavigation StrategiesStrategies

• Baseline: fly straight until border is detected, turn to stay within search area

• Random: inject small “jitter” in heading

• Repulsion: avoid UAVs within radius r

• Pheromone: avoid areas already covered (by self or others)

• Global Search: favor navigation toward unexplored sectors

(Strategies can be combined arbitrarily)

20 SampleSample CoverageCoverage PatternsPatterns

Repulsion (r=60) Pheromone

21 SystematicSystematic EvaluationEvaluation

Goal: Understand impact of strategies, parameter choices and scenarios: • 2000x2000 area, single UAV entry point • 1000-sec simulation • Swarm size (1-10, 10-110) • Navigation strategies (individual & combo) Metrics: • Area coverage • Swarm coverage efficiency • Per-UAV coverage efficiency

22 BaselineBaseline StrategyStrategy

Per-UAVPer-UAVEfficiencyEfficiency EfficiencyEfficiency ofof SwarmSwarm ofof SwarmSwarm StrategiesStrategies StrategiesStrategies

10.0%10.0% Total coverage increases 70.0%70.0% with swarm size... 9.0%9.0% 60.0%60.0%

50.0%50.0%8.0%8.0%

40.0%40.0% BaseBase 7.0%7.0% 30.0%

Coverage 30.0% Coverage 6.0%6.0% 20.0%20.0% ...but per-UAV coverage Coverage per UAV Coverage per UAV 5.0%5.0% efficiency decreases with 10.0%10.0% swarm size. 4.0%0.0%4.0%0.0% 12468101246810

UAVsUAVs 23 RandomRandom NoiseNoise StrategyStrategy

Per-UAVPer-UAVEfficiencyEfficiency Efficiency Efficiency of of Swarm Swarm of of Swarm Swarm Strategies Strategies Strategies Strategies EffectEffect ofof randomrandom jitterjitter 10.0%10.0% 70.0%70.0% isis largelylargely independentindependent 9.0%9.0% 60.0%60.0% ofof swarmswarm sizesize

50.0%50.0%8.0%8.0% 40.0% BaseBase 40.0%7.0% BaseBase 7.0% RandRand 30.0% RandRand

Coverage 30.0% Coverage 6.0%6.0% 20.0%20.0% Coverage per UAV Coverage per UAV 5.0%5.0% 10.0%10.0% 4.0% 0.0%0.0%4.0% 12468101246810 UAVsUAVsUAVsUAVs

24 PheromonePheromone StrategyStrategy

• Inspired by insect behavior

• Example of (communication through the environment)

• Each UAV lays “pheromone”

• Each UAV can sense local pheromone trace

• Navigation favors uncovered areas (Urea Strategy?)

25 PheromonePheromone StrategyStrategy ResultsResults

Per-UAVPer-UAVEfficiencyEfficiency Efficiency Efficiency of ofPheromonePheromone Swarm Swarm of of Swarm Swarm Strategies Strategies Strategies Strategies strategystrategy isis moremore effectiveeffective forfor 10.0%10.0% larger swarms 70.0%70.0% larger swarms 9.0% 60.0%60.0%9.0%

50.0%50.0%8.0%8.0% Base 40.0%40.0% Base 7.0%7.0% PherPher 30.0%30.0% Coverage Coverage 6.0%6.0% 20.0%20.0% Coverage per UAV Coverage per UAV 5.0%5.0% 10.0%10.0%

0.0%4.0%0.0%4.0% 12468101246810 UAVsUAVs

26 CombiningCombining StrategiesStrategies

EfficiencyEfficiency of of Swarm Swarm Strategies Strategies

80.0%80.0%

70.0%70.0% 60.0% 60.0% BasBas e e Rand 50.0%50.0% Rand GlobGlob 40.0%40.0% PherPher 30.0% Coverage 30.0% P+G Coverage P+G 20.0%20.0% PGJPGJ 10.0%10.0% 0.0%0.0% 12468101246810 UAVsUAVs

EvenEven aa relativelyrelatively simple,simple, decentralizeddecentralizedstrategystrategy cancan yieldyield significantsignificant improvementimprovement inin swarmswarm efficiency!efficiency!

27 ExtendingExtending toto LargeLarge SwarmsSwarms

SwarmSwarm coverage coverage efficiency efficiency

35.0%35.0%

30.0%30.0%

25.0%25.0%

20.0%20.0%

15.0%15.0% Coverage Coverage 10.0%10.0%

5.0%5.0%

0.0%0.0% 1010 30 30 50 50 70 70 90 90 110 110 UAVsUAVs

28 AdditionalAdditional Results:Results: SEADSEAD

• Allow targets to move randomly over search area

• Extend UAV behavior to track targets

• Modify simulator to carry out search and suppress missions

• Apply evolutionary computing to identify robust strategies, parameters

29 ExtendedExtended SimulatorSimulator DemoDemo

30 SampleSample SEADSEAD ResultsResults

CumulativeCumulative Hit Hit Probability Probability

100% 120%100%120% 90%90% 100%100% 80%80% 70% 80%70% 4 UAVs 80% 4 Targets4 UAVs 60%60% 48 Targets UAVs 8 Targets8 UAVs 50%60% 816 Targets UAVs 50%60% 16 Targets16 UAVs 1632 Targets UAVs 40%40% 32 UAVs % Targets Hit % Targets Hit % Targets 40% % Targets Hit % Targets Hit % Targets 40% 30%30% 20%20% 20%20% 10%10% 0% 0%0% 4816 444816 8 8 16 16 32 32

TargetsUAVsTargetsUAVs

31 FutureFuture WorkWork

• Systematic evaluation of other mission types, criteria, performance metrics

• Evolutionary design of control strategies

• Human-in-the-loop control

• Extend approach to Unmanned Ground Vehicles operating in urban scenario

• Commercialize these and other results

32