SwarmSwarm Intelligence:Intelligence: aa NewNew C2C2 ParadigmParadigm withwith anan ApplicationApplication toto thethe ControlControl ofof SwarmsSwarms ofof UAVsUAVs
PaoloPaolo GaudianoGaudiano
IcosystemIcosystem CorporationCorporation Cambridge,Cambridge, MA,MA, USAUSA www.icosystem.comwww.icosystem.com OverviewOverview
• UAVs: Definition and Examples
• Complex Systems and Swarm Intelligence
• Agent-Based Modeling
• ABM for the control of UAV Swarms
• Conclusions and future work
2 UAV:UAV: DefinitionDefinition
A powered, aerial vehicle that does not carry a human operator, uses aerodynamic forces to provide vehicle lift, can fly autonomously or be piloted remotely, can be expendable or recoverable, and can carry a lethal or non-lethal payload.
Source: DoD UAV Roadmap 2002
3 ManyMany TypesTypes ofof UAVUAV
4 ControllingControlling MultipleMultiple UAVsUAVs
Problem Statement:
• Current UAVs require at least one operator per UAV
• Technological advances make multi-UAV missions a near-term reality
Need control strategies that allow one operator to monitor/control multiple UAVs
5 UAVUAV SwarmsSwarms asas ComplexComplex SystemsSystems A system is complex when:
1. It consists of a large number of elements 2. Significant interactions exist between elements 3. System exhibits emergent behavior: cannot predict system behavior from analysis of individual elements
Traditional “reductionist” approaches cannot cope with complex systems
6 TheThe IcosystemIcosystem GameGame
7 TheThe IcosystemIcosystem GameGame
8 TheThe BadBad NewsNews
• Cannot predict emergent behavior from individual rules, even for such a “simple” complex system
• Individual participants are unaware of overall system behavior
• Small changes in rules lead to dramatically different emergent behaviors
9 TheThe GoodGood NewsNews
• It is possible to manipulate the behavior of a complex system by changing the rules that control individual elements
• We have developed a methodology to predict emergent behavior in complex systems using bottom-up simulation
Agent-Based Modeling!
10 SampleSample ComplexComplex SystemsSystems
11 ControllingControlling EmergentEmergent BehaviorBehavior
• How can we control emergence? • How do we define individual behaviors and interactions to produce desired emergent patterns?
“Here is where we think the problem is...
12 AgentAgent--basedbased modelingmodeling
• Shift viewpoint from system (centralized) to individual elements (de- centralized)
• Each agent follows local rules
• Behavior depends on interactions with other agents
• Overall system behavior emerges from local interactions
13 Example:Example: FlowFlow SimulationsSimulations • Traditional approach: mathematical description at macroscopic level. • Example: fire diffusion in airplane cabin
14 LimitationsLimitations ofof TraditionalTraditional ApproachesApproaches
• Previous simulation requires extensive computation • Any modification (e.g., number of seats, load, initial conditions) requires new computation
Compare to agent-based approach
15 AgentAgent--basedbased FlowFlow SimulationsSimulations
• The Game • Boids • Traffic
16 SwarmSwarm ControlControl ofof UAVsUAVs Supported by Air Force Research Labs SBIR
• Create Agent-Based Model of UAV swarm
• Test various swarm control strategies for two mission types: • Search (area coverage) • Search, track and hit targets (SEAD)
• Measure performance systematically under various scenarios and conditions
17 TheThe UAVUAV AgentAgent--BasedBased ModelModel
• Rectangular search area • 3-D motion: thrust, pitch, yaw control • GPS for localization • Probabilistic ground/target sensor • Circular collision sensor • Pheromone emitter & probabilistic sensor • Communications (noisy) to central control • Stationary or moving targets
18 Simulation:Simulation: AreaArea Coverage/SearchCoverage/Search
ParameterParameter SettingsSettings PanelPanel
33-D-D ViewView AreaArea PanelPanel CoverageCoverage GridGrid
19 NavigationNavigation StrategiesStrategies
• Baseline: fly straight until border is detected, turn to stay within search area
• Random: inject small “jitter” in heading
• Repulsion: avoid UAVs within radius r
• Pheromone: avoid areas already covered (by self or others)
• Global Search: favor navigation toward unexplored sectors
(Strategies can be combined arbitrarily)
20 SampleSample CoverageCoverage PatternsPatterns
Repulsion (r=60) Pheromone
21 SystematicSystematic EvaluationEvaluation
Goal: Understand impact of strategies, parameter choices and scenarios: • 2000x2000 area, single UAV entry point • 1000-sec simulation • Swarm size (1-10, 10-110) • Navigation strategies (individual & combo) Metrics: • Area coverage • Swarm coverage efficiency • Per-UAV coverage efficiency
22 BaselineBaseline StrategyStrategy
Per-UAVPer-UAVEfficiencyEfficiency EfficiencyEfficiency ofof SwarmSwarm ofof SwarmSwarm StrategiesStrategies StrategiesStrategies
10.0%10.0% Total coverage increases 70.0%70.0% with swarm size... 9.0%9.0% 60.0%60.0%
50.0%50.0%8.0%8.0%
40.0%40.0% BaseBase 7.0%7.0% 30.0%
Coverage 30.0% Coverage 6.0%6.0% 20.0%20.0% ...but per-UAV coverage Coverage per UAV Coverage per UAV 5.0%5.0% efficiency decreases with 10.0%10.0% swarm size. 4.0%0.0%4.0%0.0% 12468101246810
UAVsUAVs 23 RandomRandom NoiseNoise StrategyStrategy
Per-UAVPer-UAVEfficiencyEfficiency Efficiency Efficiency of of Swarm Swarm of of Swarm Swarm Strategies Strategies Strategies Strategies EffectEffect ofof randomrandom jitterjitter 10.0%10.0% 70.0%70.0% isis largelylargely independentindependent 9.0%9.0% 60.0%60.0% ofof swarmswarm sizesize
50.0%50.0%8.0%8.0% 40.0% BaseBase 40.0%7.0% BaseBase 7.0% RandRand 30.0% RandRand
Coverage 30.0% Coverage 6.0%6.0% 20.0%20.0% Coverage per UAV Coverage per UAV 5.0%5.0% 10.0%10.0% 4.0% 0.0%0.0%4.0% 12468101246810 UAVsUAVsUAVsUAVs
24 PheromonePheromone StrategyStrategy
• Inspired by insect behavior
• Example of stigmergy (communication through the environment)
• Each UAV lays “pheromone”
• Each UAV can sense local pheromone trace
• Navigation favors uncovered areas (Urea Strategy?)
25 PheromonePheromone StrategyStrategy ResultsResults
Per-UAVPer-UAVEfficiencyEfficiency Efficiency Efficiency of ofPheromonePheromone Swarm Swarm of of Swarm Swarm Strategies Strategies Strategies Strategies strategystrategy isis moremore effectiveeffective forfor 10.0%10.0% larger swarms 70.0%70.0% larger swarms 9.0% 60.0%60.0%9.0%
50.0%50.0%8.0%8.0% Base 40.0%40.0% Base 7.0%7.0% PherPher 30.0%30.0% Coverage Coverage 6.0%6.0% 20.0%20.0% Coverage per UAV Coverage per UAV 5.0%5.0% 10.0%10.0%
0.0%4.0%0.0%4.0% 12468101246810 UAVsUAVs
26 CombiningCombining StrategiesStrategies
EfficiencyEfficiency of of Swarm Swarm Strategies Strategies
80.0%80.0%
70.0%70.0% 60.0% 60.0% BasBas e e Rand 50.0%50.0% Rand GlobGlob 40.0%40.0% PherPher 30.0% Coverage 30.0% P+G Coverage P+G 20.0%20.0% PGJPGJ 10.0%10.0% 0.0%0.0% 12468101246810 UAVsUAVs
EvenEven aa relativelyrelatively simple,simple, decentralizeddecentralizedstrategystrategy cancan yieldyield significantsignificant improvementimprovement inin swarmswarm efficiency!efficiency!
27 ExtendingExtending toto LargeLarge SwarmsSwarms
SwarmSwarm coverage coverage efficiency efficiency
35.0%35.0%
30.0%30.0%
25.0%25.0%
20.0%20.0%
15.0%15.0% Coverage Coverage 10.0%10.0%
5.0%5.0%
0.0%0.0% 1010 30 30 50 50 70 70 90 90 110 110 UAVsUAVs
28 AdditionalAdditional Results:Results: SEADSEAD
• Allow targets to move randomly over search area
• Extend UAV behavior to track targets
• Modify simulator to carry out search and suppress missions
• Apply evolutionary computing to identify robust strategies, parameters
29 ExtendedExtended SimulatorSimulator DemoDemo
30 SampleSample SEADSEAD ResultsResults
CumulativeCumulative Hit Hit Probability Probability
100% 120%100%120% 90%90% 100%100% 80%80% 70% 80%70% 4 UAVs 80% 4 Targets4 UAVs 60%60% 48 Targets UAVs 8 Targets8 UAVs 50%60% 816 Targets UAVs 50%60% 16 Targets16 UAVs 1632 Targets UAVs 40%40% 32 UAVs % Targets Hit % Targets Hit % Targets 40% % Targets Hit % Targets Hit % Targets 40% 30%30% 20%20% 20%20% 10%10% 0% 0%0% 4816 444816 8 8 16 16 32 32
TargetsUAVsTargetsUAVs
31 FutureFuture WorkWork
• Systematic evaluation of other mission types, criteria, performance metrics
• Evolutionary design of control strategies
• Human-in-the-loop control
• Extend approach to Unmanned Ground Vehicles operating in urban scenario
• Commercialize these and other results
32