Supelec

EECI Graduate School in Control

Introduction to Flocking {Stochastic Matrices}

A. S. Morse

Yale University

Gif – sur - Yvette May 21, 2012

CRAIG REYNOLDS - 1987 BOIDS

The Lion King CRAIG REYNOLDS - 1987 BOIDSBOID

neighborhood

Flocking Rules

separation alignment cohesion Demetri Terzopoulos

Flocking Rules

separation alignment cohesion Motivated by simulation results reported in V i c s e k et al . si m u l a t e d a ° o c k of n ag e n t s f p a r t i c l e s g al l mo v i n g in t h e

pl a n e a t t h e sa m e sp ee d s, bu t wi t h di ® e re n t he a di n g s µ 1 ; µ 2 ; : : : ; µ n .

s = sp e e d s µ i µ i = he a di n g

Each agent’s heading is updated at the same time as the rest using a local rule based on the average of its own current heading plus the headings of its “neighbors.”

Vicsek’s simulations demonstrated that these nearest neighbor rules can cause all agents to eventually move in the same direction despite

1. the absence of a leader and/or centralized coordination

2. the fact that each agent’s set of neighbors changes with time.

Vicsek Model ri

neighbors of agent i

agent i Each agent is a neighbor of itself

Each agent has its own sensing radius ri

So neighbor relations are not symmetric HEADING UPDATE EQUATIONS

s = sp e e d s µ i µ i = he a di n g

0 N i(t) = set of indices of agent i s neighbors at time t

ni(t) = number of indices in N i(t)

Average at time t of headings of neighbors of agent i.

Another rule: Vicsek Flocking Problem: Under what conditions do all n headings converge to a common value? Convex combination {Requires collaboration!} Neighbor Graph N of Index Sets N 1, N 2 ,…., N n G = all directed graphs with vertex set V = {1,2,…,n}

N = graph in G with an arc from j to i whenever j 2 N i, i 2 {1,2,…,n} j is a ne i g h b or of i i j

A self-arced graph = any graph G with self-arcs at all vertices

1 (1,2) Ne i g h b or gr a p h s = se lf -a r c e d gr a ph s 2 3 4 5 7 6 State Space Model ‘ ’ Adjacency Matrix AG of a graph G 2 G: An n£ n matrix of 0 s and 1 s with aij = 1 whenever there is an arc in G from i to j.

_ 4 (1,2) 1 In-degree = 4, out-degree = 1 2 3 4 In-degree of vertex i = number of arcs entering vertex i 5 7 6 Out-degree of vertex i = number of arcs leaving vertex i State Space Model ‘ ’ Adjacency Matrix AG of a graph G 2 G: An n£ n matrix of 0 s and 1 s with aij = 1 whenever there is an arc in G from i to j.

Flocking Matrix FN of a neighbor graph N 2 G:

bijection

where f g D N = di a go n a l d1 ; d2 ; : : : ; dn an d Xn ni = di = in - d e g r e e of v er t e x i = aj i j = 1

Update Eqns:

State Model: Vicsek flocking problem: Under what conditions do all n headings converge to a common value?

A switched linear system

No common quadratic Lyapunov function exists

B u t t h e no n - n e ga t i v e f un c t i o n f g ¡ f g V ( µ ) = ma x µ i mi n µ i i i Verify this! is at least non-increasing along trajectories

But it takes much more to conclude that V ! 0

µ ( t + 1) = FN ( t ) µ ( t ) Vicsek flocking problem: Under what conditions do all n headings converge to a common value?

µ ( t + 1) = FN ( t ) µ ( t )

Problem reduces to determining conditions on the sequence N(0), N(1), ... under which

where

For if this is so, then

where

and so {Right} Stochastic Matrices

Sn£ n= stochastic if 1. it ha s on l y no n - n e g a t i v e en t r i e s

2. its row sums all equal 1

Stochastic matrices closed under multiplication – flocking matrices are not

Flocking matrices are stochastic

Therefore it is sufficient to determine conditions on an infinite sequence of n£ n stochastic matrices S1, S2, .... so that

This is a well studied problem in the theory of non-homogeneous Markov chains If S is a compact set of n£ n stochastic matrices whose members each have at least one positive column, then for each sequence of matrices S1, S2, … from S, and this limit is approached exponentially fast. Why is this true? Induced Norms and Semi-Norms

n£ n n£ n For M 2 R and p > 0, let ||M||p denote the induced matrix p norm on R .

We will be interested primarily in the cases p = 1, 2, 1 :

For any such p, define

1. Nonnegative: |M|p ¸ 0

2. Homogeneous: |rM|p = r|M|p

3. Triangle inequality: |M + M | · |M | + |M | 1 2 p 1 p 2 p verify!

These three properties mean that |¢ |p is a semi-norm

{If |M|p = 0 were to imply M = 0, then |¢|p would be a norm.} |M|p = 0 ; M = 0 Additional Properties of

· · 1. |M|p 1 if ||M||p 1 Because |M|p · ||M||p

M is semi - contractive in the p semi-norm if |M|p < 1

2. Sub-multiplicative: Suppose M is a subset of Rn£ n such that M1 = 1 for all M 2 M . Then

Proof : Let c0 ,c1 and c2 denote values of c which minimize ||M2M1 - 1c||p, ||M1-1c||p, and ||M2-1c||p respectively.

1 = M21 Suppose M is a subset of Rn£ n such that M1 = 1 for all M 2 M . Let p be fixed and let C be a compact set of semi - contractive matrices in M . Let

Then for each infinite sequence of matrices M1 , M2, ... in C, the matrix product converges as i ! 1 as fast as ¸ i converges to zero, to a rank one matrix of the form 1c.

Proof: See board We want to use this fact to prove that:

If S is a compact set of n£ n stochastic matrices whose members each have

at least one positive column, then for each sequence of matrices S1, S2, … from S,

and this limit is approached exponentially fast.

To do this , it is enough to show that:

A stochastic matrix S is semi-contractive in the semi-norm | ¢ |1 if S has a positive column. Any stochastic matrix S can be written as S = 1c + T where c is the largest row vector for which S - 1c is nonnegative and T = S – 1c

T1 = S1 – 1c1 = (1 - c1)1 so all row sums of T = (1 - c1) ¸ 0 because T ¸ 0

Moreover c ≠ 0 if and only if S has a positive column. verify!

Therefore (1 – c1) < 1 if and only if S has a positive column

0 jSj1 = m in jjS¡ 1d jj1 · jjS¡ 1cjj1 = jjT jj1 = ( 1¡ c1) d

A stochastic matrix S is semi-contractive in the semi-norm | ¢ |1 if S has a positive column. Transitioning from Matrices to Graphs

For a nonnegative matrix Mn£ n, °(M) is that graph whose adjacency matrix is the transpose of the matrix which results when each non-zero entries in M is replaced by a 1.

In other words, for a nonnegative matrix M, °(M) is that graph which has an arc (i, j) from i to j whenever mj,i ≠ 0. Transitioning from Matrices to Graphs

For a nonnegative matrix Mn£ n, °(M) is that graph whose adjacency matrix is the transpose of the matrix which results when each non-zero entries in M is replaced by a 1.

0 °( FN ) = °( A N ) = N

A graph is strongly rooted if at least one vertex is adjacent to every vertex in the graph

strongly rooted graph

Motivation for strongly rooted:

For any nonnegative matrix M, °(M) has an arc (i, j) whenever mj,i ≠ 0.

°(M) is strongly rooted , M has a positive column

Transitioning from Matrices to Graphs

If S is a compact set of n£ n stochastic matrices whose members each have at least one positive column, then for each sequence of matrices S1, S2, … from S, and this limit is approached exponentially fast.

If S is a compact set of n£ n stochastic matrices whose members each have a strongly rooted graph, then for each sequence of matrices S1, S2, … from S, and this limit is approached exponentially fast. Transitioning from Matrices to Graphs

When does

. . .

T Tq T2 T1

If

then

Thus establishing convergence to 1c of an infinite product of stochastic matrices boils down to determining when the graph of a product of stochastic matrices is strongly rooted. Transitioning from Matrices to Graphs

As before G = set of all directed graphs with vertex set {1,2,....,n}.

By the composition of graph G2 2 G with graph G1 2 G , written G2 ± G1, is that directed graph in G which has an arc (i, j) from i to j whenever there is an integer k such that (i, k) is an arc in G1 and (k, j) is an arc in G2.

What motivates this definition?

If A and B are nonnegative n£ n matrices and C = BA , then

Thus cji ≠ 0 if and only if for some k, bjk ≠ 0 and aki ≠ 0.

Therefore (i , j) is an arc in °(C) if and only if for some k, (i, k) is an arc in °(A) and (k, j) is an arc in °(B).

°(BA) = °(B) ± °(A) Transitioning from Matrices to Graphs

Graph composition is defined so that for any two n£ n stochastic matrices S1 and S2

°(S2S1) = °(S2) ± °(S1)

Thus deciding when a finite product of stochastic matrices has a strongly rooted graph is the same problem as deciding when a finite composition of graphs is strongly rooted. So......

When is the composition of a finite number of graphs strongly rooted? A rooted graph is any graph in G which has has at least one vertex v which, for each vertex i 2 V there is a directed path from v to i.

3 roots

When is the composition of a finite number of graphs strongly rooted?

rooted graph

Every composition of (n – 1)2 or more self-arced, rooted graphs in G is strongly rooted. Proof: See notes. The set of self-arced, rooted graphs in G is the largest set of set of self-arced graphs in G for which every sufficiently long composition is strongly rooted. Proof: See notes. Every composition of (n – 1)2 or more self-arced, rooted graphs in G is strongly rooted. Every composition of (n – 1)2 or more self-arced, rooted graphs in G is strongly rooted.

Define A (G ) = set of arcs in G

For given graphs G1, G2 2 G , G2 ± G1, is that graph in G for which (i, j) 2 A (G2 ± G1) whenever there is an integer k such that (i, k) 2 A (G1) and (k, j) 2 A (G2 ).

If G1 has a self-arc at i , then (i, i) 2 A (G1).

If G1 has a self-arc at i and (i, j) 2 A (G2) for some j, then (i, j) 2 A (G2 ± G1)

If G2 has a self-arc at j , then (j, j) 2 A (G2).

If G2 has a self-arc at j and (i, j) 2 A (G1) for some i, then (i, j) 2 A (G2 ± G1)

If G1 and G2 both have self-arcs at all vertices, then A (G2)[A(G1) ½ A (G2±G1).

In general for A (G2)[A(G1) ≠ A (G 2±G 1) even if both graphs are self-arced.

However for self-arced graphs, if there is a directed path between i and j in G 2±G 1 then there is a directed bath between i and j in G 2[ G 1 If S is a compact set of n£ n stochastic matrices whose members each have

a strongly rooted graph, then for each sequence of matrices S1, S2, … from S,

and this limit is approached exponentially fast.

. . .

strongly strongly strongly

rooted rooted rooted rooted

If S is a compact set of n£ n stochastic matrices whose members each have a self-arced, rooted graph, then for each sequence of matrices S1, S2, … from S, and this limit is approached exponentially fast.

We can generalize further still...... Repeatedly Jointly Rooted Sequences

An finite sequence of graphs G 1, G 2, ..., G p in G is jointly rooted if the composed graph G p ± G p-1 ±  ± G 1 is rooted.

An infinite sequence of graphs G1, G2, ... in G is repeatedly jointly rooted if there is a finite positive integer m for which each of the sequences G m(k -1)+1, ...... G k -1, k¸ 1, is jointly rooted.

. . .

repeatedly rooted rooted rooted jointly rooted

If S is a compact set of n£ n stochastic matrices whose members each have

a self-arced, rooted graph, then for each sequence of matrices S1, S2, … from S, and this limit is approached exponentially fast. Suppose S is a compact set of n£ n stochastic matrices whose members each have

a self-arced graph. Suppose that S1, S2, ..... is an infinite sequence of matrices from S whose corresponding sequence of graphs °(S1), °(S2), .... is repeatedly jointly rooted by sub-sequences of length m. Suppose in addition that the set of all products of m matrices from S with rooted graphs, written C(m), is closed. Then

and this limit is approached exponentially fast.

Construct an example for Compactness of S does not in general imply compactness of C(m). 2£ 2 matrices with m = 2. Exception: If S is finite and thus compact {as in flocking applications} so is C(m) Exception: If S is the set of stochastic matrices modeling the convex combo flocking rule, then S and C(m) are both compact. verify this!

Flocking Theorem: For each trajectory of the Vicsek flocking system

µ(t +1) = FN(t) µ(t) along which the sequence of neighbor graphs N(0), N(1), .... is repeatedly jointly rooted, there is a constant steady state heading µss which µ(t) approaches exponentially fast, as t ! 1 . Collectively Rooted Sequences

The flocking theorem relies on the notion of jointly rooted sequences:

An finite sequence of graphs G1, G2, ..., G p in G is jointly rooted if the composed graph Gp ± Gp-1 ±  ± G 1 is rooted.

By the union of G 1 , G 2 2 G is meant that graph G 1 [ G 2 in G with arc set A (G 1) [ A (G 2).

An finite sequence of graphs G 1, G 2, ..., G p in G is collectively rooted if the union graph G p [ G p-1 [ [ G 1 is rooted.

In general, for self-arced graphs A ( G p [ G p-1 [ [ G 1) is a strictly proper subset of A (G p ± G p-1 ±  ± G 1)

However, for each arc (i, j) 2 A (G p ± G p-1 ±  ± G 1) there must be a directed path between (i, j) in G p [ G p-1 [ [ G 1

Therefore for self-arced graphs, the sequence G 1, G 2, ..., G p in G is jointly rooted if and only if it is collectively rooted. Leader Following

Suppose that one of the agents in the group, namely agent k, ignores Vicsek’s update rule and decides instead to move with some arbitrary but fixed heading θ0.

Suppose that the remaining agents are unaware of this non-conformist’s decision and continue to follow Vicsek’s rule just as before.

Note that under these conditions, agent k must have no neighbors to follow which means that vertex k of any neighbor graph N for the group cannot have any incident arcs.

Because of this, the only possible way such a graph N could be rooted or strongly rooted would be if vertex k were the root of N and the only root of N.

All of the preceding results are applicable to this case without change.

Thus for example, all agents in the group will eventually move in the same direction as agent k if the sequence of neighbor graphs is repeatedly jointly rooted.

However more can be said in this special case Leader Following

For example, suppose that the neighbor graphs N(1), N(2), ..... are all rooted.

Then each N(t) must be rooted at k.

It was noted before that the composition of any (n -1)2 self-arced rooted graphs in G must be strongly rooted.

However in the special case of self-arced, rooted graphs in G which all have a root at the same vertex v, it takes the composition of only (n -1) of them to produce a strongly rooted graph. See notes for a proof

Because of this, one would expect faster convergence than in the leaderless case, all other things being equal. FOLLOWING RED LEADER FOLLOWING RED LEADER Leader’s Neighbors Yellow FOLLOWING RED LEADER Rectangle Pattern Leader’s Neighbors Yellow Symmetric Neighbor Relations

The original version of the flocking problem considered the case when all neighbor relations were symmetric – that is if agent i is a neighbor of agent j then agent j is a neighbor of agent i.

Mathematically, a symmetric neighbor relation means that i 2 N j , j 2 N i The corresponding neighbor graph N would thus be “symmetric” as well.

A directed graph G 2 G is symmetric if (i, j) 2 A (G) , (j, i) 2 A (G )

A rooted symmetric graph is the same thing as a “strongly connected’’ symmetric graph A graph G 2 G is strongly connected if there is a directed path between any two distinct vertices i and j. Another Way to Write the Vicsek Flocking System

L(t) is a symmetric matrix if N(t) is symmetric. Simplified Rule for Symmetric Neighbor Relations

Simplified flocking matrix:

1. Symmetric

2. Nonnegative if g > max di

3. Fs1 = 1 L1 = D1 – A1 = 0

Fs is stochastic if g > max di Comparing Flocking Matrices

Let N be a given self-arcd directed, symmetric, neighbor graph.

A = A0

°(Fs) = °(F) = N

Therefore all convergence results hold without change for the simplified flocking rule assuming symmetric neighbor relations.

Can extend the symmetric case to continuous time. Convergence Rates

First we will consider this matter in relation to the semi-norm |¢ |1 Let C be a compact set of n£ n stochastic matrices which are semi - contractive in

the infinity norm. Then for each infinite sequence of matrices S1 S2, ... in C, the matrix product converges as i ! 1 to a rank one matrix as fast as

converges to zero.

So what we’d like is a uniform upper bound on |S|1 over C. = a convergence rate bound

Any stochastic matrix S can be written as S = 1c + T where c is the largest row vector for which S - 1c is nonnegative and T = S – 1c

0 jSj1 = m in jjS¡ 1d jj1 · jjS¡ 1cjj1 = jjT jj1 = ( 1¡ c1) d

Note that the ith entry ci in c must be the smallest entry in the ith column of S.

Since (1 – c1) · 1 - ci for any i,

jSj1 · ( 1 ¡ ci ) 8i Convergence Rate Bound for Flocking Matrices with Strongly Rooted Graphs

If ci is the smallest element in the ith column of S, then |S| 1 · 1 – ci

Suppose that F is a flocking matrix whose graph is strongly rooted at vertex k

Then °(F) must have an arc from vertex k to each other vertex in the graph which means that the kth row of adjacency matrix A of °(F) must be [1 1 1  1]

-1 0 Since F = D A where D = diagonal {n1,n2, ..., nn}, the kth column of F must be

The smallest entry in this column is bounded below by

Therefore for any flocking matrix F with a strongly rooted graph

= a convergence rate bound An Explicit Formula for the Infinity Semi-Norm

For any nonnegative n£ n matrix M,

See notes of a proof of this fact. If M is a stochastic matrix, the quantity on the right is known as the coefficient of ergodicity.

For any real numbers x and y

So for a stochastic matrix S *

1. |S|1 · 1 Because |S|1 · ||S||1 = 1

2. |S|1 = 0 if and only if all rows are equal = iff S = 1c

> > For fixed i and j, the kth term in the sum in * will be positive iff sik 0 and sjk 0

Therefore the sum in will be positive iff sik > 0 and sjk > 0 for at least one value of k * Therefore |S|1 < 1 iff for each distinct i and j, sik > 0 and sjk > 0 for at least one value of k.

A stochastic matrix with this property is called a scrambling matrix Summary

A stochastic matrix S is a scrambling matrix for each distinct i and j, sik > 0 and sjk > 0 for at least one value of k.

Equivalently, a stochastic matrix is a scrambling matrix if no two rows are orthogonal.

A stochastic matrix is a semi-contraction in the infinity norm iff it is a scrambling matrix.

An explicit formula for the infinity semi-norm of any stochastic matrix S is The Graph of a Scrambling Matrix.

A stochastic matrix S is a scrambling matrix for each distinct i and j, sik > 0 and sjk > 0 for at least one value of k.

A graph G 2 G is neighbor shared if each two distinct vertices i and j have a common neighbor k

A stochastic matrix S is a scrambling matrix if and only if its graph is neighbor shared.

In a strongly rooted graph there must be a root which is the neighbor of each vertex in the graph. So......

Every strongly rooted graph is neighbor shared.

The converse is clearly false. Neighbor-Shared Directed Graph

A neighbor shared graph is a directed graph in which each pair of distinct vertices share a common neighbor

3 4

2 1

1 and 2 share 2

1 and 3 share 2 1 and 4 share 4 2 and 3 share 2 2 and 4 share 4

3 and 4 share 1 Suppose G 2 G is neighbor shared.

Then any pair of vertices (i, j) must be reachable from a {common neighbor} vertex k.

Suppose for some integer p 2 {2, 3, ..., n -1}, each subset of p vertices is reachable from a single vertex.

Let {v1, v2, ..., vp} be any any such set and let v be a vertex from which all of the vi can be reached.

Let w be any vertex not in the set {v1, v2, ..., vp}. Since G is neighbor shared, w and v can be reached from a common vertex y

Therefore every vertex in the set {v1, v2, ..., vp , w} can be reached from y.

So every subset of p + 1 vertices in the graph is reachable from a single vertex. So by induction all n vertices are reachable from a single vertex.

Every neighbor-shared graph in G is rooted.

Converse is false. Verify by constructing an example.

Strongly rooted graphs ½ neighbor shared graphs ½ rooted graphs Compositions of rooted and neighbor shared graphs

The composition on any n – 1 or more self-arced, rooted graphs in G is neighbor shared. Proof: See notes.

We will use this fact a little later to get a convergence rate bound for products of flocking matrices whose sequence of graphs is repeatedly jointly rooted.

The composition on any n – 1 or more neighbor-shared graphs in G is strongly rooted.

Let’s outline a proof of this:

Strongly rooted graphs ½ neighbor shared graphs ½ rooted graphs The composition on any n – 1 or more neighbor-shared graphs in G is strongly rooted.

A graph G 2 G is k neighbor shared if each set of k distinct vertices in G share a common neighbor.

A 2 neighbor shared graph is thus a neighbor shared graph and an n neighbor shared graph is obviously strongly rooted.

Suppose G is neighbor shared and H is k neighbor shared for some k < n

Let {v1, v2, ..., vk+1} be distinct vertices.

Since H is k neighbor shared, in H {v1, v2, ..., vk} share a common neighbor p and {v2, v3, ..., vk +1} share a common neighbor q Since G is neighbor shared, in G p and q share a common neighbor w.

In H±G , vertices v1, v2, ..., vk must have w for a neighbor as must vertices v2, v3, ..., vk +1

Therefore in H±G , vertices v1, v2, ..., vk+1 must have w for a neighbor. Therefore H±G must be k + 1 neighbor shared. Complete the proof using induction. Convergence rate bounds for products of scrambling matrices

Let C be a compact set of n£ n stochastic matrices which are semi - contractive in the infinity norm. Then for each infinite sequence of matrices S1 S2, ... in C, the matrix product converges as i ! 1 to a rank one matrix as fast as

converges to zero.

Scrambling matrices are semi-contractive in the infinity norm.

Let C be a compact set of n£ n scrambling matrices. Then for each infinite sequence of matrices S1 S2, ... in C, the matrix product converges as i ! 1 to a rank one matrix as fast as

converges to zero.

What can be said about convergence rate for scrambling matrices which are also flocking matrices? -1 0 Worst Case |F|1 for F = D A = Scrambling

°(F) = N = neighbor shared

aij = 1 if i is a neighbor of j A = [aij] aij = 0 otherwise

D = diagonal {d1, d2, …, dn}n£ n di = in-degree of vertex i

Since all di · n, all non-zero fij satisfy -1 0 Worst Case |F|1 for F = D A = Scrambling

°(F) = N = neighbor shared

aij = 1 if i is a neighbor of j A = [aij] aij = 0 otherwise

D = diagonal {d1, d2, …, dn}n£ n di = in-degree of vertex i

Since all di · n, all non-zero fij satisfy

Fix distinct i and j and let k be a shared neighbor. Then fik ≠ 0 ≠ fjk.

-1 0 Worst Case |F|1 for F = D A = Scrambling

°(F) = N = neighbor shared n ¸ 3 Vertex 1 has only itself as a neighbor Vertex 2 has every vertex as a neighbor For i > 2, vertex i has only itself and vertex 1 as neighbors

3 4

2 1

How tight is this bound? Summary Flocking Matrices with Neighbor-Shared Graphs

Every infinite product of n £ n flocking matrices with neighbor-shared graphs converges to a rank-one matrix product 1c at a rate no slower than

There exist infinite product of n £ n flocking matrices with neighbor-shared graphs which actually converge to a rank-one matrix product 1c at this rate. Convergence rates for products of stochastic matrices with rooted graphs

The composition on any n – 1 or more self-arced, rooted graphs in G is neighbor shared.

Let C be a compact set of n£ n scrambling matrices. Then for each infinite

sequence of matrices S1 S2, ... in C, the matrix product converges as i ! 1 to a rank one matrix as fast as

converges to zero.

Let S be a compact set of n£ n rooted matrices and write C for the compact set of all products of n – 1 matrices from S. Then for each infinite sequence of

matrices S1 S2, ... in S, the matrix product converges as i ! 1 to a rank one matrix as fast as

converges to zero

What can be said about the convergence rate for the product of an infinite sequence of flocking matrices whose sequence of graphs is repeatedly jointly rooted? We need a few ideas

For any nonzero matrix M ¸ 0, define Á(M) = smallest nonzero element of M.

Note that M can be written as where

For S1 and S2 n£ n stochastic matrices

By induction Recall that

Suppose S is scrambling Claim that

Since S is scrambling, for any distinct i and j there must be a k such that If S is scrambling |S|1 · 1 - Á(S) The composition on any n – 1 or more self-arced, rooted graphs in G is neighbor shared.

F = set of all n£ n flocking matrices

F(p) = { Fp Fp-1  F1: Fi 2 F , { ° (F1 ), ° (F2 ) , ... ,° (Fp) } is jointly rooted } Each matrix in F (p) is rooted F k(p) = set of all products of k matrices from F(p) Each matrix in F k(p) is scrambling if k ¸ n - 1

For any F 2 F,

If S 2 F k(p) , then S is the product of kp flocking matrices so

If k = n – 1, then S is scrambling and

Therefore a convergence rate bound for the infinite product of flocking matrices whose sequence of graphs is repeatedly jointly rooted is