Chordal Structure and Polynomial Systems

Chordal structure and polynomial systems

Diego Cifuentes

Laboratory for Information and Decision Systems Electrical Engineering and Computer Science Massachusetts Institute of Technology

Joint work with Pablo A. Parrilo (MIT)

UC Davis - November 2014

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 1 / 23 What it is solving? Decide if it is consistent. Find a solution. Describe all solutions. Find a Gr¨obner basis.

Polynomial ideals

Consider a system of m polynomial equations in n variables:

fi (x0,..., xn−1) = 0, i = 1,..., m

The objective is to “solve” these equations.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 2 / 23 Polynomial ideals

Consider a system of m polynomial equations in n variables:

fi (x0,..., xn−1) = 0, i = 1,..., m

The objective is to “solve” these equations.

What it is solving? Decide if it is consistent. Find a solution. Describe all solutions. Find a Gr¨obner basis.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 2 / 23 2 2 I = {(x0 x1x2+2x1+1) g1+(x1 +x2) g2+(x1+x2)g3+(x2x3)g4 : g1, g2, g3, g3}

2 I = {(x0 − 3) g1 + (x1 − 1) g2 + (x2 + 1)g3 + (x3)g4 : g1, g2, g3, g3}

Gr¨obner basis: 2 I = hx0 − 3, x1 − 1, x2 + 1, x3i

There are two solutions: √ √ V(I) = ( 3, 1, −1, 0), (− 3, 1, −1, 0)

Polynomial ideals

Example: 2 2 I = hx0 x1x2 + 2x1 + 1, x1 + x2, x1 + x2, x2x3i

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 3 / 23 2 I = {(x0 − 3) g1 + (x1 − 1) g2 + (x2 + 1)g3 + (x3)g4 : g1, g2, g3, g3}

Gr¨obner basis: 2 I = hx0 − 3, x1 − 1, x2 + 1, x3i

There are two solutions: √ √ V(I) = ( 3, 1, −1, 0), (− 3, 1, −1, 0)

Polynomial ideals

Example: 2 2 I = hx0 x1x2 + 2x1 + 1, x1 + x2, x1 + x2, x2x3i

2 2 I = {(x0 x1x2+2x1+1) g1+(x1 +x2) g2+(x1+x2)g3+(x2x3)g4 : g1, g2, g3, g3}

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 3 / 23 Gr¨obner basis: 2 I = hx0 − 3, x1 − 1, x2 + 1, x3i

There are two solutions: √ √ V(I) = ( 3, 1, −1, 0), (− 3, 1, −1, 0)

Polynomial ideals

Example: 2 2 I = hx0 x1x2 + 2x1 + 1, x1 + x2, x1 + x2, x2x3i

2 2 I = {(x0 x1x2+2x1+1) g1+(x1 +x2) g2+(x1+x2)g3+(x2x3)g4 : g1, g2, g3, g3}

2 I = {(x0 − 3) g1 + (x1 − 1) g2 + (x2 + 1)g3 + (x3)g4 : g1, g2, g3, g3}

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 3 / 23 Polynomial ideals

Example: 2 2 I = hx0 x1x2 + 2x1 + 1, x1 + x2, x1 + x2, x2x3i

2 2 I = {(x0 x1x2+2x1+1) g1+(x1 +x2) g2+(x1+x2)g3+(x2x3)g4 : g1, g2, g3, g3}

2 I = {(x0 − 3) g1 + (x1 − 1) g2 + (x2 + 1)g3 + (x3)g4 : g1, g2, g3, g3}

Gr¨obner basis: 2 I = hx0 − 3, x1 − 1, x2 + 1, x3i

There are two solutions: √ √ V(I) = ( 3, 1, −1, 0), (− 3, 1, −1, 0)

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 3 / 23 Gr¨obner basis:

I = hx0 − x8, x1 − x8, x2 − x8, x3 + x8 + x9, x4 + x8 + x9, 2 2 3 x5 − x9, x6 + x8 + x9, x7 − x9, x8 + x8x9 + x9 , x9 − 1i

There are six solutions: three choices for x9, two choices for x8.

Polynomial ideals

Example 2: 0 Let I be given by the equations: 6 7 3 xi − 1 = 0, 0 ≤ i ≤ 9 2 2 9 8 3 xi + xi xj + xj = 0, (i, j) edge

1 4 5 2

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 4 / 23 Polynomial ideals

Example 2: 0 Let I be given by the equations: 6 7 3 xi − 1 = 0, 0 ≤ i ≤ 9 2 2 9 8 3 xi + xi xj + xj = 0, (i, j) edge

1 4 5 2 Gr¨obner basis:

I = hx0 − x8, x1 − x8, x2 − x8, x3 + x8 + x9, x4 + x8 + x9, 2 2 3 x5 − x9, x6 + x8 + x9, x7 − x9, x8 + x8x9 + x9 , x9 − 1i

There are six solutions: three choices for x9, two choices for x8.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 4 / 23 Finding a solution to a system of quadratic equations is NP-hard. Computing Gr¨obner bases may require (doubly) exponential time.

Grobner¨ bases

Given an ordering of the variables x0 > x1 > . . . > xn−1 there is a unique reduced lex Gröbner basis. The system is inconsistent iff the reduced Gröbner basis is h1i. If the system has finite solutions, we can find them recursively: solve a univariate polynomial in xn−1, for each solution xˆn−1, solve a univariate polynomial in xn−2, etc. For an arbitrary ideal I, we can get the elimination ideals

eliml (I) = I ∩ K[xl , xl+1,..., xn−1]

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 5 / 23 Grobner¨ bases

eliml (I) = I ∩ K[xl , xl+1,..., xn−1]

Finding a solution to a system of quadratic equations is NP-hard. Computing Gr¨obner bases may require (doubly) exponential time.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 5 / 23 Example:

2 2 I = hx0 x1x2 + 2x1 + 1, x1 + x2, x1 + x2, x2x3i

Polynomial systems and graphs

A polynomial system deﬁned by m equations in n variables:

fi (x0,..., xn−1) = 0, i = 1,..., m

Construct a graph G (“primal graph”) with n nodes, as:

Nodes are variables {x0,..., xn−1}. For each equation, add a clique connecting the variables appearing in that equation

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 6 / 23 Polynomial systems and graphs

A polynomial system deﬁned by m equations in n variables:

fi (x0,..., xn−1) = 0, i = 1,..., m

Construct a graph G (“primal graph”) with n nodes, as:

Nodes are variables {x0,..., xn−1}. For each equation, add a clique connecting the variables appearing in that equation

Example:

2 2 I = hx0 x1x2 + 2x1 + 1, x1 + x2, x1 + x2, x2x3i

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 6 / 23 Can the graph structure help solve this system? For instance, to compute Groebner bases? Or, perhaps we can do something better? Preserve graph (sparsity) structure? Complexity aspects?

Questions

“Abstracted” the polynomial system to a graph.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 7 / 23 Questions

“Abstracted” the polynomial system to a graph.

Can the graph structure help solve this system? For instance, to compute Groebner bases? Or, perhaps we can do something better? Preserve graph (sparsity) structure? Complexity aspects?

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 7 / 23 Remarkably (AFAIK) almost no work in computational algebraic geometry exploits this structure. We hope to change this... ;)

Graphical modelling

Pervasive idea in many areas, in particular: numerical linear algebra, graphical models, constraint satisfaction, database theory, . . .

Key notions: chordality and treewidth.

Many names: Arnborg, Beeri/Fagin/Maier/Yannakakis, Blair/Peyton, Bodlaender, Courcelle, Dechter, Lauritzen/Spiegelhalter, Pearl, Robertson/Seymour, . . .

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 8 / 23 We hope to change this... ;)

Graphical modelling

Pervasive idea in many areas, in particular: numerical linear algebra, graphical models, constraint satisfaction, database theory, . . .

Key notions: chordality and treewidth.

Many names: Arnborg, Beeri/Fagin/Maier/Yannakakis, Blair/Peyton, Bodlaender, Courcelle, Dechter, Lauritzen/Spiegelhalter, Pearl, Robertson/Seymour, . . .

Remarkably (AFAIK) almost no work in computational algebraic geometry exploits this structure.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 8 / 23 Graphical modelling

Pervasive idea in many areas, in particular: numerical linear algebra, graphical models, constraint satisfaction, database theory, . . .

Key notions: chordality and treewidth.

Many names: Arnborg, Beeri/Fagin/Maier/Yannakakis, Blair/Peyton, Bodlaender, Courcelle, Dechter, Lauritzen/Spiegelhalter, Pearl, Robertson/Seymour, . . .

Remarkably (AFAIK) almost no work in computational algebraic geometry exploits this structure. We hope to change this... ;)

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 8 / 23 A chordal completion of G is a chordal graph with the same vertex set as G, and which contains all edges of G. The treewidth of a graph is the clique number (minus one) of its smallest chordal completion. Meta-theorem: NP-complete problems are “easy” on graphs of small treewidth.

Chordality and treewidth

Let G be a graph with vertices x0,..., xn−1. A vertex ordering x0 > x1 > ··· > xn−1 is a perfect elimination ordering if for each xl , the set

Xl := {xl } ∪ {xm : xm is adjacent to xl , xl > xm}

is such that the restriction G|Xl is a clique. A graph is chordal if it has a perfect elimination ordering.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 9 / 23 Meta-theorem: NP-complete problems are “easy” on graphs of small treewidth.

Chordality and treewidth

Let G be a graph with vertices x0,..., xn−1. A vertex ordering x0 > x1 > ··· > xn−1 is a perfect elimination ordering if for each xl , the set

Xl := {xl } ∪ {xm : xm is adjacent to xl , xl > xm}

is such that the restriction G|Xl is a clique. A graph is chordal if it has a perfect elimination ordering. A chordal completion of G is a chordal graph with the same vertex set as G, and which contains all edges of G. The treewidth of a graph is the clique number (minus one) of its smallest chordal completion.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 9 / 23 Chordality and treewidth

Let G be a graph with vertices x0,..., xn−1. A vertex ordering x0 > x1 > ··· > xn−1 is a perfect elimination ordering if for each xl , the set

Xl := {xl } ∪ {xm : xm is adjacent to xl , xl > xm}

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 9 / 23 Bad news? (I)

Subset sum problem, with data A = {a1,..., an} ⊂ Z. Is there a subset of A that adds up to S?

Letting si be the partial sums, we can write a polynomial system:

0 = s0

0 = (si − si−1)(si − si−1 − ai )

S = sn

The graph associated with these equations is a path

s0 — s1 — s2 —··· — sn

But, subset sum is NP-complete...

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 10 / 23 For polynomials, however, Groebner bases can destroy chordality.

Ex: Consider I = hx0x2 − 1, x1x2 − 1i,

whose associated graph is the path x0 — x2 — x1 .

Every Groebner basis must contain the polynomial x0 − x1, breaking the sparsity structure.

Bad news? (II)

For linear equations, “good” elimination preserves graph structure (perfect!)

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 11 / 23 Every Groebner basis must contain the polynomial x0 − x1, breaking the sparsity structure.

Bad news? (II)

For linear equations, “good” elimination preserves graph structure (perfect!) For polynomials, however, Groebner bases can destroy chordality.

Ex: Consider I = hx0x2 − 1, x1x2 − 1i,

whose associated graph is the path x0 — x2 — x1 .

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 11 / 23 Bad news? (II)

For linear equations, “good” elimination preserves graph structure (perfect!) For polynomials, however, Groebner bases can destroy chordality.

Ex: Consider I = hx0x2 − 1, x1x2 − 1i,

whose associated graph is the path x0 — x2 — x1 .

Every Groebner basis must contain the polynomial x0 − x1, breaking the sparsity structure.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 11 / 23 Our results

A chordal elimination algorithm, to exploit graphical structure. Conditions under which chordal elimination succeeds. Recursive method for computing elimination ideals of maximal cliques For a certain class, complexity is linear in number of variables! (exponential in treewidth) Implementation and experimental results

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 12 / 23 Chordal elimination (sketch)

Given equations, construct graph G, a chordal completion, and a perfect elimination ordering.

Will produce a decreasing sequence of ideals I = I0 ⊇ I1 ⊇ · · · ⊇ In−1.

Given current ideal Il , split the generators

Il = Jl + Kl+1 |{z} | {z } ∈K[Xl ] 6∈K[Xl ]

and eliminate variable xl

Il+1 = eliml+1(Jl ) + Kl+1

“Ideally” (!), Il should be the l-th elimination ideal eliml (I)... Notice that by chordality, graph structure is always preserved!

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 13 / 23 Def: A polynomial f is simplicial if for each variable xl , the monomial ml d of largest degree in xl is unique and has the form ml = x l .

Thm 2: Let I = hf1,..., fs i be an ideal such that for each 1 ≤ i ≤ s, fi is generic simplicial. Then, chordal elimination succeeds.

[Intuition: interaction of (iterated) “closure/extension thm” + chordality]

When does chordal elimination succeed?

We need conditions for this to work, i.e., for V(Il ) = V(eliml (I)).

Thm 1: Let I be an ideal and assume that for each l such that Xl is a maximal clique of G, the ideal Jl ⊆ K[Xl ] is zero dimensional. Then, chordal elimination succeeds.

In particular, ﬁnite ﬁelds Fq, and 0/1 problems.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 14 / 23 [Intuition: interaction of (iterated) “closure/extension thm” + chordality]

When does chordal elimination succeed?

We need conditions for this to work, i.e., for V(Il ) = V(eliml (I)).

Thm 1: Let I be an ideal and assume that for each l such that Xl is a maximal clique of G, the ideal Jl ⊆ K[Xl ] is zero dimensional. Then, chordal elimination succeeds.

In particular, ﬁnite ﬁelds Fq, and 0/1 problems.

Def: A polynomial f is simplicial if for each variable xl , the monomial ml d of largest degree in xl is unique and has the form ml = x l .

Thm 2: Let I = hf1,..., fs i be an ideal such that for each 1 ≤ i ≤ s, fi is generic simplicial. Then, chordal elimination succeeds.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 14 / 23 When does chordal elimination succeed?

We need conditions for this to work, i.e., for V(Il ) = V(eliml (I)).

Thm 1: Let I be an ideal and assume that for each l such that Xl is a maximal clique of G, the ideal Jl ⊆ K[Xl ] is zero dimensional. Then, chordal elimination succeeds.

In particular, ﬁnite ﬁelds Fq, and 0/1 problems.

Def: A polynomial f is simplicial if for each variable xl , the monomial ml d of largest degree in xl is unique and has the form ml = x l .

Thm 2: Let I = hf1,..., fs i be an ideal such that for each 1 ≤ i ≤ s, fi is generic simplicial. Then, chordal elimination succeeds.

[Intuition: interaction of (iterated) “closure/extension thm” + chordality]

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 14 / 23 Naive chordal elimination can fail

2 0 1 I = hx0x2 + 1, x1 + x2, x1 + x2, x2x3i

2 Groebner basis:

3 {x0 − 1, x1 − 1, x2 + 1, x3} Elimination:

x0x2 + 1 } 2 2 )  x1 + x2 x1 + x2 2  → x2 + x2  x1 + x2 x1 + x2 → 0  x2x3 x2x3 x2x3 

We got I3 = h0i, but really elim3(I) = hx3i.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 15 / 23 For some purposes, ∪l gb(Hl ) has same information as gb(I), and is much smaller/sparser.

Compute the maximal clique ideals K[Xl ] from the output of the chordal elimination algorithm, in a structure-preserving way.

[Intuition: variety has “small” coordinate projections, can compute those, and glue them]

Elimination ideals of maximal cliques

In general, Groebner bases can be very large, and destroy chordality. Can we do something nearly as good, preserving graph structure?

Idea: Compute elimination ideals Hl := I ∩ K[Xl ] for the maximal cliques. (A chordal graph has at most n maximal cliques)

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 16 / 23 Elimination ideals of maximal cliques

In general, Groebner bases can be very large, and destroy chordality. Can we do something nearly as good, preserving graph structure?

Idea: Compute elimination ideals Hl := I ∩ K[Xl ] for the maximal cliques. (A chordal graph has at most n maximal cliques)

For some purposes, ∪l gb(Hl ) has same information as gb(I), and is much smaller/sparser.

Compute the maximal clique ideals K[Xl ] from the output of the chordal elimination algorithm, in a structure-preserving way.

[Intuition: variety has “small” coordinate projections, can compute those, and glue them]

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 16 / 23 Example: graph colorings

0 Let I be given by the equations:

3 6 7 xi − 1 = 0, 0 ≤ i ≤ 8

x9 − 1 = 0 9 8 3 2 2 xi + xi xj + xj = 0, (i, j) blue edge 1 4 5 2

Graph G (blue) and its chordal completion G¯ (green). There are 7 maximal cliques:

X0 = {x0, x6, x7}, X1 = {x1, x4, x9}, X2 = {x2, x3, x5}, X3 = {x3, x5, x7, x8},

X4 = {x4, x5, x8, x9}, X5 = {x5, x7, x8, x9}, X6 = {x6, x7, x8, x9}

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 17 / 23 0

6 7 Elimination tree of the graph G¯. The root/sink is 9. 9 8 3

1 4 5 2 Some of the clique elimination ideals: 2 H0 = hx0 + x6 + 1, x6 + x6 + 1, x7 − 1i 2 H5 = hx5 − 1, x7 − 1, x8 + x8 + 1, x9 − 1i 2 H6 = hx6 + x8 + 1, x7 − 1, x8 + x8 + 1, x9 − 1i The corresponding varieties are:

n 2 o n 2 o H0 : {x0, x6, x7} → ζ, ζ , 1 , ζ , ζ, 1 n 2 o H5 : {x5, x7, x8, x9} → {1, 1, ζ, 1} , 1, 1, ζ , 1 n 2 o n 2 o H6 : {x6, x7, x8, x9} → ζ , 1, ζ, 1 , ζ, 1, ζ , 1

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 18 / 23 Complexity

For “nice” cases, complexity is linear in number of variables n, number of equations s, and exponential in treewidth κ.

Thm: Let I be such that each (maximal) H˜ j is q-dominated. The ακ complexity of computing Il is O˜ (s + lq ). We can ﬁnd all elimination ideals in O˜ (nqακ).

E.g., we recover known results on linear-time colorability for bounded treewidth: Cor: Let G be a graph and G¯ a chordal completion with largest clique of size κ. We can describe all q-colorings of G in O˜ (nqακ).

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 19 / 23 Implementation and examples

Implemented in Sage, using Singular and PolyBoRi (for F2). Graph colorings (counting q-colorings) Cryptography (“baby” AES, Cid et al.) Sensor Network localization Discretization of polynomial equations

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 20 / 23 Results: Crypto - AES variant (Cid et al.) - F2[x] Performance on SR(n, 1, 2, 4) for chordal elimination, and computing (lex/degrevlex) Gr¨obner bases (PolyBoRi).

n Variables Equations Seed ChordElim LexGB DegrevlexGB

0 575.516 402.255 256.253 6 176 320 1 609.529 284.216 144.316

2 649.408 258.965 133.367

0 941.068 > 1100, aborted 1279.879 10 288 528 1 784.709 > 1400, aborted 1150.332

2 1124.942 > 3600, aborted > 2500, aborted

For small problems standard Gröbner bases outperform chordal elimination, particularly using degrevlex order. Nevertheless, chordal elimination scales better, being faster than both methods for n = 10. In addition, standard Gröbner bases have higher memory requirements, which is reflected in the many experiments that aborted for this reason.

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 21 / 23 Results: Sensor network localization - Q[x]

Find positions, given a few known ﬁxed anchors and pairwise distances. Comparison with Singular: DegrevlexGB, LexFGLM Natural graph structure

2 2 kxi − xj k = dij ij ∈ A 2 2 kxi − ak k = eij ik ∈ B

Simplicial, therefore exact elimination Underconstrained regime: chordal is much better Overconstrained regime: competitive (plot)

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 22 / 23 If you want to know more: D. Cifuentes, P.A. Parrilo, Exploiting chordal structure in polynomial ideals: a Groebner basis approach. arXiv:1411.1745. D. Cifuentes, Exploiting chordal structure in systems of polynomial equations, S.M. thesis, MIT, 2014. Thanks for your attention!

Summary

Chordal structure can notably simplify polynomial system solving Under assumptions (treewidth+ algebraic structure), tractable! Yields practical, competitive, implementable algorithms

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 23 / 23 Summary

Chordal structure can notably simplify polynomial system solving Under assumptions (treewidth+ algebraic structure), tractable! Yields practical, competitive, implementable algorithms

If you want to know more: D. Cifuentes, P.A. Parrilo, Exploiting chordal structure in polynomial ideals: a Groebner basis approach. arXiv:1411.1745. D. Cifuentes, Exploiting chordal structure in systems of polynomial equations, S.M. thesis, MIT, 2014. Thanks for your attention!

Cifuentes, Parrilo (MIT) Chordal structure and polynomial systems UC Davis - 2014 23 / 23