When Boolean Satisfiability Meets Gaussian Elimination in Simplex Way

Jie-Hong R. Jiang (Joint work with Cheng-Shen Han)

EE Dept / Grad. Inst. of Electronics Eng. National Taiwan University

2014/4/25 Yuan Ze University Outline

Introduction SAT solving under XOR constraints Interpolant generation from combined CNF and XOR reasoning Experimental results Conclusions

2014/4/25 Yuan Ze University 2 Introduction SAT in Theory

Boolean satisfiability (SAT)  Determine whether a CNF formula can satisfied under some variable assignment  .g.,

Literal Clause

(¬∨∨)( ∨¬∨)( ∨∨¬ )(¬ ∨¬∨¬)

Conjunctive Normal Form (CNF)

 NP-complete problem [Cook 1971]

2014/4/25 Yuan Ze University 3 Introduction SAT in Practice

 In practice, modern SAT solvers work ‘mysteriously’ well on application formulas with ~100,000 variables and ~1,000,000 clauses  It enables various applications, and inspires QBF and SMT (Satisfiability Modulo Theories) solver development

2014/4/25 Yuan Ze University 4 Introduction Pre-Modern SAT Procedure

Algorithm DPLL(Φ) { while there is a unit clause {l} in Φ Φ = BCP(Φ, l); while there is a pure literal l in Φ Φ = assign(Φ, l); if all clauses of Φ satisfied return true; if Φ has a conflicting clause return false; l := choose_literal(Φ); return DPLL(assign(Φ,l))  DPLL(assign(Φ,l)); }

2014/4/25 Yuan Ze University 5 Introduction SAT Solving with Case Splitting

 Example 1 (a + b + c) a 2 (a + b + ¬c) 3 (¬a + b + ¬c) b b 4 (a + c + d) 5 (¬a + c + d) c c c 6 (¬a + c + ¬d) 7 (¬b + ¬c + ¬d) d d d d d 8 (¬b + ¬c + d)

Source: Karem A. Sakallah, Univ. of Michigan

2014/4/25 Yuan Ze University 6 Introduction Modern SAT Procedure

Algorithm CDCL(Φ) { while(1) while there is a unit clause {l} in Φ Φ = BCP(Φ, l); while there is a pure literal l in Φ Φ = assign(Φ, l); if Φ contains no conflicting clause if all clauses of Φ are satisfied return true; l := choose_literal(Φ); assign(Φ,l); else if conflict at top decision level return false; analyze_conflict(); undo assignments; Φ := add_conflict_clause(Φ); }

2014/4/25 Yuan Ze University 7 Introduction SAT in Action

 SAT solving with implication and learning 1 (a + b + c) 9 (¬b + ¬c) a 2 (a b ¬c)10 (¬a ¬b) + + + b b 3 (¬a + b + ¬c11) (¬a)abcab¬¬¬  4 (a + c + d) c (¬(¬ba+a)¬cb)) 5 (¬a + c + d) 6 (¬a c ¬d) + + 6 7 (¬b + ¬c + ¬d)a 8 5 6 11ba a 7 3 45 8 (¬b + ¬c + d) d 8 10bc b 79 3 c c 5 5 d d 6 6 b 9 c 48 d Source: Karem A. Sakallah, Univ. of Michigan 6 6 2014/4/25 Yuan Ze University 8 Introduction Conflict Analysis & Clause Learning

 There can be many learnt clauses from a conflict  Clause learning admits non- chorological backtrack

 E.g., {x10587, x10588, x10592} … {x10374, x10582, x10578, x10373, x10629} … {x10646, x9444, x10373, x10635, x10637} Box: decision node Oval: implication node Inside: literal (decision level) Courtesy of Niklas Een 2014/4/25 Yuan Ze University 9 Introduction Clause Learning as Resolution

 Resolution of two clauses C1xand C2x:

C1xC2x C1C2

where x is pivot variable and C1C2 is the resolvant, .e., C1C2 = x.(C1x)(C2x)

 A learnt clause can be obtained from a sequence of resolution steps  Exercise: Find a resolution sequence leading to the learnt clause {x10374, x10582, x10578, x10373, x10629} in the previous slides

2014/4/25 Yuan Ze University 10 Introduction Modern Techniques

 Ingredients of modern SAT solvers:  DPLL-style search [Davis, Putnam, Logemann, Loveland, 1962]  Conflict-driven clause learning (CDCL) [Marques-Silva, Sakallah, 1996 (GRASP)]  Boolean constraint propagation (BCP) with two-literal watch [Moskewicz, Modigan, Zhao, Zhang, Malik, 2001 (Chaff)]  Decision heuristics using variable activity [Moskewicz, Modigan, Zhao, Zhang, Malik, 2001 (Chaff)]  Restart  Preprocessing  Support for incremental solving [Een, Sorensson, 2003 (MiniSat)]

2014/4/25 Yuan Ze University 11 Introduction Solver Evolution

 SAT competition statistics http://www.satcompetition.org/PoS11/

2014/4/25 Yuan Ze University 12 Introduction Certification

SAT certification  SAT instance: truth assignment

(a)(a  b)(a  b) Satisfied under a=1, b=1 Useful in model finding  UNSAT instance: resolution proof (a)(a  b)(a  b) Unsatisfiability can be shown by a resolution sequence leading to (a) empty clause Useful in interpolant derivation ()

2014/4/25 Yuan Ze University 13 Introduction Resolution

 Resolution is complete for SAT solving  A CNF formula is unsatisfiable if and only if there exists a resolution sequence leading to the empty clause

 Example (abc)(ac)(bd)(c)(cd)

(bc) (d)

(cd)

(d)

() 2014/4/25 Yuan Ze University 14 Introduction Craig Interpolation

 [Craig Interpolation Thm, 1957] If AB is UNSAT for formulae A and B, there exists an interpolant I of A such that I 1. AI 2. IB is UNSAT A B 3. I refers only to the common variables of A and B I is an abstraction of A

2014/4/25 Yuan Ze University 15 Introduction Interpolant and Resolution Proof

 SAT solver may produce the resolution proof of an UNSAT CNF 

 For = AB specified, the corresponding interpolant can be obtained in time linear in the resolution proof

A B (abc)(ac)(bd)(c)(cd) (bc)(c)(1)(1)(1)

(bc) (d)

(cd) = (bc)

(d) [McMillan, 2003]

2014/4/25() Yuan Ze University 16 Introduction Incremental SAT Solving

To solve, in a row, multiple CNF formulae, which are similar except for a few clauses, can reuse the learnt clauses?  What if adding a clause to ?  What if deleting a clause from ?

2014/4/25 Yuan Ze University 17 Introduction MiniSat

 MiniSat API  void addClause(Vec clause)  bool solve(Vec assumps)  bool readModel(Var x) − for SAT results  bool assumpUsed(Lit p) − for UNSAT results

 The method solve() treats the literals in assumps as unit clauses to be temporary assumed during the SAT- solving.  More clauses can be added after solve() returns, then incrementally another SAT-solving executed.

Courtesy of Niklas Een 2014/4/25 Yuan Ze University 18 Introduction Applications

Pervasive applications  Hardware/software verification  Logic synthesis  Artificial intelligence  Combinatorial optimization  …

2014/4/25 Yuan Ze University 19 Introduction Limitation

Pure CNF SAT solvers are not scalable in solving formulas that in part encode XOR constraints, which arise in applications such as  Cryptanalysis  Model counting  Decoder synthesis, etc.

Some modern SAT solvers integrate special XOR handling into SAT solving

2014/4/25 Yuan Ze University 20 XOR Constraints

XOR-clauses  Example

¬ ∨∨ ∨¬∨ ⨁¬⨁ ∨∨¬ ¬ ∨¬∨¬

 A length-n XOR clause can be translated into 2 clauses

 Linear equation over GF(2)  ⨁⨁ =0

2014/4/25 Yuan Ze University 21 XOR Constraints

Matrix form : ⨁ =0 10010 :⨁ =1 01011 :⨁⨁ =1 11101

 =  = |

Deductive power of XOR-reasoning may play an essential role in the effectiveness of SAT solving

2014/4/25 Yuan Ze University 22 Prior Methods

 Non-interactive  Interactive  Performing XOR  Invoking XOR reasoning and SAT reasoning on-the-fly solving in separate during SAT solving phases  Inference-rule based  Weak reasoning  Solver:  Solver:  subst-minisat  MoRsat  ec-minisat  Linear-algebra based = …  Solver: (⨁⨁ =1)  CryptoMiniSat (⨁ =0) ⨁⨁ =1 … imply = 2014/4/25 Yuan Ze University 23 Motivations

Enhance deductive power of XOR-reasoning  Complete propagation/conflict detection

Reduce computation overhead of XOR- reasoning

Derive compact Craig interpolants from combined CNF and XOR reasoning  Not supported by any current solver equipped with XOR-reasoning capability

2014/4/25 Yuan Ze University 24 Procedure

2014/4/25 Yuan Ze University 25 Deductive Power of XOR-reasoning

Gaussian elimination ()  Matrix in upper triangular form (row echelon form) Gauss-Jordan elimination ()  Matrix in diagonal form (reduced row echelon form)

111110 100000 imply = 011110 010001 [A|b]= [A’|b’]= imply = 001111 001000 imply = 000111 000111

2014/4/25 Yuan Ze University 26 Gauss-Jordan Elimination

Proposition: Gauss-Jordan elimination guarantees complete propagation/conflict detection for a given the matrix = | with respect to some variable assignment

Computation is expensive during the DPLL search procedure  ()  Frequently update matrix

2014/4/25 Yuan Ze University 27 Reduce Computation Overhead

Gauss-Jordan elimination  = | → ′ = ||′  Basic and nonbasic variables correspond to the columns of and Simplex-style matrix update  Two-literal watching scheme  Lazy and incremental matrix update

I N 10010 1 1 01001 1 0 0011111 2014/4/25 Yuan Ze University 28 Enforced Invariant

For each row of , two non- assigned variables are watched st  1 watched variable, (), must be a basic variable nd  2 watched variable, (), must be a nonbasic variable

2014/4/25 Yuan Ze University 29 Example

10010 1 1 01001 1 0 00111 1 1

2014/4/25 Yuan Ze University 30 Example

10010 1 1 01001 1 0 00111 1 1

Nonbasic variable is assigned

 =  Select non-assigned nonbasic variable as the new second watched variable  No other rows are affected by this action

2014/4/25 Yuan Ze University 31 Example

10010 1 1 01001 1 0 00111 1 1

Basic variable is assigned

 =, =  Select non-assigned nonbasic variable as the new first watched variable  Perform pivoting operation on the column of new basic variable

2014/4/25 Yuan Ze University 32 Example

10010 1 1 01001 1 0 10101 0 0

Basic variable is assigned

 =, =  Select non-assigned nonbasic variable as the new first watched variable  Perform pivoting operation on the column of new basic variable

2014/4/25 Yuan Ze University 33 Example

10010 1 1 01001 1 0 :⨁⨁ =0 10101 0 0 imply =

Basic variable is assigned

 =, =  Select non-assigned nonbasic variable as the new first watched variable  Perform pivoting operation on the column of new basic variable

2014/4/25 Yuan Ze University 34 Interpolant Generation

Craig interpolation becomes an indispensable tool in verification and synthesis

Interpolant generation [McMillan 2005]  CNF constraints: Clause interpolation rules  XOR constraints: Linear interpolation rules  Hybrid constraints?

2014/4/25 Yuan Ze University 35 Interpolant Generation

A B , , ∈

…( ∨)()…(⨁⨁ =1)…(⨁⨁ =1)…(¬)…..

(¬ ∨)( ∨¬) (⨁ =0)

()

(⊥) XOR-constraints expressed in CNF?  Hard to solve  Interpolants can be unreasonably large

2014/4/25 Yuan Ze University 36 Derivation Rules

Derivation rules for partial interpolants

- ∈A :< | > <>: partial interpolant T: matrix transpose +: (modulo 2) matrix addition - ∈B :<0|0 >

:<[∗ ∗ > :<[∗ ∗ > - ∗ ∗ ∗ ∗ + :<[ +[ >

2014/4/25 Yuan Ze University 37 Derivation Rules

, , ∈ A B <( ⨁⨁ =1)> <(0=0)>

…( ∨)()…(⨁⨁ =1)…(⨁⨁ =1)…(¬)…..

(¬ ∨)( ∨¬) (⨁ =0)<( ⨁⨁ =1)>

()

(⊥) Our answer:

∈ :<[∗ |∗]> ∗ ∗ ∗ :< ∨( =)|¬ >

2014/4/25 Yuan Ze University 38 Derivation Rules

, , ∈ A B <( ⨁⨁ =1)> <(0=0)>

…( ∨)()…(⨁⨁ =1)…(⨁⨁ =1)…(¬)…..

(¬ ∨)( ∨¬) (⨁ =0)<( ⨁⨁ =1)> < (¬⋁>

()

(⊥) Our answer:

∈ :<[∗ |∗]> ∗ ∗ ∗ :< ∨( =)|¬ >

2014/4/25 Yuan Ze University 39 Derivation Rules

, , ∈ A B <( ⨁⨁ =1)> <(0=0)>

…( ∨)()…(⨁⨁ =1)…(⨁⨁ =1)…(¬)…..

(¬ ∨)( ∨¬) (⨁ =0)<( ⨁⨁ =1)> < (¬⋁( ⨁ =0)>

()

(⊥) Our answer:

∈ :<[∗ |∗]> ∗ ∗ ∗ :< ∨ ( =)|¬ >

2014/4/25 Yuan Ze University 40 Experimental Results (Part 1)

SimpSat was implemented in the C++ language based on CryptoMiniSat 2.9.1

Benchmark  Cryptanalysis benchmarks  Equivalence checking benchmarks from Altera CRC (cyclic redundancy check) circuits  3-regular graph benchmarks

2014/4/25 Yuan Ze University 41 Cryptanalysis Benchmarks

CPU time

2014/4/25 Yuan Ze University 42 Cryptanalysis Benchmarks

GE utility (#useful GE calls: #all GE calls) 80%

70% CMS+ (GE Util)

60% SimpSat (GE

50% Util)

40%

30%

20%

10%

0% HiTag2-9 Grain-106 Grain-107 Grain-108 Grain-109 Bivium-45 Bivium-46 Bivium-47 Bivium-48 Bivium-49 Bivium-50 Bivium-51 Bivium-52 Bivium-53 Bivium-54 Bivium-55 Bivium-56 HiTag2-10 HiTag2-11 HiTag2-12 HiTag2-13 HiTag2-14 HiTag2-15 Trivium-151 Trivium-152 Trivium-153 Trivium-154 Trivium-155 Trivium-156 Trivium-157 2014/4/25 Yuan Ze University 43 Cryptanalysis Benchmarks

Speedup per GE call 3.5

3 GE Spdup per call

2.5

2

1.5

1

0.5

0

2014/4/25 Yuan Ze University 44 Equivalence Checking Benchmarks

CPU time

2014/4/25 Yuan Ze University 45 Equivalence Checking Benchmarks

# discovered short clauses ( 2 literals) 40 35 CMS+ (GE #2xcl) 30 SimpSat (GE #2xcl) 25 20 15 10 5 0

2014/4/25 Yuan Ze University 46 3-regular Graph Benchmarks

CPU time

2014/4/25 Yuan Ze University 47 Experimental Results (Part 2)

 MiniSAT-GE was built upon MiniSAT  with XOR-constraint solving

 Interpolants were synthesized using ABC tool  AIG (And-Inverter Graph)

 Benchmarks  Created from the unsatisfiable instances of 3- regular graph benchmarks

 Clauses evenly partitioned into and for interpolation

2014/4/25 Yuan Ze University 48 Interpolant Generation

2014/4/25 Yuan Ze University 49 Conclusions

SimpSat  Integrating SAT solving tightly with Gaussian elimination in the style of simplex method

Enhanced deductive power of XOR- reasoning  Complete propagation/conflict detection with Gauss-Jordan elimination

2014/4/25 Yuan Ze University 50 Conclusions

Reduced computation overhead for XOR- reasoning  Two-literal watching scheme  Simplex-style matrix update

Compacted Craig interpolants from combined CNF and XOR reasoning  Interpolant derivation rules

2014/4/25 Yuan Ze University 51 Thanks for Your Attention

2014/4/25 Yuan Ze University 52