2010 MarceloFinger SatisifiabilityandProbabilisticSatisifiability

(1)

Satisifiability and

Probabilistic Satisifiability

Marcelo Finger

Department of Computer Science Instituto de Matem´atica e Estat´ıstica

Universidade de S˜ao Paulo

2010

(2)

Introduction SAT PSAT History Canonical PSAT via Logic Conclusions

Topics

1 Introduction

2 SAT

Empirical Properties

3 Probabilistic Satisfiability

4 A Brief History of PSAT

5 Canonical Reduction of PSAT to SAT

6 PSAT via Logic

7 Conclusions and Conjectures

(3)

Are they all saying the truth?

Every night, at least two of them are at the table Each says he comes “only” 60% of the nights

(4)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(5)

The SAT Problem

The Centrality of SAT

Satisfiability is a central problem in Computer Science Both theoreticaland practicalinterests

(6)

The SAT Problem

SAT was the 1st NP-complete problem

(7)

The SAT Problem

SAT was the 1st NP-complete problem SAT received a lot of attention [1960-now]

(8)

The SAT Problem

SAT has very efficient implementations

(9)

The SAT Problem

SAT has become the “assembly language” of hard-problems

(10)

The SAT Problem

SAT has become the “assembly language” of hard-problems SAT is logic

(11)

The PSAT Problem

Probabilistic satisfiability (PSAT) was proposed by [Boole 1854]

On the Laws of Thought Classical probability

No assumption of a priori statistical independence Rediscovered several times since Boole

PSAT is also NP-complete

(12)

The PSAT Problem

Probabilistic satisfiability (PSAT) was proposed by [Boole 1854]

On the Laws of Thought Classical probability

PSAT is also NP-complete

But PSAT seems to be more complex than SAT Why?

(13)

Applications of SAT

SAT has “good” implementations SAT has many applications

(14)

Applications of SAT

SAT has “good” implementations SAT has many applications

Planning: SatPlan Answer Set Programming Integer Programming

Map colouring, combinatorial problems Optimisation problems

etc

NP-complete problems reduced to SAT

(15)

Potential Applications of PSAT

SAT has many potential applications

(16)

Potential Applications of PSAT

SAT has many potential applications Computer models of biological processes Machine learning

Fault tolerance

Software design and analysis.

Economics, econometrics, etc.

But there are no practical algorithms for PSAT

(17)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(18)

The Setting: the language

Atoms: P ={p₁, . . . ,p_n} Literals: p_i and¬p_j

¯

p =¬p,¬p =p

A clause is a set of literals. Ex: {p,¯q,r}or p∨ ¬q∨r A formula C is a set of clauses

(19)

The Setting: semantics

Valuation for atoms v :P → {0,1}

(20)

The Setting: semantics

An atomp is satisfiedif v(p) = 1

(21)

The Setting: semantics

An atomp is satisfiedif v(p) = 1 Valuations are extended to all formulas

(22)

The Setting: semantics

An atomp is satisfiedif v(p) = 1 Valuations are extended to all formulas v(¯λ) = 1⇔v(λ) = 0

(23)

The Setting: semantics

A clause c is satisfied (v(c) = 1) ifsomeliteral λ∈c is satisfied

(24)

The Setting: semantics

A clause c is satisfied (v(c) = 1) ifsomeliteral λ∈c is satisfied

A formula C is satisfied (v(C) = 1) ifallclauses in C are satisfied

(25)

The Problem

A formula C is satisfiable if exitsv,v(C) = 1.

Otherwise, C is unsatisfiable

(26)

The Problem

Otherwise, C is unsatisfiable The SAT Problem

Given a formulaC, decide ifC is satisfiable.

Witnesses: IfC is satisfiable, provide a v such that v(C) = 1.

(27)

The Problem

Otherwise, C is unsatisfiable The SAT Problem

Given a formulaC, decide ifC is satisfiable.

Witnesses: IfC is satisfiable, provide a v such that v(C) = 1.

SAT has small witnesses

(28)

An NP Algorithm for SAT

NP-SAT(C)

Input: C, a formula in clausal form Output: v, ifv(C) = 1; no, otherwise.

1: Guess a v

2: Show, in polynomial time, that v(C) = 1

3: returnv

4: if no suchv is guessablethen

5: returnno

6: end if

(29)

A Na¨ıve SAT Solver

NaiveSAT(C)

Input: C, a formula in clausal form Output: v, ifv(C) = 1; no, otherwise.

1: for every valuationv overp₁, . . . ,p_n do

2: if v(C) = 1 then

3: returnv

4: end if

5: end for

6: returnno

(30)

Introduction SAT PSAT History Canonical PSAT via Logic Conclusions Empirical

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(31)

The SAT Phase Transition

(32)

Introduction SAT PSAT History Canonical PSAT via Logic Conclusions Empirical

The Phase Transition Diagram

3-SAT, N is fixed

Higher N, more abrupt transition M/N: low (SAT); high (UNSAT)

Phase transition point: M/N≈4.3, 50%SAT [Toby & Walsh 1994]

Invariant withN

Invariant with algorithm!

(33)

The Phase Transition Diagram

3-SAT, N is fixed

Higher N, more abrupt transition M/N: low (SAT); high (UNSAT)

Phase transition point: M/N≈4.3, 50%SAT [Toby & Walsh 1994]

Invariant withN

Invariant with algorithm!

No theoretical explanation

There is another phase-transition for SAT based on

(34)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(35)

Why PSAT?

3 forms of reasoning (According to Peirce) Deductive reasoning

Inductive reasoning Abductive reasoning

(36)

Why PSAT?

Probabilisticreasoning Abductive reasoning

(37)

Why PSAT?

Probabilisticreasoning Abductive reasoning PSAT is at the intersection

(38)

The Setting: the language

Logical variables or atoms: P ={x₁, . . . ,x_n} Connectives: ∧,∨,¬,→,↔.

Formulas (L) are inductively composed form atoms using connectives

Formulas can be brought to clausal form, but need not be.

(39)

Basics

Propositional valuation v :P → {0,1}

Generalised for any propositional formula (clausal or not) v :L → {0,1}

(40)

Basics

A probability distribution over propositional valuations π:V →[0,1]

2ⁿ

X

i=1

π(vi) = 1

(41)

Basics

A probability distribution over propositional valuations π:V →[0,1]

2ⁿ

X

i=1

π(vi) = 1 Probability of a formulaα according to π

(42)

Kolmogorov Axioms for Probability

α, β∈ L

(K1) 0≤P(α)≤1 (K2) P(⊤) = 1

(K3) If|=¬(α∧β) then P(α∨β) =P(α) +P(β)

(43)

Kolmogorov Axioms for Probability

α, β∈ L

(K1) 0≤P(α)≤1 (K2) P(⊤) = 1

(K3) If|=¬(α∧β) then P(α∨β) =P(α) +P(β) From which we can derive:

(K4) P(¬α) = 1−P(α)

(K5) Ifα|=β thenP(α)≤P(β) (K6) Ifα≡β thenP(α) =P(β)

(44)

The PSAT Problem

Considerk formulasα₁, . . . , α_k defined onn atoms {x1, . . . ,x_n}

A PSAT problem Σ is a set of k restrictions Σ ={P(α_i) =p_i|1≤i ≤k}

Probabilistic Satisfiability: are these restrictions consistent?

(45)

A PSAT example

(46)

A PSAT example

Σ =

P(a∨b) =P(a∨c) =P(b∨c) = 1 P(a) =P(b) =P(c) = 0.6

(47)

The PSAT Problem: An Algebraic Formalisation

Vector of probabilities p of dimensionk×1 (given) Consider a “large” matrix A_k×2ⁿ = [a_ij] (computed)

a_ij =v_j(αi)∈ {0,1}

PSAT: decide if there is vector πof dimension 2ⁿ×1 such that Aπ = p

Pπ_i = 1

π ≥ 0

(48)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(49)

The PSAT Problem

Probabilistic satisfiability (PSAT) was proposed in On the Laws of Thought[Boole 1854]

Classical probability

Hailperin 1965 De Finetti 1937, 1974

Nilsson 1986, (re)introduces PSAT to AI

and many others (see [Hansen & Jaumard 1996])

(50)

Potential Applications of PSAT

PSAT has many potential applications

(51)

Potential Applications of PSAT

PSAT has many potential applications Computer models of biological processes Machine learning

Fault tolerance.

Software design and analysis.

Economics, econometrics, etc.

But there are no practical algorithms for PSAT

(52)

PSAT is NP-complete

PSAT is NP-complete: [Georgakopoulos & Kavvadias &

Papadimitriou 1988]

A PSAT problem has a solution, then there is a solution π with at mostk+ 1 elementsπi >0

Carath´eodory’s Lemma

So PSAT has a polynomial size witness







1 · · · 1

0/1 · · · ^0/1 ... . .. ...

0/1 · · · 0/1







·





 π₁ π2

... π_k+1







=





 1 p₁

... p_k







(53)

PSAT is NP-complete

PSAT is NP-complete: [Georgakopoulos & Kavvadias &

Papadimitriou 1988]

A PSAT problem has a solution, then there is a solution π with at mostk+ 1 elementsπi >0

Carath´eodory’s Lemma

So PSAT has a polynomial size witness







1 · · · 1

0/1 · · · ^0/1 ... . .. ...







·





 π₁ π2

...







=





 1 p₁

...







(54)

Existing Algorithms for PSAT

All algorithms are algebraic

Try to solve an exponential linear program Column generation heuristics

Easy cases: quickly generates witness (k≈300)

Hard cases: solves aweighted MAXSAT instance at each step weighted MAXSAT is a “hard” NP-complete problem

(55)

Existing Algorithms for PSAT

All algorithms are algebraic

Try to solve an exponential linear program Column generation heuristics

Easy cases: quickly generates witness (k≈300)

Hard cases: solves aweighted MAXSAT instance at each step weighted MAXSAT is a “hard” NP-complete problem

No phase-transition observed

[Kavvadias-Papadimitriou 1990, Hansen-Jaumard 1997]

(56)

Idea

Goal:

Obtain a “decent” algorithm for PSAT Study its empiric properties

Both SAT and PSAT are NP-complete problems SAT has good implementations

(57)

Idea

Goal:

Obtain a “decent” algorithm for PSAT Study its empiric properties

Both SAT and PSAT are NP-complete problems SAT has good implementations

Let’s reduce PSAT to SAT.

(58)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(59)

Canonical Reductions

A reductiontransforms an instance of PSAT into one of SAT One such reduction in polynomial time exists, due to

NP-completeness

(60)

Canonical Reductions

A reductiontransforms an instance of PSAT into one of SAT One such reduction in polynomial time exists, due to

NP-completeness

A reduction iscanonical if it encodes an NP-witness Easier if witness has a simple format

Normal forms simplify the study of complex objects

(61)

A PSAT Normal Form

PSAT atomic normal formΣ = (Γ,Ψ) Γ is a SAT problem, ie{α|P(α) = 1}

Ψ ={P(xi) =pi|xi is an atom}

(62)

A PSAT Normal Form

PSAT atomic normal formΣ = (Γ,Ψ) Γ is a SAT problem, ie{α|P(α) = 1}

Ψ ={P(xi) =pi|xi is an atom}

Theorem

Every PSAT problem has an equivalent atomic normal form, that can be computed in linear time.

(63)

That PSAT example is already normal form!

Γ =



 a∨b a∨c





Ψ ={P(a) =P(b) =P(c) = 0.6}

(64)

A Second Example

Σ =

P(a∨b∨c) = 1

P(a∧b) = 0.61;P(a∧c) = 0.60;P(b∧c) = 0.59

Can be put in normal form (Γ,Ψ)

Γ =







¯

x∨a ¯y∨a ¯z∨b a∨b∨c

¯

x∨b ¯y∨c ¯z∨c

¯

a∨¯b∨x ¯a∨¯c∨y ¯b∨¯c∨z







Ψ ={P(x) = 0.61;P(y) = 0.60;P(z) = 0.59}

(65)

Algebraic Notation

A_n×m is a matrix of n lines by mcolumns.

A^j is the j-th column of matrix A Let p be a vector of dimension n.

A[j :=p] is obtained fromAby replacingA^j withp IfA is a square matrix,|A|isA’s determinant IfA·x =b then, by Cramer’s Rule

x_j = |A[j :=b]|

|A|

(66)

Properties of Atomic Normal Form

(Γ,Ψ) has a solution iff there isA= [a_i,j] and π satisfying







1 · · · 1 a_1,1 · · · a_1,k+1

... . .. ... a_k,1 · · · a_k,k₊₁







·





 π₁ π₂ ... π_k₊₁







=





 1 p₁

... p_k







a_i,j ∈ {0,1} πj ≥0

(1)

Ifπ_j >0 thenA^j is a valuation satisfying Γ.

(67)

Second Example – Part II

Γ =







¯





 Ψ ={P(x) = 0.61;P(y) = 0.60;P(z) = 0.59}

is P-satisfiable because

x y z







1 1 1 1 1 1 0 0 1 0 1 0 1 0 0 1







·





 0.4 0.21

0.2 0.19







=





 1 0.61 0.60 0.59







a 1 1 1 0

(68)

The Canonical Reduction

Canonical reduction direct encoding of the solution A·π=p of the normal form (Γ,Ψ).

Γ(x1, . . . ,xk;y1, . . .yl) Ψ ={P(xi) =pi}

A^j = [1 a_1,j · · · a_k,j]^t, so Γ(a_1,j, . . . ,a_k,j;y₁^j, . . .y_l^j) must hold Each πi is encoded as a sequence of b bits

Each p_i is encoded as a sequence of b+k bits Direct encoding of sums of bits

Direct encoding of bit product as a conjunction Number of variables isO(b·k²)

(69)

The Canonical Reduction

Canonical reduction direct encoding of the solution A·π=p of the normal form (Γ,Ψ).

Γ(x1, . . . ,xk;y1, . . .yl) Ψ ={P(xi) =pi}

A^j = [1 a_1,j · · · a_k,j]^t, so Γ(a_1,j, . . . ,a_k,j;y₁^j, . . .y_l^j) must hold Each πi is encoded as a sequence of b bits

Each p_i is encoded as a sequence of b+k bits Direct encoding of sums of bits

Direct encoding of bit product as a conjunction Number of variables isO(b·k²)

(70)

Number of Bits in the Canonical Reduction

Cramer’s Rule x_j = ^|A[j_|A|^:=b]|

(71)

Number of Bits in the Canonical Reduction

Theorem (Hadamard’s Maximum Determinant Problem 1893)

Let A= [a_ij]_n×n, a_ij ∈ {0,1}. Then

|A| ≤ (n+ 1)^(n+1)/2 2ⁿ

(72)

Number of Bits in the Canonical Reduction

|A| ≤ (n+ 1)^(n+1)/2 2ⁿ b = ^(k+1)₂ ⌈log(k+ 1)⌉ −k

Number of variables in the canonical reduction isO(k³·logk)

(73)

Number of Bits in the Canonical Reduction

|A| ≤ (n+ 1)^(n+1)/2 2ⁿ b = ^(k+1)₂ ⌈log(k+ 1)⌉ −k

Number of variables in the canonical reduction is (k³·logk)

EC AA !

(74)

Implementation

A canonical reduction of PSAT to SAT n = 80 variables

k = 5, b= 10

52 hours to compute!

(75)

Implementation

k = 5, b= 10

52 hours to compute! ECAA!

(76)

Implementation

k = 5, b= 10

Phase-transition found for PSAT

(77)

Implementation

k = 5, b= 10

Phase-transition found for PSATYippie!

(78)

PSAT Phase Transition

(79)

Lessons Learned So Far

Canonical reduction in nota “decent” implementation of a PSAT-solver

A “decent” one may not exits

There may not exist a linear reduction of PSAT to SAT PSAT may be more complex than SAT

even if P=NP or not

(80)

What to do now?

Explore other reduction strategies Explore other forms of reductions

Turing reduction: an algorithm that invokes a polynomial number of SAT problems

Turing reductions explore properties of PSAT problems We are going to explore some logical properties of PSAT

(81)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(82)

Probabilistic Entailment

Normal form problem (Γ,Ψ) Ψ|≈α iff ({¬α},Ψ) is P-UNSAT Intuition: Ψ forces P(α)>0

(83)

Probabilistic Entailment

Normal form problem (Γ,Ψ) Ψ|≈α iff ({¬α},Ψ) is P-UNSAT Intuition: Ψ forces P(α)>0

Theorem (PSAT Logic-Algebraic Characterization) The following are equivalent:

(Γ,Ψ)is P-satisfiable

∃ A, π such that A·π=p, aij∈ {0,1},π≥0, vA^j(Γ) = 1

(84)

Partial Solutions

A partial solution (A, π),

A·π =p, π≥0, a_ij ∈ {0,1}

Allowed: A^j such thatv_Aj(Γ) = 0

A partial solution always exists (relaxedsolution)







1 · · · 1

0 . .. ...

... . .. . .. ...

0 · · · 0 1







·





 π1

π₂ ... π_k+1







=





 1 p₁

... p_k







(85)

Second Example – Part III

Γ =







¯





 Ψ ={P(x) = 0.61;P(y) = 0.60;P(z) = 0.59}

The relaxed solution is

x y z







1 1 1 1

0 1 1 1

0 0 1 1

0 0 0 1







·





 0.39 0.01 0.01 0.59







=





 1 0.61 0.60 0.59







(86)

New PSAT Algorithm

1 Input: (Γ,Ψ)

2 Start with a relaxed solution

3 Transform partial solution into partial solution

4 Until either a solution is found, or (Γ,Ψ) is proven P-UNSAT.

(87)

New PSAT Algorithm

1 Input: (Γ,Ψ)

Transformation is by finding formulas α,ψ|≈α Submit Γ∪ {α} to SAT solver.

if UNSAT, (Γ,Ψ) is P-UNSAT

else, takev(Γ∪ {α}) = 1 and substitute one column of partial solution with it

This can always be done so as to obtain a new partial solution

(88)

New PSAT Algorithm

1 Input: (Γ,Ψ)

Transformation is by finding formulas α,ψ|≈α Submit Γ∪ {α} to SAT solver.

if UNSAT, (Γ,Ψ) is P-UNSAT

else, takev(Γ∪ {α}) = 1 and substitute one column of partial solution with it

This can always be done so as to obtain a new partial solution 2 Problems:

How to compute conditionα

(89)

Second Example – Part IV

Choose a columnj, replace it with [1 x₁ · · · x_k]^t

A^′=







1 1 1 1

0 1 x 1

0 0 y 1

0 0 z 1







Cramer’s:ruleπ^′_l = ^|A^′^[l:=p]|_|A′| ≥0, l ∈[1,k+ 1]. For l =j:

1 1 1 1

0 1 .61 1 0 0 .60 1 0 0 .59 1

1 1 1 1 ≥0 =⇒ y¯+z≤0

(90)

Second Example – Part IV (cont.)

Whenx,y,z ∈ {0,1},α= [¯y+z ≤0]is a formula Any linear (in)equality with {0,1}-variablesis a formula There is a linear-time SAT-translation of (in)equalities Lemma: Ψ|≈α

So Γ∪ {α}must be satisfiable if problem is P-SAT Valuation: [10 1 0]^t

Lemma: any column can replace some column of A, keeping π ≥0

(91)

Second Example – Part IV (cont.)

x y z







1 1 1 1

0 1 0 1

0 0 1 1

0 0 0 1







·





 0.38 0.02 0.01 0.59







=





 1 0.61 0.60 0.59







As all columns are Γ-consistent, the problem is P-satisfiable (We were lucky!)

(92)

Back to the First Example

Γ =





 a∨b a∨c b∨c







Ψ ={P(a) =P(b) =P(c) = 0.6}

(93)

First Example (cont)

Relaxed solution

a b c







1 1 1 1 0 1 1 1 0 0 1 1 0 0 0 1







·





 0.4

0 0 0.6







=





 1 0.6 0.6 0.6







Applying the process, obtain Ψ|≈[¯b≥1]∧[a+c ≤1]

But Γ∪ {[¯b≥1]∧[a+c ≤1]} is UNSAT So the problem is P-UNSAT

(94)

Computing Conditions from Partial Solutions

Partial solution A·π=p, computeA^′·π^′ =p Choose a column j, replace it with [1 x₁ · · · x_k]^t







1 · · · 1 · · · 1 a_1,1 · · · x₁ · · · a_1,k+1

... ... ...

a_k,1 · · · x_k · · · a_k,k₊₁







·





 π₁ π₂ ... π_k+1







=





 1 p₁

... p_k







(95)

Computing Conditions from Partial Solutions

Partial solution A·π=p, computeA^′·π^′ =p Choose a column j, replace it with [1 x₁ · · · x_k]^t







1 · · · 1 · · · 1 a_1,1 · · · x₁ · · · a_1,k+1

... ... ...

a_k,1 · · · x_k · · · a_k,k₊₁







·





 π₁ π₂ ... π_k+1







=





 1 p₁

... p_k







By Cramer’s rule π_j^′ = ^|A^′^[j:=p]|_|A_′_| ≥0

Note: |A^′[j :=p]|is a number, |A^′|is a polynomial of degree 1

(96)

Computing Conditions (Cont.)

Linear inequalities with variables over {0,1}are the inteface between SAT, algebra and probabilities

Other conditions obtained from Cramer’s Rule for other columns

π_j^′′ = |A^′[j^′ :=p]|

|A^′| ≥0

A suitable combination of those lead to a halting condition This algorithm is aTuring Reduction

Complexity of SAT calls is O(k³) variables We hope to bring it down toO(k²) variables

(97)

Computing Conditions (Cont.)

Linear inequalities with variables over {0,1}are the inteface between SAT, algebra and probabilities

Other conditions obtained from Cramer’s Rule for other columns

π_j^′′ = |A^′[j^′ :=p]|

|A^′| ≥0

A suitable combination of those lead to a halting condition This algorithm is aTuring Reduction

Complexity of SAT calls is O(k³) variables We hope to bring it down toO(k²) variables

(98)

Next Topic

1 Introduction

2 SAT

6 PSAT via Logic

(99)

Conclusions and Conjectures

Relationship between Deductive and Probabilistic reasoning Deductive bridge: Linear Algebra Probabilistic Not sure if PSAT will have a use in practice.

(100)

Conclusions and Conjectures

Relationship between Deductive and Probabilistic reasoning Deductive bridge: Linear Algebra Probabilistic Not sure if PSAT will have a use in practice.

Conjecture: the internal structure of class NP revealed by reductions to SAT

Subclasses NP^f ⊆NP: A problem Π∈NP^f if there is a reduction of Π into SAT of sizeO(f)-variables

PSAT is NPⁿ³^logⁿ

If Turing reductions are used, PSAT is in NPⁿ³