Lecture 23: NP-Completeness

(CLRS 34.1-34.4)

June 24th, 2002

1 Introduction

• Until now we have been designing algorithms for specific problems – We have seen running times O(log n), O(n), O(n log n), O(n^{2}), O(n^{3}) …

– We have also discussed lower bounds (for comparison based sorting).

• It is natural to ask if we can somehow classify problems according to their hardness – What we call complexity theory.

• First natural question to ask is if there are problems we cannot solve at all?

– Yes, e.g. Turing halting problem (deciding if a given Turing machine halts on every

input)

• We often think about problems we can solve in polynomial time O(n^{k}) as being practically

solvable

– We have seen a lot of those, e.g Shortest Path, Minimum Spanning Tree, …

• Similarly we think about problems we need exponential time O(2^{n}) to solve as being practically unsolvable.

• We would like to be able to prove if a problem is practically solvable without actually having to develop an O(n^{k}) algorithm or proving a Ω(2^{n}) lower bound!

• As we will discuss there is a huge class of problems for which we do not know if they are

practically solvable or not

– On the other hand, we can identify a subclass for which we have strong evidence that problems in it are practically unsolvable

2 Decision problems

• In order to keep things simple we will focus on decision problems:

– Problems for which the output is Yes or No.

– We say that an algorithm for a decision problem accepts an input if it answers Yes on the input.

• Examples:

– Is input sorted?

– Is graph acyclic?

– Is there a shortest path of length < k between u and v in graph?

• In terms of proving lower bounds, considering decision problems do not really restrict us— decision problems often easier than corresponding optimization problem

– e.g. we can decide if there is a shortest path of length < k between u and v by actually computing the shortest path and comparing its length to k ⇓ lower bound on decision problem gives lower bound on optimization problem.

3 P, NP and EXP

• We are now ready to define our complexity classes a little more formally

– Note that if we should really do it right we would have to introduce a lot more formalism (out of the scope of this class - note however that CLRS goes into somewhat more detail than we do here)

EXP = {Decision problems solvable in exponential time} P = {Decision problems solvable in polynomial time}

– Note that for a given decision problem, k in the polynomial bound O(n^{k}) cannot depend on the problem instance.

• In order to investigate the relationship between EXP and P we define another class

NP = {Decision problems for which we given a Yes solution can verify in polynomial time that it is correct}

• Examples of problems in NP

– Is there a path of length < k between u and v? Given path we can easily verify if it really has length < k.

– Hamiltonian cycle problem: Is there a simple cycle containing all vertices? Again easy to verify solution.

• Note: N in NP really stands for “non-deterministic”

– If we can “guess” the solution we can solve the problem in polynomial time.

• P ⊆ NP ⊆ EXP

– P ⊆ NP obvious.

– NP ⊆ EXP since we can enumerate all the (exponential number of) possible solutions to the problem and check each of them in polynomial time.

• The big question is if P = NP?

– Intuitively no—often much harder to solve problem than to verify a solution.

– We really do not have any clue if P = NP or not, or NP = EXP, or …, but we have strong evidence that there is a core of problems in NP that are not in P. We call this class of problems NP C.

4 Polynomial time reduction

• In order to define NP C we need the notion of polynomial time reductions

– Just the idea of using the solution to one problem to solve another

• X ≤_{P} Y:

A problem X is polynomial time reducible to a problem Y (X ≤_{P} Y) if we can solve X in a polynomial number of calls to an algorithm for Y (and the instance of problem Y we solve can be computed in polynomial time from the instance of problem X).

• Note: X ≤_{P} Y and Y ∈ P ⇒ X ∈ P

– Explains ≤_{P} notation: X ≤_{P} Y, “X not more than a polynomial factor harder than Y ”

• Examples:

– Traveling Salesman problem (TSP): Given a complete (edges between every pair of vertices) weighted undirected graph G = (V, E), find the minimal weight simple cycle that visits every vertex in V.

– Decision problem version of TSP: Is there a TSP path/tour of weight < k?

– Hamiltonian cycle ≤_{P} TSP:

∗ Proof: Let all edges in the graph we want to solve Hamiltonian cycle for have weight 0. Make graph complete by adding edges with weight 1. Run TSP algorithm. The graph has a Hamiltonian cycle if and only if it has a TSP tour of weight 0.

5 NP -completeness

• We are now ready to define NP C

A problem Y is in NP C (it is NP -complete) if

a) Y ∈ NP

b) X ≤_{P} Y for all X ∈ NP – Note: The problems in NP C are the “hardest” problems in NP

– The following Theorem formalizes this and explains why NP C is an important class:

Theorem:

a) If any problem in NP C is in P then P = NP

b) If any problem in NP is not in P then NP C ∩ P = ∅

– Proof:

a) Y ∈ P ∩ NP C ⇒ for all X ∈ NP we have X ≤_{P} Y ⇒ X ∈ P

b) We have X ∈ NP and X ∉ P. Assume Y ∈ NP C ∩P. As X ≤_{P} Y we have X ∈ P, which is a contradiction.

• Note:

– The above theorem is the reason why we focus on the problems in NP C.

– We think the world looks like this—but we really do not know:

– If someone found a polynomial time solution to a problem in NP C our world would “collapse” and a lot of smart people have tried really hard to solve NP C problems efficiently ⇓ We regard Y ∈ NP C a strong evidence for Y being hard!

• But how do we know that there are actually any problem in NP C?

– If we can just find one problem in NP C the following lemma helps us to find more:

Lemma: If Y ∈ NP and X ≤_{P} Y for some X ∈ NP C then Y ∈ NP C

– Proof:

a) Y ∈ NP

b) For all Z ∈ NP we have Z ≤_{P} X which means that Z ≤_{P} Y (Z ≤_{P} X ≤_{P} Y)

• The lemma shows that we just need to prove Y ∈ NP (easy) and reduce problem in NP C to Y to prove that Y is in NP C

– We do not have to prove lower bound!

• Finding the first problem in NP C is somewhat difficult and require quite a lot of formalism

– The first problem proved to be in NP C was SAT: Give a boolean formula, is there an assignment of true and false to the variables that makes the formula true?

– For example:

Can

x_{10} ∧ (x_{4} ⇔ ¬x_{3}) ∧ (x_{5} ⇔ (x_{1} ∨ x_{2})) ∧ (x_{6} ⇔ ¬x_{4}) ∧ (x_{7} ⇔ (x_{1} ∧ x_{2} ∧ x_{4})) ∧ (x_{8} ⇔ (x_{5} ∨ x_{6})) ∧ (x_{9} ⇔ (x_{6} ∨ x_{7}) ∧ (x_{10} ⇔ (x_{7} ∧ x_{8} ∧ x_{9})) be satisfied?

• By now a lot of problems have been proved NP -complete using the lemma: – e.g. Hamiltonian cycle, TSP, …

– Whole books with NP C problems have been written.

– Next time we will look at some of the NP C proofs. ⇓ We really think problems in NP C do not have polynomial time solutions (they are hard!). Nevertheless, every year someone claims to have found a polynomial time solution to a problem in NP C ... until now they have all been wrong.