Lecture 15: Greedy Algorithms

CLRS 16.1-16.2

June 7th, 2002

1 Greedy Algorithms

• We have previously discussed dynamic programming—a way of improving on inefficient divideand-conquer algorithm:

– If same subproblem is solved several times, use table to store result of a subproblem the first time it is computed and never compute it again.

– Alternatively, we can think about filling up a table of subproblem solutions from the bottom.

• In divide-and-conquer (and thus dynamic programming) we used the fact that the solution to a problems depends on solutions to smaller subproblems.

• Another, simpler and often less powerful (and less well-defined), technique that uses the same feature is greediness

• Like in the case of dynamic programming, we will introduce greedy algorithms via an example.

1.1 Activity Selection

• Problem: Given a set A = {A_{1}, A_{2}, ⋯, A_{n}} of n activities with start and finish times (s_{i}, f_{i}), 1 ≤ i ≤ n, select maximal set S of “non-overlapping” activities.

– One can think of the problem as corresponding to scheduling the maximal number of classes (given their start and finish times) in one classroom.

• Solution:

– Sort activity by finish time (let A_{1}, A_{2}, ⋯, A_{n} denote sorted sequence)

– Pick first activity A_{1}

– Remove all activities with start time before finish time of A_{1}

– Recursively solve problem on remaining activities.

• Program:

Sort A by finish time S = {A_{1}} j = 1 FOR i = 2 to n DO

IF s_{i} ≥ s_{j} THEN

S = S ∪ {A_{i}}
j = i

FI OD

• Example:

– 11 activities sorted by finish time: (1, 4), (3, 5), (0, 6), (5, 7), (3, 8), (5, 9), (6, 10), (8, 11), (8, 12), (2, 13), (12, 14)

• Running time is obviously O(n log n).

• Is algorithm correct?

– Output is set of non-overlapping activities, but is it the largest possible?

• Proof of correctness:

– Given activities A = {A_{1}, A_{2}, ⋯, A_{n}} ordered by finish time, there is an optimal solution containing A_{1}:

∗ Suppose S ⊆ A is optimal solution

∗ If A_{1} ∈ S, we are done

∗ If A_{1} ∉ S:

· Let first activity in S be A_{k}

· Make new solution S′ = S \ {A_{k}} ∪ {A_{1}} by removing A_{k} and using A_{1} instead · S′ is valid solution (f_{1} < f_{k}) of maximal size (|S′| = |S|)

– S is an optimal solution for A containing A_{1} ⇒ S′ = S \ {A_{1}} optimal solution for A′ = {A_{i} ∈ A : s_{j} ≥ f_{1}} (e.g. after choosing A_{1} the problem reduces to finding optimal solution for activities not overlapping with A_{1})

∗ Suppose we have solution S′′ to A′ such that |S′′| > |S′| = |S| − 1

∗ S′′′ = S′′ ∪ {A_{1}} would be solution to A

∗ Contradiction since we would have |S′′′| > |S|

– Correctness follows by induction on size of S

• Comparison of greedy algorithm technique with dynamic programming (divide-and-conquer):

– In greedy algorithm we choose what looks like best solution at any given moment and recurse (choice does not depend on solution to subproblems).

– In dynamic programming, solution depends on solution to subproblems.

– Both techniques use optimal solution to subproblems (optimal solution “contains optimal solution for subproblems within it”).

• It is often hard to figure out when being greedy works!

Example:

– 0 − 1 knapsack problem: Given n items, with item i being worth $ v_{i} and having weight w_{i} pounds, fill knapsack of capacity w pounds with maximal value.

– Fractional knapsack problem: As 0 − 1 knapsack problem but we can take fractions of items.

– Problems appear very similar, but only fractional knapsack problem can be solved greedily:

∗ Compute value per pound \frac{v_{i}}{w_{i}} for each item

∗ Sort items by value per pound.

∗ Fill knapsack greedily (take objects in order)

⇓

O(n log n) time, easy to show that solution is optimal.

– Example that 0 − 1 knapsack problem cannot be solved greedily:

Note: In fractional knapsack problem we can take \frac{2}{3} of $120 object and get $240 solution.

• 0 − 1 knapsack problem can be solved in time O(n · w) using dynamic-programming (homework).