Lecture 14: Dynamic Programming

(CLRS 15.2-15.3)

June 6th, 2002

1 Dynamic programming

• We have previously discussed how divide-and-conquer can often be used to obtain efficient algorithms.

– Examples: matrix multiplication, merge-sort, quick-sort,....

• Sometimes direct use of divide-and-conquer does not yield efficient algorithms—in fact, sometimes it results in really bad algorithms.

• Today we will discuss a technique which can often be used to improve upon an inefficient divide-and-conquer algorithm.

– The technique is called ”Dynamic programming”. It is neither especially ’dynamic’ nor especially ’programming’ related.

– We will discuss dynamic programming by looking at an example.

1.1 Matrix-chain multiplication

• Problem: Given a sequence of matrices A_{1}, A_{2}, A_{3}, ..., A_{n}, find the best way (using the minimal number of multiplications) to compute their product.

– Isn’t there only one way? ((⋯ ((A_{1} · A_{2}) · A_{3}) ⋯) · A_{n})

– No, matrix multiplication is associative.

e.g. A_{1} · (A_{2} · (A_{3} · (⋯ (A_{n−1} · A_{n}) ⋯))) yields the same matrix.

– Different multiplication orders do not cost the same:

∗ Multiplying p × q matrix A and q × r matrix B takes p · q · r multiplications; result is a p × r matrix.

∗ Consider multiplying 10 × 100 matrix A_{1} with 100 × 5 matrix A_{2} and 5 × 50 matrix A_{3}.

– (A_{1} · A_{2}) · A_{3} takes 10 · 100 · 5 + 10 · 5 · 50 = 7500 multiplications.

– A_{1} · (A_{2} · A_{3}) takes 100 · 5 · 50 + 10 · 50 · 100 = 75000 multiplications.

• In general, let A_{i} be p_{i−1} × p_{i} matrix.

– A_{1}, A_{2}, A_{3}, …, A_{n} can be represented by p_{0}, p_{1}, p_{2}, p_{3}, …, p_{n}

• Let m(i, j) denote minimal number of multiplications needed to compute A_{i} · A_{i+1} ⋯ A_{j}

– We want to compute m(1, n).

• Divide-and-conquer solution/recursive algorithm:

– Divide into j − i − 1 subproblems by trying to set parenthesis in all j − i − 1 positions. (e.g. (A_{i} · A_{i+1} ⋯ A_{k}) · (A_{k+1} ⋯ A_{j}) corresponds to multiplying p_{i−1} × p_{k} and p_{k} × p_{j} matrices.)

– Recursively find best way of solving sub-problems. (e.g. best way of computing A_{i} · A_{i+1} ⋯ A_{k} and A_{k+1} · A_{k+2} ⋯ A_{j})

– Pick best solution.

• Algorithm expressed in terms of m(i, j):

m(i, j) = \{ \begin{matrix}0 & If i = j \\ min_{i≤k<j}\{m(i, k) + m(k + 1, j) + p_{i−1} · p_{k} · p_{j}\} & If i < j \end{matrix}

• Program:

Matrix-chain(i, j)

IF i = j THEN return 0 m(i, j) = ∞ FOR k = i TO j − 1 DO

q = Matrix-chain(i, k) + Matrix-chain(k + 1, j) +p_{i−1} · p_{k} · p_{j}
IF q < m(i, j) THEN m(i, j) = q

OD

Return m(i, j) END Matrix-chain

Return Matrix-chain(1, n)

• Running time:

T (n) = ∑_{k=1}^{n−1}(T (k) + T (n − k) + O(1))
= 2 · ∑_{k=1}^{n−1} T (k) + O(n)
≥ 2 · T (n − 1)
≥ 2 · 2 · T (n − 2)
≥ 2 · 2 · 2 …
= 2^{n}

• Problem is that we compute the same result over and over again. – Example: Recursion tree for Matrix-chain(1, 4)

We for example compute Matrix-chain(3, 4) twice

• Solution is to ”remember” values we have already computed in a table—memorization

Matrix-chain(i, j)

IF i = j THEN return 0 IF m(i, j) < ∞ THEN return m(i, j) /* This line has changed */ FOR k = i to j − 1 DO

q = Matrix-chain(i, k) + Matrix-chain(k + 1, j)+p_{i−1} · p_{k} · p_{j}

IF q < m(i, j) THEN m(i, j) = q

OD return m(i, j)

END Matrix-chain

FOR i = 1 to n DO

FOR j = i to n DO

m(i, j) = ∞ OD

OD

return Matrix-chain(1, n)

• Running time:

– Θ(n^{2}) different calls to matrix-chain(i, j).

– The first time a call is made it takes O(n) time, not counting recursive calls.

– When a call has been made once it costs O(1) time to make it again.

⇓ O(n^{3}) time

– Another way of thinking about it: Θ(n^{2}) total entries to fill, it takes O(n) to fill one.

1.2 Alternative view of Dynamic Programming

• Often (including in the book) dynamic programming is presented in a different way; As filling up a table from the bottom.

• Matrix-chain example: Key is that m(i, j) only depends on m(i, k) and m(k + 1, j) where i ≤ k < j ⇒ if we have computed them, we can compute m(i, j)

– We can easily compute m(i, i) for all 1 ≤ i ≤ n (m(i, i) = 0)

– Then we can easily compute m(i, i + 1) for all 1 ≤ i ≤ n − 1 m(i, i + 1) = m(i, i) + m(i + 1, i + 1) + p_{i−1} · p_{i} · p_{i+1}

– Then we can compute m(i, i + 2) for all 1 ≤ i ≤ n − 2 m(i, i + 2) = min{m(i, i) + m(i + 1, i + 2) + p_{i−1} · p_{i} · p_{i+2}, m(i, i + 1) + m(i + 2, i + 2) + p_{i−1} · p_{i+1} · p_{i+2}}

⋮

– Until we compute m(1, n)

– Computation order:

• Program:

FOR i = 1 to n DO

m(i, i) = 0

OD FOR l = 1 to n − 1 DO

FOR i = 1 to n − l DO

j = i + l m(i, j) = ∞ FOR k = 1 to j − 1 DO

q = m(i, k) + m(k + 1, j) + p_{i−1} · p_{k} · p_{j}
IF q < m(i, j) THEN m(i, j) = q

OD

OD

OD • Analysis:

– O(n^{2}) entries, O(n) time to compute each ⇒ O(n^{3}).

• Note:

– I like recursive (divide-and-conquer) thinking.

– Book seems to like table method better.

– I like divide-and-conquer because one does not need to get new idea (write new program)— just use table!