dstefan
03-22-2008, 01:13 PM
Chapter 1, Problem 9:
Let f : {\mathbb R}^3 \to {\mathbb R} given by f(x) = 2 x_1^2 - x_1 x_2 + 3 x_2 x_3 - x_3^2, where x = (x_1, x_2, x_3).
(i) Compute the gradient and Hessian of the function f(x) at the point a = (1, -1, 0), i.e., compute Df(1, -1, 0) and D^2 f(1, -1, 0).
(ii) Show that
f(x) ~=~ f(a)+ Df(a) ~(x - a) + \frac{1}{2} ~(x-a)^t ~D^2 f(a) ~(x-a), where, x, a, and x-a are 3x1 column vectors, i.e., x ~=~ \left( \begin{array}{c} x_1 \\ x_2 \\ x_3 \end{array} \right); a ~=~ \left( \begin{array}{r} 1 \\ -1 \\ 0 \end{array} \right); x - a ~=~ \left( \begin{array}{c} x_1 - 1 \\ x_2 + 1 \\ x_3 \end{array} \right).
Solution:
Df(x) ~=~ \left( \frac{\partial f}{\partial x_1}(x) ~~\frac{\partial f}{\partial x_2}(x) ~~\frac{\partial f}{\partial x_n}(x) \right) ~=~ \left( 4x_1-x_2, ~~-x_1+3x_3, ~~3x_2-2x_3 \right)
D^2 f(x) ~=~ \left( \begin{array}{ccc} \frac{\partial^2 f}{\partial x_1^2}(x) & \frac{\partial^2 f}{\partial x_2 \partial x_1}(x) & \frac{\partial^2 f}{\partial x_3 \partial x_1 }(x) \\ \frac{\partial^2 f}{\partial x_1 \partial x_2}(x) & \frac{\partial^2 f}{\partial x_2^2}(x) & \frac{\partial^2 f}{\partial x_3 \partial x_2}(x) \\ \frac{\partial^2 f}{\partial x_1 \partial x_3}(x) & \frac{\partial^2 f}{\partial x_2 \partial x_3}(x) & \frac{\partial^2 f}{\partial x_3^2}(x) \end{array} \right)
(i) Df(a) ~=~ Df(1, -1, 0) ~=~ (5, ~-1, ~-3 )
D^2 f(a) ~=~ D^2 f(1, -1, 0) ~=~ \left( \begin{array}{ccc} 4 & -1 & 0 \\ -1 & 0 & 3 \\ 0 & 3 & -2 \end{array} \right)
(ii) f(a) = f(1, -1, 0) = 3;
f(a)+ Df(a) ~(x - a) + \frac{1}{2} ~(x-a)^t ~D^2 f(a) ~(x-a) ~=~ 3 ~+~ (5, ~-1, ~-3 ) ~\left( \begin{array}{c} x_1 - 1 \\ x_2 + 1 \\ x_3 \end{array} \right) ~+~ \frac{1}{2} ~\left( x_1 - 1, x_2 + 1, x_3 \right) ~\left( \begin{array}{ccc} 4 & -1 & 0 \\ -1 & 0 & 3 \\ 0 & 3 & -2 \end{array} \right) ~\left( \begin{array}{c} x_1 - 1 \\ x_2 + 1 \\ x_3 \end{array} \right)
~=~ 3 ~+~ \left( 5x_1 - x_2 - 3x_3 - 6 \right) ~+~ \left( 2x_1^2 - 5 x_1 - x_1x_2 + x_2 + 3x_2 x_3 + 3x_3 - x_3^2 + 3 \right) ~=~ 2x_1^2 - x_1x_2 + 3x_2 x_3 - x_3^2 ~=~ f(x)
Let f : {\mathbb R}^3 \to {\mathbb R} given by f(x) = 2 x_1^2 - x_1 x_2 + 3 x_2 x_3 - x_3^2, where x = (x_1, x_2, x_3).
(i) Compute the gradient and Hessian of the function f(x) at the point a = (1, -1, 0), i.e., compute Df(1, -1, 0) and D^2 f(1, -1, 0).
(ii) Show that
f(x) ~=~ f(a)+ Df(a) ~(x - a) + \frac{1}{2} ~(x-a)^t ~D^2 f(a) ~(x-a), where, x, a, and x-a are 3x1 column vectors, i.e., x ~=~ \left( \begin{array}{c} x_1 \\ x_2 \\ x_3 \end{array} \right); a ~=~ \left( \begin{array}{r} 1 \\ -1 \\ 0 \end{array} \right); x - a ~=~ \left( \begin{array}{c} x_1 - 1 \\ x_2 + 1 \\ x_3 \end{array} \right).
Solution:
Df(x) ~=~ \left( \frac{\partial f}{\partial x_1}(x) ~~\frac{\partial f}{\partial x_2}(x) ~~\frac{\partial f}{\partial x_n}(x) \right) ~=~ \left( 4x_1-x_2, ~~-x_1+3x_3, ~~3x_2-2x_3 \right)
D^2 f(x) ~=~ \left( \begin{array}{ccc} \frac{\partial^2 f}{\partial x_1^2}(x) & \frac{\partial^2 f}{\partial x_2 \partial x_1}(x) & \frac{\partial^2 f}{\partial x_3 \partial x_1 }(x) \\ \frac{\partial^2 f}{\partial x_1 \partial x_2}(x) & \frac{\partial^2 f}{\partial x_2^2}(x) & \frac{\partial^2 f}{\partial x_3 \partial x_2}(x) \\ \frac{\partial^2 f}{\partial x_1 \partial x_3}(x) & \frac{\partial^2 f}{\partial x_2 \partial x_3}(x) & \frac{\partial^2 f}{\partial x_3^2}(x) \end{array} \right)
(i) Df(a) ~=~ Df(1, -1, 0) ~=~ (5, ~-1, ~-3 )
D^2 f(a) ~=~ D^2 f(1, -1, 0) ~=~ \left( \begin{array}{ccc} 4 & -1 & 0 \\ -1 & 0 & 3 \\ 0 & 3 & -2 \end{array} \right)
(ii) f(a) = f(1, -1, 0) = 3;
f(a)+ Df(a) ~(x - a) + \frac{1}{2} ~(x-a)^t ~D^2 f(a) ~(x-a) ~=~ 3 ~+~ (5, ~-1, ~-3 ) ~\left( \begin{array}{c} x_1 - 1 \\ x_2 + 1 \\ x_3 \end{array} \right) ~+~ \frac{1}{2} ~\left( x_1 - 1, x_2 + 1, x_3 \right) ~\left( \begin{array}{ccc} 4 & -1 & 0 \\ -1 & 0 & 3 \\ 0 & 3 & -2 \end{array} \right) ~\left( \begin{array}{c} x_1 - 1 \\ x_2 + 1 \\ x_3 \end{array} \right)
~=~ 3 ~+~ \left( 5x_1 - x_2 - 3x_3 - 6 \right) ~+~ \left( 2x_1^2 - 5 x_1 - x_1x_2 + x_2 + 3x_2 x_3 + 3x_3 - x_3^2 + 3 \right) ~=~ 2x_1^2 - x_1x_2 + 3x_2 x_3 - x_3^2 ~=~ f(x)