Matrix Algebra

Orthogonal Projection

Orthogonal projection. The inner product $\langle\mathbf{x},\mathbf{y}\rangle$ is viewed as $\Vert\mathbf{x}\Vert\cdot\Vert\mathbf{y}\Vert\cos\theta$ where $\theta$ is referred as the angle between $\mathbf{x}$ and $\mathbf{y}$ . In particular when $\mathbf{u}$ is a unit vector $\langle\mathbf{u},\mathbf{y}\rangle = \Vert\mathbf{y}\Vert\cos\theta$ becomes the length of the adjacent side of the right triangle as shown below.

Then $\langle\mathbf{u},\mathbf{y}\rangle \mathbf{u}$ becomes the adjacent vector, and called the orthogonal projection of $\mathbf{y}$ onto $\mathbf{u}$ .

Projection and linear combination. We begin with any nonzero vector $\mathbf{x}$ , and normalize it to the unit vector $\mathbf{u} = \mathbf{x}/\Vert\mathbf{x}\Vert$ . Then we define the projection onto $\mathbf{x}$ by

$\displaystyle \hat{\mathbf{y}} = \dfrac{\langle\mathbf{x},\mathbf{y}\rangle}{\langle\mathbf{x},\mathbf{x}\rangle} \mathbf{x}$

Observe that $\langle\mathbf{y}-\hat{\mathbf{y}}, \mathbf{x}\rangle = 0$ ; thus, the vector $\mathbf{y}-\hat{\mathbf{y}}$ is orthogonal to $\mathbf{x}$ .

Let $\{\mathbf{v}_1,\ldots,\mathbf{v}_k\}$ be an orthogonal set, and let be the subspace spanned by $\mathbf{v}_1, \ldots, \mathbf{v}_k$ . Then we can introduce the projection of $\mathbf{y}$ onto by

$\displaystyle \hat{\mathbf{y}} = \dfrac{\langle\mathbf{v}_1,\mathbf{y}\rangle... ...v}_k,\mathbf{y}\rangle}{\langle\mathbf{v}_k,\mathbf{v}_k\rangle} \mathbf{v}_k$

so that the vector $\mathbf{y}-\hat{\mathbf{y}}$ is orthogonal to all $\mathbf{v}_i$ 's. In particular, if $\mathbf{y}\in$ span $\{\mathbf{v}_1,\ldots,\mathbf{v}_k\}$ , then $\mathbf{y}-\hat{\mathbf{y}} = \mathbf{0}$ and $\hat{\mathbf{y}}$ represents a linear combination of $\mathbf{v}_1, \ldots, \mathbf{v}_k$ .

EXAMPLE 4. Let $\mathbf{v}_1 = \left[\begin{array}{c} 3 \\ 1 \\ 1 \end{array}\right]$ , $\mathbf{v}_2 = \left[\begin{array}{c} -1 \\ 2 \\ 1 \end{array}\right]$ , and $\mathbf{v}_3 = \left[\begin{array}{c} -1/2 \\ -2 \\ 7/2 \end{array}\right]$ be vectors in $\mathbb{R}^3$ . Show that $\mathbf{v}_1$ , $\mathbf{v}_2$ , and $\mathbf{v}_3$ forms an orthogonal basis for $\mathbb{R}^3$ . Express $\mathbf{y} = \left[\begin{array}{c} 6 \\ 1 \\ -8 \end{array}\right]$ as a linear combination of the basis.