Site Feedback

# Introduction

Given a line 'l' and a point 'p' in the plane, what's the closest point 'm' to 'p' on 'l'? Figure 1: Figure of point 'p' and line 'l' mentioned above.

Same problem: Let $xx$ and $vv$ be vectors in $R2 2$. Say $∥v∥=1 v 1$. For what value of $αα$ is $∥x−α⁢v∥ 2 x α v 2$ minimized? (what point in span{v} best approximates $xx$?)

The condition is that $x− α ^ ⁢v x α ^ v$ and $α⁢v α v$ are orthogonal.

# Calculating α

How to calculate $α ^ α ^$?

We know that ( $x− α ^ ⁢v x α ^ v$) is perpendicular to every vector in span{v}, so $∀β,∀⁢β:⟨x− α ^ ⁢v,β⁢v⟩=0 β ∀ β x α ^ v β v 0$ $β¯⁢⟨(x,v)⟩− α ^ ⁢β¯⁢⟨(v,v)⟩=0 β x v α ^ β v v 0$ because $⟨v,v⟩=1 v v 1$, so $(⟨(x,v)⟩− α ^ =0)⇒( α ^ =⟨x,v⟩) x v α ^ 0 α ^ x v$ Closest vector in span{v} = $⟨(x,v)⟩⁢v x v v$, where $⟨(x,v)⟩⁢v x v v$ is the projection of $xx$ onto $vv$.

We can do the same thing in higher dimensions.

Exercise 1

Let $V⊂H V H$ be a subspace of a Hilbert space H. Let $x∈H x H$ be given. Find the $y∈V y V$ that best approximates $xx$. i.e., $∥x−y∥ x y$ is minimized.

Solution

1. Find an orthonormal basis $b1…bk b1 … bk$ for $VV$
2. Project $xx$ onto $VV$ using $y=∑i=1k⟨(x,bi)⟩⁢bi y i 1 k x bi bi$ then $yy$ is the closest point in V to x and (x-y) ⊥ V ( $∀v,∀⁢v∈V:⟨x−y,v⟩=0 v ∀ v V x y v 0$

Example 1

$x∈R3 x 3$, $V=span⁢( 1 0 0 )( 0 1 0 ) V span 1 0 0 0 1 0$, $x=( a b c ) x a b c$. So, $y=∑i=12⟨(x,bi)⟩⁢bi=a⁢( 1 0 0 )+b⁢( 0 1 0 )=( a b 0 ) y i 1 2 x bi bi a 1 0 0 b 0 1 0 a b 0$

Example 2

V = {space of periodic signals with frequency no greater than $3⁢ w0 3 w0$}. Given periodic f(t), what is the signal in V that best approximates f?

1. { $1T⁢ei⁢ w0 ⁢k⁢t 1 T w0 k t$, k = -3, -2, ..., 2, 3} is an ONB for V
2. $g⁢t=1T⁢∑k=-33⟨(f⁢t,ei⁢ w0 ⁢k⁢t)⟩⁢ei⁢ w0 ⁢k⁢t g t 1 T k -3 3 f t w0 k t w0 k t$ is the closest signal in V to f(t) ⇒ reconstruct f(t) using only 7 terms of its Fourier series.

Example 3

Let V = {functions piecewise constant between the integers}

1. ONB for V.

where {$bibi$} is an ONB.

Best piecewise constant approximation? $g⁢t=∑i=−∞∞⟨(f, bi )⟩⁢ bi g t i f bi bi$ $⟨f, bi ⟩=∫−∞∞f⁢t⁢ bi ⁢tdt=∫i−1if⁢tdt f bi t f t bi t t i 1 i f t$

Example 4

This demonstration explores approximation using a Fourier basis and a Haar Wavelet basis.See here for instructions on how to use the demo.