Connexions

Site Feedback

Approximation and Projections in Hilbert Space

By Justin Romberg

Introduction

Given a line 'l' and a point 'p' in the plane, what's the closest point 'm' to 'p' on 'l'?

Figure 1: Figure of point 'p' and line 'l' mentioned above.

Same problem: Let xx and vv be vectors in R2 2 . Say v=1 v 1 . For what value of αα is xαv 2 x α v 2 minimized? (what point in span{v} best approximates xx?)

Figure 2

The condition is that x α ^ v x α ^ v and αv α v are orthogonal.

Calculating α

How to calculate α ^ α ^ ?

We know that ( x α ^ v x α ^ v ) is perpendicular to every vector in span{v}, so β,β:x α ^ v,βv=0 β β x α ^ v β v 0 β¯(x,v) α ^ β¯(v,v)=0 β x v α ^ β v v 0 because v,v=1 v v 1 , so ((x,v) α ^ =0)( α ^ =x,v) x v α ^ 0 α ^ x v Closest vector in span{v} = (x,v)v x v v , where (x,v)v x v v is the projection of xx onto vv.

We can do the same thing in higher dimensions.

Exercise 1

Let VH V H be a subspace of a Hilbert space H. Let xH x H be given. Find the yV y V that best approximates xx. i.e., xy x y is minimized.

Solution

  1. Find an orthonormal basis b1bk b1 bk for VV
  2. Project xx onto VV using y=i=1k(x,bi)bi y i 1 k x bi bi then yy is the closest point in V to x and (x-y) ⊥ V ( v,vV:xy,v=0 v v V x y v 0

Example 1

xR3 x 3 , V=span( 1 0 0 )( 0 1 0 ) V span 1 0 0 0 1 0 , x=( a b c ) x a b c . So, y=i=12(x,bi)bi=a( 1 0 0 )+b( 0 1 0 )=( a b 0 ) y i 1 2 x bi bi a 1 0 0 b 0 1 0 a b 0

Example 2

V = {space of periodic signals with frequency no greater than 3 w0 3 w0 }. Given periodic f(t), what is the signal in V that best approximates f?

  1. { 1Tei w0 kt 1 T w0 k t , k = -3, -2, ..., 2, 3} is an ONB for V
  2. gt=1Tk=-33(ft,ei w0 kt)ei w0 kt g t 1 T k -3 3 f t w0 k t w0 k t is the closest signal in V to f(t) ⇒ reconstruct f(t) using only 7 terms of its Fourier series.

Example 3

Let V = {functions piecewise constant between the integers}

  1. ONB for V.

bi ={1  if  i1t<i0  otherwise   bi 1 i 1 t i 0 where {bibi} is an ONB.

Best piecewise constant approximation? gt=i=(f, bi ) bi g t i f bi bi f, bi =ft bi tdt=i1iftdt f bi t f t bi t t i 1 i f t

Example 4

This demonstration explores approximation using a Fourier basis and a Haar Wavelet basis.See here for instructions on how to use the demo.