Connexions

Site Feedback

Matrix Diagonalization

By Michael Haag

From our understanding of eigenvalues and eigenvectors we have discovered several things about our operator matrix, AA. We know that if the eigenvectors of AA span Cn n and we know how to express any vector xx in terms of v 1 v 2 v n v 1 v 2 v n , then we have the operator AA all figured out. If we have AA acting on xx, then this is equal to AA acting on the combinations of eigenvectors. Which we know proves to be fairly easy!

We are still left with two questions that need to be addressed:

  1. When do the eigenvectors v 1 v 2 v n v 1 v 2 v n of AA span Cn n (assuming v 1 v 2 v n v 1 v 2 v n are linearly independent)?
  2. How do we express a given vector xx in terms of v 1 v 2 v n v 1 v 2 v n ?

Answer to Question #1

Question #1:
When do the eigenvectors v 1 v 2 v n v 1 v 2 v n of AA span Cn n ?
If AA has nn distinct eigenvalues i ,ij: λ i λ j i i j λ i λ j where ii and jj are integers, then AA has nn linearly independent eigenvectors v 1 v 2 v n v 1 v 2 v n which then span Cn n .
Aside:
The proof of this statement is not very hard, but is not really interesting enough to include here. If you wish to research this idea further, read Strang, G., "Linear Algebra and its Application" for the proof.
Furthermore, nn distinct eigenvalues means det(AλI)= c n λn+ c n 1 λn1++ c 1 λ+ c 0 =0 A λ I c n λ n c n 1 λ n 1 c 1 λ c 0 0 has nn distinct roots.

Answer to Question #2

Question #2:
How do we express a given vector xx in terms of v 1 v 2 v n v 1 v 2 v n ?
We want to find α 1 α 2 α n C α 1 α 2 α n such that
x= α 1 v 1 + α 2 v 2 ++ α n v n x α 1 v 1 α 2 v 2 α n v n
1
In order to find this set of variables, we will begin by collecting the vectors v 1 v 2 v n v 1 v 2 v n as columns in a n×n matrix V V. V=( v 1 v 2 v n ) V v 1 v 2 v n Now Equation 1 becomes x=( v 1 v 2 v n ) α 1 α n x v 1 v 2 v n α 1 α n or x=Vα x V α which gives us an easy form to solve for our variables in question, αα: α=V-1x α V -1 x Note that VV is invertible since it has nn linearly independent columns.

Aside

Let us recall our knowledge of functions and their basis and examine the role of VV. x=Vα x V α x 1 x n =V α 1 α n x 1 x n V α 1 α n where αα is just xx expressed in a different basis: x= x 1 100+ x 2 010++ x n 001 x x 1 1 0 0 x 2 0 1 0 x n 0 0 1 x= α 1 v 1 + α 2 v 2 ++ α n v n x α 1 v 1 α 2 v 2 α n v n VV transforms xx from the standard basis to the basis v 1 v 2 v n v 1 v 2 v n

Matrix Diagonalization and Output

We can also use the vectors v 1 v 2 v n v 1 v 2 v n to represent the output, b b, of a system: b=Ax=A( α 1 v 1 + α 2 v 2 ++ α n v n ) b A x A α 1 v 1 α 2 v 2 α n v n Ax= α 1 λ 1 v 1 + α 2 λ 2 v 2 ++ α n λ n v n =b A x α 1 λ 1 v 1 α 2 λ 2 v 2 α n λ n v n b Ax=( v 1 v 2 v n ) λ 1 α 1 λ 1 α n A x v 1 v 2 v n λ 1 α 1 λ 1 α n Ax=VΛα A x V Λ α Ax=VΛV-1x A x V Λ V -1 x where ΛΛ is the matrix with the eigenvalues down the diagonal: Λ=( λ 1 00 0 λ 2 0 00 λ n ) Λ λ 1 0 0 0 λ 2 0 0 0 λ n Finally, we can cancel out the x x and are left with a final equation for AA: A=VΛV-1 A V Λ V -1

Interpretation

For our interpretation, recall our key formulas: α=V-1x α V -1 x b=i α i λ i v i b i α i λ i v i We can interpret operating on xx with AA as: x 1 x n α 1 α n λ 1 α 1 λ 1 α n b 1 b n x 1 x n α 1 α n λ 1 α 1 λ 1 α n b 1 b n where the three steps (arrows) in the above illustration represent the following three operations:

  1. Transform x x using V-1 V -1 , which yields αα
  2. Multiplication by Λ Λ
  3. Inverse transform using V V, which gives us bb
This is the paradigm we will use for LTI systems!

Figure 1: Simple illustration of LTI system!