Linear Algebra: Invariant Subspaces and Why I Think They Are Pret Cool.
July 7, 2010
We’ve been talkin’ a lot about group theory and the beginnings of topology, but we really should go back to linear algebra for just a bit and talk about some important concepts! Secretly, we will be applying these when we go over representation theory, but, for now, they’re just going to be points of interest.
First, let’s review: a vector space is this really nice space that has a whole bunch of nice additive and scalar multiplicative properties, as well as a few other coo’ things. We usually call our vector spaces or or . Pret much any capital italics letter around the end of the alphabet will do. A subspace, of the vector space is a subset of which is also a vector space. We don’t have very good notation, so I usually write “ is a subspace.”
So, now we have maps between vector spaces. If you forget what all of this is about, go back to here and read up! These maps are called linear maps, and they have a really nice property: if is a linear map and and (where is our underlying field), then we have:
or, more concisely, .
Of course, needs to go from a vector space to another space, and we write that like this: meaning goes from the vector space to the vector space .
Now, what happens if we have an element and we want to apply a linear map to it? Actually, let’s given an example of this: let’s have our vector space be and let’s let our linear map and be defined by
So, switches the first and second coordinate. You can check that this is actually linear. What if we want to apply this to the element ? Well, this is relatively easy: we have .
But we can think of this some other way: we can say that
which is, in turn,
This is now written in terms of a basis, and we can compute this by applying our linear map to the basis. Yeah, this makes it way more complicated looking, but now we can easily express every element in terms of something like this.
Now, how can we talk about this? We can say that, applying to gives us something that “switches” the first and second coordinate. What about the third coordinate? It stays the same! So, we could really break up this vector space into two parts:
where this last isn’t really going to be affected by the transform. Therefore, we can really just define this transform on the first two parts of the vector space and just kind of translate the last part over via the identity map.
Now, this seems really trivial, right? But what if we had a general vector space and a really complicated map? Wouldn’t you want to break it down as much as possible? Of course you would! In the previous example, we noticed that was kind of “closed” in the first two coordinates — yes, they swapped with each other, but the third coordinate didn’t even come into play! It’s almost like having two mixing bowls and putting eggs and flour into one and water into the other: the water doesn’t go into the eggs and flour, and the eggs and flour keep to themselves as well. This is the notion of an invariant subspace.
Specifically, an invariant subspace (also called an invariant subspace with respect to the linear map T) is a subspace such that . In other words, after applying a linear map to , every element stays in .
Now, wouldn’t it look really nice if we had, for our vector space , some kind of partition
Wouldn’t that be cute? Because the map would act on the spaces in the same sort of way that it would act on the elements? Yeah, well, sorry to rain on your parade, but it actually doesn’t work as nicely as that all the time.
So, let’s dry our eyes and think about what we can do in general. Let’s think about the simplest kind of invariant subspace, shall we? If a vector space is, say, 5 dimensions, what would we ideally want to break it up into? Probably five 1-dimensional subspaces, right? In this way, we’d be able to operate on each one individually, and our lives would be so damn good. We’d live like kings. One-dimensional subspaces (like one-dimensional men) are extremely easy to work with, and, therefore, they are the topic of the first part of our investigation of invariant subspaces.
One-Dimensional Invariant Subspaces!
But what would this mean? Suppose we had a one-dimensional subspace. We could obviously find a basis for it: it’s simply some element where is the subspace (since our basis must consist of one element, as is one-dimensional). Now, we have that for is our field. What does this say? It says that is generated by all scalar multiples of . This says exactly the same thing as “ is a basis for “, but this actually gives us a nice way to manipulate elements with a linear map.
Suppose, then, that we have some map , and we have that is invariant under . Good. This means that . But, wait, is generated by , so, in fact, for some (Why is this true?). Cool. So, . Wait, hold on a second. This looks familliar. Let’s just let for convenience, and then we have . WAIT. So, this means that is an eigenvector with corresponding eigenvalue !
Okay, okay, don’t get too excited. We have just figured out that if there is a one-dimensional invariant subspace in , then it is generated by an eigenvalue of . That’s pretty important. But, let’s try this the other way around.
Suppose that is an eigenvalue of with respect to the linear map . This means that . So consider the subspace of defined by . Is this invariant under ? You bet your sweet meats it is! Why? Take an arbitrary element, , and apply to it: . Therefore, this subspace is invariant. We have proved something pretty damn sweet.
Theorem: If is a finite-dimensional vector space and is a linear map, then is an invariant subspace under if and only if is an eigenvector of .
Proof: See above.
Well, this is pretty damn impressive. We know how to find eigenvectors of a map. From this, we can directly decompose into little pieces. In fact, suppose we have a bunch of invariant subspaces generated by the eigenvectors . Do any of these “overlap”? In other words, is the intersection ever nontrivial? This is the same as asking if the set of eigenvectors is linearly independent in (why?), and so this is exactly what we’re going to prove right now!
Theorem: For is a finite-dimensional vector space and is a linear map, then if if the set of distinct eigenvalues corresponding to the set of (non-zero) eigenvectors , then this set of eigenvectors is linearly independent in .
Proof: This clever proof is characteristic of theorems involving linear independence. The plan is to assume linear dependence, get some element as a combo of the others, and then apply some stuff to get a contradiction somehow. Let’s begin.
Let’s assume that is linearly dependent. Then let’s let be the smallest integer such that . In other words, we have that is the smallest indexed eigenvector such that it’s a linear combination of all the eigenvectors with indexes smaller than it. Okay. Okay. This means, in particular, that . Okay, now, (here’s the clever step) apply to both sides. We get:
Make sure you see how we got this! That’s an important part. Okay, now we have this equation that we just derived:
and we also have the original equation:
but, if we take the original equation and multiply to both sides, we get
which is similar to the equation above! Okay, okay, calm down. Now, subtract this equation from the one we derived right before it (so, both of those equations that have on the left side.) and we’ll get:
Make sure you see where we got this! This is an important step. Now, look at this: it’s a linear combination of which is linearly independent (by assumption; otherwise, there’d be a smaller such that and so, if this linear combination is equal to zero, it implies that all of the coefficients are equal to 0. Cool. so, for all we have:
and note that, by assumption, for , so we have that every , which implies that (using our original equation) . But all of the eigenvectors, by assumption, are non-zero! We have reached a contradiction. . .
This means, in particular, if we have and we have for some , then by using that theorem above, we have a whole bunch of subspaces that are invariant! In particular, let’s call , and then we have that each of these eigenvectors creates some one-dimensional invariant subspace. Meaning that,
for some . Let’s take a breather here and just think about this. Okay, cool, we partitioned this space up into one-dimensional spaces with eigenvector bases. Nice. Only one thing kind of bugs me though…
…what’s that on the end? Well, it isn’t anything particularly nice. We just sort of build it. This is a normal kind of thing that we do in linear algebra, so let’s do it explicitly:
We wanna express as a direct sum of these things, so since the eigenvectors are linearly independent and span exactly , we can let these be the beginnings of a basis for . So, now, consider without all of the elements in . If there’s some non-zero element left, call it , pick it, and make it a new basis element, in addition to all of the eigenvectors. Take out all vectors in the space , and then if there’s anything left in minus all this, pick that element out and do the same thing as we did the first time. Eventually, we will span since it is finite dimensional. We get a basis that looks like . It follows that the subspace above is exactly the subspace of spanned by . It’s up to you to show that this is disjoint (except for the 0 element) from every one of the ‘s. Cute.
So, What Did We Just Do?
So, given any random finite-dimensional space and a linear map , we can find the eigenvectors, right? Yeah. Now, we’ve taken that one step farther, and partitioned into a space that looks like with the eigenvectors as basis elements each generating (spanning) each of the spaces. In addition, as a bonus, each of those one-dimensional spaces is invariant under . It is, therefore, the case that we have the very cute and very familiar relation:
which is significantly easier to work with.
After this, we work with spaces that don’t have nice eigenvectors, and we show that some spaces must have one- or two-dimensional invariant subspaces. Eventually, we’re gonna do some examples of such spaces, and we’ll even show some cool stuff with this. For now, you shouldn’t really have too good’a grip on what this “says” — it’s somewhat abstract and kind of confusing. On the other hand, this gives us a really good reason to study eigenvectors and gives us a nice way of thinking about linear transforms. In particular, some linear transforms look very strange if we think about the basis being the standard basis, but, on the other hand, if we switch to an eigenvector basis, it may not look too strange after all! Examples to come.