The Inverse of an Orthogonal Matrix is its Transpose.

November 30, 2010

This is a really sweet deal.  In particular, if we know that our matrix is orthogonal, we can cut down on time finding the inverse significantly.  Combined with the spectral theorem (which states that if the matrix is symmetric, there is an orthogonal matrix S such that S^{-1}AS is a diagonal matrix with entries the eigenvalues) this gives us a tool for finding diagnalizations of matrices. 

All I want to do is state and prove this.  Before we do, let’s just define the transpose really quickly.


Definition: The transpose of an m\times n matrix A is the n\times m matrix denoted A^{T} whose ij-th entry a_{ij} is the ji-th entry of A


In other words, you flip the indices.  Each a_{ij} becomes a_{ji}.  Now, this gives us a really nice way to compute the standard dot product.


Claim: For vectors u,v of the same dimension, we have that u\cdot v = u^{T}v where the right-hand-term is just matrix multiplication.


Why is this?  Well, just think about it for a second and look at this example. 

u = \left(\begin{array}{c} 1 \\ 2 \\ 3\end{array}\right) \, \, \, v = \left(\begin{array}{c} 6 \\ 5 \\ 4\end{array}\right)

Then we have the dot product is just 1(6) + 2(5) + 3(4) = 6 + 10 + 12 = 28.  But what is u^{T}

u^{T} = \left(\begin{array}{ccc} 1 & 2 & 3\end{array}\right)

and matrix multiplication gives us

\left(\begin{array}{ccc} 1 & 2 & 3\end{array}\right)\left(\begin{array}{c} 6 \\ 5 \\ 4\end{array}\right) = 1(6) + 2(5) + 3(4) = 28

same as above.  This should give you a clear picture of what is going on here.  Now the main theorem.


Theorem: A is orthogonal (all of its columns are orthonormal, not just orthogonal), if and only if we have A^{-1} = A^{T}


Proof.  This proof is not as hard as you’d expect, and uses only one main idea to make the proof elegant: the claim above.  We first do a little prep work to get down to the "main matrix" and then the if and only if will follow.  Let’s let A be defined as follows, with each v_{i}\in{\mathbb R}^{n}:

A = \left(\begin{array}{cccc} \, & \, & \, & \, \\ v_{1} & v_{2} & \cdots & v_{n} \\ \, & \, & \, & \, \end{array}\right)

Now, note that A^{T} is equal to almost the same thing, except the columns are now the rows, and we’ve turned them up.  Check this.  It’s equal to:

A^{T} = \left(\begin{array}{ccc} \, & v_{1}^{T} & \, \\ \, & v_{2}^{T} & \, \\ \, & \vdots & \, \\ \, & v_{n}^{T} & \, \end{array}\right)

Now we just multiply these two together.  We obtain, as you can check:

A^{T}A = \left(\begin{array}{ccc} \, & v_{1}^{T} & \, \\ \, & v_{2}^{T} & \, \\ \, & \vdots & \, \\ \, & v_{n}^{T} & \, \end{array}\right)\, \left(\begin{array}{cccc} \, & \, & \, & \, \\ v_{1} & v_{2} & \cdots & v_{n} \\ \, & \, & \, & \, \end{array}\right)

= \left(\begin{array}{cccc} v_{1}^{T}v_{1} & v_{1}^{T}v_{2} & \cdots & v_{1}^{T}v_{n} \\ v_{2}^{T}v_{1} & v_{2}^{T}v_{2} & \cdots & v_{2}^{T}v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ v_{n}^{T}v_{1} & v_{n}^{T}v_{2} & \cdots & v_{n}^{T}v_{n}\end{array}\right)

But, because of our claim above, we can simplify this nightmarish mess.

= \left(\begin{array}{cccc} v_{1}\cdot v_{1} & v_{1}\cdot v_{2} & \cdots & v_{1}\cdot v_{n} \\ v_{2}\cdot v_{1} & v_{2}\cdot v_{2} & \cdots & v_{2}\cdot v_{n} \\ \vdots & \vdots & \ddots & \vdots \\ v_{n}\cdot v_{1} & v_{n}\cdot v_{2} & \cdots & v_{n}\cdot v_{n}\end{array}\right)

This is the "main matrix" that I was describing above.  This matrix is the meat of this proof.  Here’s why.

Suppose that A is orthogonal.  Then v_{i}\cdot v_{i} = 1 for each i, and v_{i}\cdot v_{j} = 0 if i\neq j.  Replacing these values in the "main matrix", we get:

A^{T}A = \left(\begin{array}{cccc} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1\end{array}\right).

Which is the identity matrix.  By the uniqueness of inverses for matrices, this means that A^{-1} = A^{T}

Now suppose the converse is true: that A^{-1} = A^{T}.  Well, what does this say?  It says that for our "main matrix" that the diagonals are all 1 and the other elements are 0; in other words, v_{i}\cdot v_{i} = 1 for each i and v_{i}\cdot v_{j} = 0 if i \neq j.  The latter expression tells us the columns are orthogonal, and the former tells us that these columns are actually orthonormal.  Hence A is an orthogonal matrix.  \Box

14 Responses to “The Inverse of an Orthogonal Matrix is its Transpose.”

  1. sirlandonc said

    This is excellent. Thank you.

  2. sanpol said

    This is magnificent! I was readying myself for some head-banging while trying to find out why the inverse is the transpose… and then you came and saved the day! A bow of gratitude for you, dear sir!

  3. Zach said

    Nice proof. Very easy to follow.

  4. Anonymous said

    it helped a lot and saved our time

  5. prasanna said

    After reading it now I understood it very conveniently. Thanks sir.

  6. Anonymous said

    wow!this is very well written! thank u!

  7. Sairam Subramaniam said

    Vow. Thanks a Ton. This was exactly what I was looking for! :)

    Sairam S

  8. Ramon said

    Great blog! Do you have any tips and hints for aspiring writers?
    I’m planning to start my own blog soon but I’m a little lost on everything.
    Would you recommend starting with a free platform like WordPress or go for a paid option?
    There are so many choices out there that I’m completely
    overwhelmed .. Any recommendations? Thanks!

    • James said

      I almost never comment on here anymore, but I dig when people want to start their own blogs!

      There’s two steps here that I feel are the most important: (1) Only worry about the content; (2) Don’t sweat the small stuff.

      I started on wordpress, and it was fantastic — it’s great blogging practice since it’s all free, it looks pretty, and you can really find your voice with it. It’s also a fairly nice platform to do math on, due to some of the nice LaTeX additions they have. Having said that, at some point, you may want to upgrade to something paid if you want more control over what you’re doing (for me, that took around 4 – 5 years; wordpress was fine in the meantime). But don’t worry about that now. See if you can make good content first. The rest will follow.

      Don’t spend too much time worrying about how your blog looks, what your blog is called, etc. (all this can be changed later!), just start writing.

      So, go. Start writing.

  9. Anonymous said

    Thanks alot.

  10. Anonymous said

    you don’t ‘prove’ the claim though..?

  11. harmyder said

    So, is it inverse or left inverse?

  12. Reblogged this on soksereyoudomlife and commented:
    TD I solve , exercise 4 solved :D

  13. Anonymous said

    Lovely proof, thank you.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: