What is the significance of abstract linear algebra in machine learning/computer vision research?

Question

I am a computer science research student working in application of Machine Learning to solve Computer Vision problems.

Since, lot of linear algebra(eigen-values, SVD etc.) comes up when reading Machine Learning/Vision literature, I decided to take a linear algebra course this semester.

Much to my surprise, the course didn't look at all like Gilbert Strang's Applied Linear algebra(on OCW) I had started taking earlier. The course textbook is Linear Algebra by Hoffman and Kunze. We started with concepts of Abstract algebra like groups, fields, rings, isomorphism, quotient groups etc. And then moved on to study "theoretical" linear algebra over finite fields, where we cover proofs for important theorms/lemmas in the following topics:

Vector spaces, linear span, linear independence, existence of basis. Linear transformations. Solutions of linear equations, row reduced echelon form, complete echelon form,rank. Minimal polynomial of a linear transformation. Jordan canonical form. Determinants. Characteristic polynomial, eigenvalues and eigenvectors. Inner product space. Gram Schmidt orthoganalization. Unitary and Hermitian transformations. Diagonalization of Hermitian transformations.

I wanted to understand if there is any significance/application of understanding these proofs in machine learning/computer vision research or should I be better off focusing on the applied Linear Algebra?

Answer 1

In my experience (YMMV) linear algebra is very much used as a tool in Machine Learning and Computer Vision, with little interest in the underlying mathematics.

That said, you'll need a more mathematical understanding than, say, a games programmer or a basic researcher in the natural sciences. Basically you'll need to know enough to take a method like PCA apart and put it back together again, but you don't need to worry about your matrices containing anything other than real values.

It sounds like this course is starting off deeper and more general than you want, but the list of topics suggests that you are moving in the right direction. So long as some attention is given to how to translate the mathematical methods into the computer. It's one thing to suggest that an SVD decomposition exists, and another to actually compute it.

The deep mathematical understanding won't hurt, but there's no guarantee that all of it will be useful to you.