The Joys of Eigendecomposition

I’m currently working on my dissertation, part of which is implementing a dimensionality reduction technique called Linear Discriminant Analysis (LDA). As far as d-r techniques go it’s relatively straightforward, and it was, mainly relatively straightforward to implement. The biggest problem was working out how to calculate the eigendecomposition (that is the eigenvectors and eigenvalues) of two matrices that satisfy the equation

This is implemented in MATLAB as the eig(A, B) function, easy. Unfortunately none of the matrix libraries that are available for Java (such as JAMA, JMT, etc) seemed to implement a function that could solve this eigendecomposition. LAPACK does, kind of. You can get Java bindings of LAPACK (called, suprisingly, JLAPACK), but when I tried using it nothing happened. The function returned succesful, but it just didn’t work.

So it was a case of back to the drawing board. After some research I found that the above equation was synonymous to solving the eigendecomposition of the matrix

Why couldn’t I have found this out earlier? All I needed to do was inverse one of the matrices, multiply them and then perform eigendecomposition on the resultant matrix to get the eigenvalues and eigenvectors. Easy. All it took was 8 hours of painstaking working and re-working of code and equations. If I’d started with the maths first and then worked on the implementation maybe I could have saved myself a lot of hassle.