Principal Component Analysis in a cluster via MPI

Question

Hi All

I am setting up a set of computers where to run math programs on top of MPI. Do you know whether exist some library doing PCA - Principal Component Analysis using MPI so to use all the resources of the networked pcs? I will have a look at Scalapack, but do you know other libraries? My language is C++ on linux but if there is a good lib also for windows is the same

Thanks

Answer 1

A PCA is a reasonably cheap operation so your ratio of communication (getting data to the nodes) relative to computation (the actual operation, here the PCA) is likely to be relatively poor.

This means that clustering may not be a great solution for this particular problem.

Moreover, PCA is really a linear algebra operation so you are better off looking at optimised BLAS such as ATLAS, Goto, MKL, ... which (these days) can make use of multiple cores giving you implicit parallelism which is easier to use than the explicit parallelism using MPI.

Do not get me wrong -- I really like MPI (and have some tutorials here on using it with R) but you need to keep in mind that not all tools are appropriate for all problems.