Clojure Matrix Representation

Question

What is a good representation for matrices in Clojure? I'm interested in dealing with dense matrices of floating point numbers. The "list of lists" representation springs to mind, but is there something better?

Some criteria for a good representation include:

• Efficiency: They won't be used for constant processing of huge data sets, but I don't want to spend hours calculating results that could have been done in minutes with a better design.
• Java Interoperability: It would be nice to pass the data back and forth between the two languages easily.
• Easy Parallelization: If I can use all the cores available simply by replacing map with pmap, that would be nice.
• Amenable to the use of reduce: It seems like lots of the calculations I'm doing work very well with reduce.
• Ability to Represent Image Scan Lines in Matrix Rows: Not really very important, but would be nice to have.

Any thoughts?

Answer 1

I'm no expert, but here's my opinion anyway :)

list-of-lists is probably the most natural Clojure idiom for representing matrices. This structure also lends itself nicely to map/reduce kinds of operations. Clojure is also pretty efficient at handling sequences - probably better than most alternatives.

I can't swear to this, but I think I've seen Clojure working 3 or all 4 of my CPUs hard on programs I wrote that were functional in style but didn't make any attempt to be parallel. I suspect the compiler is finding some opportunities for parallel processing on its own.

I think the sequence types created by Clojure will work as Lists in Java, or at least be Iterable. That's probably good enough for what you want, though you may run into problems if you try to treat those structures as modifiable in Java.

Lists are best accessed sequentially. If you're planning to jump around a lot in the matrix, a vector-of-vectors may suit you a little better, performance-wise. I suspect that beats using the nth function.

As a former C programmer, I briefly considered that you could implement your matrix as a one-dimensional structure (i.e. a straight sequence or better a vector), and do your own index computations to find the right element. You could use the partition function to step through it... well, that could be made to work but I suspect there are very good reasons not to.

Answer 2

Incanter supplies a wrapper around some of Parallel Colt, including what looks to be a pretty decent implementation of fast, parallelized dense matrices that interface with Clojure's seq-based libraries. I haven't used it, but it should be what you're looking for.

Example.

Answer 3

I am presently using the list of lists approach in cryptovide because its very important for this application to keep things lazy. I am also considering switching to a more efficient approach as long as it kept at least the outward representation lazy.

Answer 4

Rich Hickey’s Clojure is a JVM-based Lisp that represents PersistentVector (not a PersistentList) with a 32-way tree.

If you would like to write your own matrix Type i would use PersistentVector otherwise the best choice is to use Parallel Colt with Incanter.