tags:

views:

329

answers:

4
+7  Q: 

JVM (embarrasingly) parallel processing libraries/tools

I am looking for something that will make it easy to run (correctly coded) embarrassingly parallel JVM code on a cluster (so that I can use Clojure + Incanter).

I have used Parallel Python in the past to do this. We have a new PBS cluster and our admin will soon set up IPython nodes that use PBS as the backend. Both of these systems make it almost a no-brainer to run certain types of code in a cluster.

I made the mistake of using Hadoop in the past (Hadoop is just not suited to the kind of data that I use) - the latency made even small runs execute for 1-2 minutes.

Is JPPF or Gridgain better for what I need? Does anyone here have any experience with either? Is there anything else you can recommend?

A: 

Look at Skandium

Pangea  2010-03-24 16:25:45
Looks promising, but GPL is a pain in the ass.
Robert Harvey  2010-03-27 19:44:44
A: 

I have heard that Scala has better support for concurrency than Java does.

mlaverd  2010-03-26 04:51:04
Question refers to Clojure, not Scala. Both have good concurrency libraries, and both compile to Java bytecode, so JVM-aware tools are the relevant answer.
Stuart Sierra  2010-03-27 19:33:03
+2  A: 

Clojure is reported to work on Terracotta, subject to some patching.

Stuart Sierra  2010-03-27 19:30:15
A: 

Check out cascalog - http://github.com/nathanmarz/cascalog

simon-says  2010-07-19 19:07:38

related questions

Java Time Zone is messed up
Eclipse on win64
Automate builds for Java RCP for deployment with JNLP
Why are professors or schools picking Java over C++ to teach to students?
Is there a real benefit of using J#?
Public/Popular Websites using JavaServer Faces
Why can't I use a try block around my super() call?
Accessing post variables using Java Servlets
Personal Linux web server
Is this really widening vs autoboxing?
How can I Java webstart multiple, dependent, native libraries?
Why can't I call toString() on a Java primitive?
How do I use Java to read from a file that is actively being written?
What code analysis tools do you use for your Java projects?
IllegalArgumentException or NullPointerException for a null parameter?
How do I configure and communicate with a serial port?
What is the best way to parse strings in Java
Getting started with a custom JXTA PeerGroup
Creating a custom button in Java
How to get started "writing" a code coverage tool?
Which Build-/Configuration Management Tool?
What is the difference between an int and an Integer in Java/C#?
What is the meaning of the type safety warning in certain Java generics casts?
How would you access Object properties from within an object method?
Converting CSV File to XML in Java