Java & memory management

Question

Dear StackOverflow,

I'm new to java world from C++ background. I'd like to port some C++ code to Java. The code uses Sparse vectors:

struct Feature{
int index;
double value;
};

typedef std::vector<Feature> featvec_t;

As I understood, if one makes an object, there will be some overhead on memory usage. So naive implementation of Feature will overhead signifiantly when there will be 10-100 millions of Features in a set of featvec_t.

How to represent this structure memory efficiently in Java?

Answer 1

An object in Java (I guess) has:

• sizeof(index)
• sizeof(value)
• sizeof(Class*) <-- pointer to concrete class

So the diference is four bytes from the pointer. If your struct is 4+8=12 bytes it's a 33% overhead... but I can't think other (better) way to do it.

Answer 2

To avoid memory overhead for each entry, you could write a java.util.List<Feature> implementation that wraps arrays of int and double, and builds Feature objects on demand.

To have it resize automatically, you could use TIntArrayList and TDoubleArrayList from GNU trove.

Answer 3

Is the question about space for the struct itself or the sparse vector? Since others have answered the former, I'll shoot for the latter...

There aren't any sparse lists/matrices in the standard Java collections to my knowledge.

You could build an equivalent using a TreeMap keyed on the index.

Answer 4

If memory is really your bottleneck, try storing your data in two separate arrays: int[] index and double[] value.

But in most cases with such big structures performance (time) will be the main issue. Depending on operations mostly performed on your data (insert, delete, get, etc.) you need to choose appropriate data structure to store objects of class Feature. Start your explorations with java.util.Collection interface, its subinterfaces (List, Set, etc) and their implementations provided in java.util package.