views:

320

answers:

2

I'm doing some heavy processing (building inverse indices) using ints/ longs in Java.

I've determined that (un)boxing of standard java.collections maps takes a big portion of the total processing time. (as compared to a similiar implementation using arrays, which I can't use due to memory constraints).

I'm looking for a fast 3rd-party implementation (or any implementation at all for that matter) that could support the following structure:

Map with characteristics:

-keys in the map are sparse (+/- 10.000.000 keys in range [0,2^64] -values are always appended to the end of the list -fast insert (amortized O(1) if possible) -fast iteration in key-order.

I've looked at trove, fastutil, etc. but couldn't find a multimap implementation using primitives (only normal maps)

any help is appreciated.

Thanks, Geert-Jan

A: 

Have you considered implementing the multi-portion yourself using a primitive long -> Object-map and primitive int-set as the value?

Tuure Laurinolli
yeah I was thinking about that..Never done something like this. What would be the overhead against having a primitive long --> long map? I'm asking because I'm thinking of the alternative of decoding {int} as 1 long (using some bit-manipulation, thereby eliminating the 'multi'-part. This alterative would however require me to do a lookup of the value each time I insert / append a new value so the new decoded value can be calculated.. (hope that makes sense) From the top what would you say would be more performant?
Geert-Jan
I honestly don't know. Perhaps you should do a quick test?
Tuure Laurinolli
yeah. I've gone with my 2nd approach, which turns out to be lighning fast. (Haven't compared though) I'm flagging your answer since (in combination with the comments) it was the most helpful. Thanks.
Geert-Jan
A: 

What about Google collections library? http://code.google.com/p/google-collections/

Dmitry
We don't support primitive-backed collections, so every int will have to be boxed. Sounds like that would be a problem..
Kevin Bourrillion