Multiple threads modifying a collection in Java??

views:

122

answers:

+5 Q:

Multiple threads modifying a collection in Java??

Hi,

The project I am working on requires a whole bunch of queries towards a database. In principle there are two types of queries I am using:

read from excel file, check for a couple of parameters and do a query for hits in the database. These hits are then registered as a series of custom classes. Any hit may (and most likely will) occur more than once so this part of the code checks and updates the occurrence in a custom list implementation that extends ArrayList.
for each hit found, do a detail query and parse the output, so that the classes created in (I) get detailed info.

I figured I would use multiple threads to optimize time-wise. However I can't really come up with a good way to solve the problem that occurs with the collection these items are stored in. To elaborate a little bit; throughout the execution objects are supposed to be modified by both (I) and (II).

I deliberately didn't c/p any code, as it would be big chunks of code to make any sense.. I hope it make some sense with the description above.

Thanks,

+7 A:

In Java 5 and above, you may either use CopyOnWriteArrayList or a synchronized wrapper around your list. In earlier Java versions, only the latter choice is available. The same is true if you absolutely want to stick to the custom ArrayList implementation you mention.

CopyOnWriteArrayList is feasible if the container is read much more often than written (changed), which seems to be true based on your explanation. Its atomic addIfAbsent() method may even help simplify your code.

[Update] On second thought, a map sounds more fitting to the use case you describe. So if changing from a list to e.g. a map is an option, you should consider ConcurrentHashMap. [/Update]

Changing the objects within the container does not affect the container itself, however you need to ensure that the objects themselves are thread-safe.

Péter Török 2010-04-29 10:09:14

CopyOnWriteArrayList is a very slow (and memory hungry) data structure - it also doesn't extend ArrayList so he can use a more effecient class like ConcurrentLinkedQueue.

Gandalf 2010-04-29 16:46:08

Also, as long as you stick to a producer/consumer model (i.e. a thread removes an object from the queue, edits it, then puts it back (or to a different queue)) then the objects themselves have no need to be thread-safe.

Gandalf 2010-04-29 16:53:35

@Gandalf, CopyOnWriteArrayList is slow only on writes. What is described in the post above, to me, is very different from a producer-consumer model, so queues are not an option.

Péter Török 2010-04-30 07:51:19

Thanks to both Peter and Gandalf, and sorry for the late reply. I have been working on a number of other functions for the project. I havent had the chance to try these on as the multithreading was deprioritised in favor of implementation of some other functions. Will get back when/if I get the chance to try these out.Thanks again

posdef 2010-05-26 08:39:00

+2 A:

Just use the new java.util.concurrent packages.

Classes like ConcurrentLinkedQueue and ConcurrentHashMap are already there for you to use and are all thread-safe.

Gandalf 2010-04-29 16:43:06

They work well :)

Chris Dennett 2010-04-29 16:50:42

ansaurus

tags:

views:

answers:

Multiple threads modifying a collection in Java??

related questions