Java - When to use Iterators?

Question

Hi all,

I am trying to better understand when I should and should not use Iterators. To me, whenever I have a potentially large amount of data to iterate through, I write an Iterator for it. If it also lends itself to the Iterator interface, then it seems like a win.

I was reading a little bit that there is a lot of overhead with using an Iterator.

A good example of where I used an Iterator was to iterate through a bunch of SQL scripts to execute one query at a time, reading it in, then executing it.

Is there another performance trade off I should be aware of? Before I used iterators, I would read the entire String of SQL commands to execute into an ArrayList, and the iterate through that. If the import is rather large (like for geolocation data, then the server tends to get bogged down).

Walter

Answer 1

Pretty good answer already here:

http://stackoverflow.com/questions/1879255/traditional-for-loop-vs-iterator-in-java

Answer 2

A good example of where I used an Iterator was to iterate through a bunch of SQL scripts to execute one query at a time, reading it in, then executing it.

In this scenario the overhead of an Iterator is likely dwarfed by the time it takes to run the queries.

Before I used iterators, I would read the entire String of SQL commands to execute into an ArrayList, and the iterate through that. If the import is rather large (like for geolocation data, then the server tends to get bogged down).

Any particular reason you need to collect them all into an ArrayList? You could just execute them one by one as you read the statements.

Iterators are particularly suited for streaming cases where the data is loaded/created on the fly/lazily. They do not require the data to be completely in memory upfront.

Answer 3

I think your question is when you should 'stream' input rather than load it all into memory and the process it. It's not really a question of using Iterator or not I think.

"It depends," of course, though in your given example it sounds like streaming the input rather than loading it all into memory is a clear win, so iterate indeed.

The benefit of loading into memory is usually that the code is simpler, and maybe you get some benefit from loading large chunks into memory at once rather than reading bits at a time. The benefit of "streaming" is that you limit your memory requirements, and, gain performance associated with that.

As a very crude rule of thumb, I wouldn't load anything like this into memory unless I were sure it was under 100K or so.