views:

245

answers:

2

In the queue class from the Queue module, there are a few methods, namely, qsize, empty and full, whose documentation claims they are "not reliable".

What exactly is not reliable about them?

I did notice that on the Python docs site, the following is said about qsize:

Note, qsize() > 0 doesn’t guarantee that a subsequent get() will not block, nor will qsize() < maxsize guarantee that put() will not block.

I personally don't consider that behavior "unreliable". But is this what is meant by "unreliable," or is there some more sinister defect in these methods?

+2  A: 

I don't know which Queue module you're referring to, please can you provide a link?

One possible source of unreliability: Generally, a queue is read by one thread and written by another. If you are the only thread accessing a queue, then reliable implementations of qsize(), empty() and full() are possible. But once other threads get involved, the return value of these methods might be out-of-date by the time you test it.

user9876
Queue.py, part of the standard library.
cool-RR
There's a Queue.py module in Python's standard library, I added comments to the question to point to its docs. Source is at http://svn.python.org/view/python/trunk/Lib/Queue.py?view=markup .
Alex Martelli
Ah ok, I'd foolishly assumed that Queue couldn't be in the stdlib because it doesn't comply with the PEP8 naming conventions. I guess it's an exception. Thanks for the link!
user9876
+6  A: 

Yes, the docs use "unreliable" here to convey exactly this meaning: for example, in a sense, qsize doesn't tell you how many entries there are "right now", a concept that is not necessarily very meaningful in a multithreaded world (except at specific points where synchronization precautions are being taken) -- it tells you how many entries it had "a while ago"... when you act upon that information, even in the very next opcode, the queue might have more entries, or fewer, or none at all maybe, depending on what other threads have been up to in the meantime (if anything;-).

Alex Martelli
"Unreliable" suggests any number of problems. These functions should be completely reliable for certain use cases (eg. detecting if an insert might block, from a queue with a single producer thread). "Unreliable" is meaningless.
Glenn Maynard
@Alex Martelli: How can the size reported by qsize not be the size right now? After all, the caller of qsize is holding the mutex, which means that it's the only thread accessing the queue in that moment in time. Or are using the term "right now" as in "right _after_ you finished calling qsize"?
cool-RR
It's the size "back when" the caller thread was holding the mutex -- which gets released (in a `finally`) as a part of the `return`; that's an earlier instant than the instant in which the calling code in the caller thread RECEIVES the result. The "right now" is the instant in which the result is being bound to a variable or used in any other way whatsoever: by that time, the result being bound or otherwise used could already be out of date.
Alex Martelli
@Glenn, see my reply comment to your comment on the question: far from meaningless or useless, the extremely concise summaries in question usefully and meaningfully REMIND the reader of the much more detailed explanations in the manual. e.g. even if `full` is true that does NOT mean you WILL block even in a single producer scenario... it might not be true any more the very next instant (in this extremely specialized case you know no other thread will cause the false->true transition, but you don't know about the true->false transition _even in such a artificial scenario_!-).
Alex Martelli