Deadlock in ruby code using SizedQueue

Question

I think I'm running up against a fundamental misunderstanding on my part of how threading works in ruby and I'm hoping to get some insight.

I'd like to have a simple producer and consumer. First, a producer thread that pulls lines from a file and sticks them into a SizedQueue; when those run out, stick some tokens on the end to let the consumer(s) know things are done.

require 'thread'
numthreads = 2
filename = 'edition-2009-09-11.txt'

bq = SizedQueue.new(4)
producerthread = Thread.new(bq) do |queue|
  File.open(filename) do |f|
    f.each  do |r|
      queue << r
    end
  end
  numthreads.times do
    queue << :end_of_producer
  end
end

Now a few consumers. For simplicity, let's have them do nothing.

consumerthreads = []

numthreads.times do
  consumerthreads << Thread.new(bq) do |queue|
    until (line = queue.pop) === :end_of_producer
       # do stuff in here
    end
  end
end

producerthread.join
consumerthreads.each {|t| t.join}

puts "All done"

My understanding is that (a) the producer thread will block once the SizedQueue is full and eventually get back to filling it up, and (b) the consumer threads will pull from the SizedQueue, blocking when it empties, and eventually finish.

But under ruby1.9 (ruby 1.9.1p243 (2009-07-16 revision 24175) [i386-darwin9]) I get a deadlock error on the joins. What's going on here? I just don't see where there's any interaction between the threads except via the SizedQueue,which is supposed to be thread-safe.

Any insight would be much-appreciated.

Answer 1

Your understanding is correct and your code works on my machine, on a slightly newer version of Ruby (both ruby 1.9.2dev (2009-08-30 trunk 24705) [i386-darwin10.0.0] and ruby 1.9.2dev (2009-08-30 trunk 24705) [i386-darwin10.0.0])