Streaming data from Sinatra/Rack application

Question

I am trying to stream textual data (XML/JSON) from a Ruby (1.9.1p378) Sinatra (1.0) Rack (1.2.1) application. The suggested solutions (e.g. http://stackoverflow.com/questions/3027435/is-there-a-way-to-flush-html-to-the-wire-in-sinatra) do not seem to work - the server just blocks when I yield elements of some infinite stream (e.g. from %w(foo bar).cycle). I tried webrick and thin as servers.

Any suggestions on getting this done? Should I use http://sinatra.rubyforge.org/api/classes/Sinatra/Streaming.html and if so how would I use it in my application?

Answer 1

Neither Webrick nor Thin support streaming that way. You could try Mongrel or Unicorn. If you want to use Thin or Rainbows!, you have to hook into the event loop in order to achieve streaming:

require 'sinatra'

class Stream
  include EventMachine::Deferrable
  def initialize
    @counter = 0
  end

  def each(&block)
    if @counter > 10
      succeed
    else
      EM.next_tick do
        yield counter
        each(&block)
      end
    end
  end
end

get '/' do
  Stream.new
end

I recently wrote a EventSource implementation that way:

require 'sinatra'

class EventStream
  include EventMachine::Deferrable
  def each
    count = 0
    timer = EventMachine::PeriodicTimer.new(1) do
      yield "data: #{count += 1}\n\n"
    end
    errback { timer.cancel }
  end
end

get '/' do
  EventMachine.next_tick do
    request.env['async.callback'].call [
      200, {'Content-Type' => 'text/event-stream'},
      EventStream.new ]
  end
  [-1, {}, []]
end

If you want to use Webrick for Streaming: here is a patch.