views:

78

answers:

1

Say I have a Sinatra route ala:

put '/data' do
  request.body.read
  # ...
end

It appears that the entire request.body is read into memory. Is there a way to consume the body as it comes into the system, rather than having it all buffered in Rack/Sinatra beforehand?

I see I can do this to read the body in parts, but the entire body still seems to be read into memory beforehand.

put '/data' do
  while request.body.read(1024) != nil 
    # ...
  end
  # ...
end
+1  A: 

You cannot avoid this in general without patching Sinatra and/or Rack. It is done by Rack::Request when request.POST is called by Sinatra to create params.

But you could place a middleware in front of Sinatra to remove the body:

require 'sinatra'
require 'stringio'

use Rack::Config do |env|
  if env['PATH_INFO'] == '/data' and env['REQUEST_METHOD'] == 'PUT'
    env['rack.input'], env['data.input'] = StringIO.new, env['rack.input']
  end
end

put '/data' do
  while request.env['data.input'].body.read(1024) != nil 
    # ...
  end
  # ...
end
Konstantin Haase
Konstantin - Thanks for the detailed response... Some questions for you an an edit or two and I'll accept your answer.1. (the edit) In your code fragment, can you please fix `StrinIO.new` to be `StringIO.new`?2. If you used Rack to remove the body, won't the entire body still be read by Rack from the network into memory? If that's the case, maybe a small edit to your answer to reflect it?3. I haven't seen Ruby like `env['rack.input'], env['data.input'] = StrinIO.new, env['rack.input']` before. Can you please explain in a comment what this code fragment is doing?Thanks again!
Chris Markle
`a,b = 1,2` assigns 1 to a and 2 to b, so `env['rack.input'], env['data.input'] = StringIO.new, env['rack.input']` sets `env['rack.input']` to an empty `StringIO` and puts its old value(probably a temp file handle) in `env['data.input']` so you can reference it later.
BaroqueBobcat
BaroqueBobca is right. This is pattern is especially common if you want to exchange the values of two variables. Imagine you have `a` and `b` and want to set both `a = b` and `b = a`. To not lose the old value of `a` you would have to create a temporary variable, like `old = a; a = b; b = c`. Ruby supports another syntax (and so does Python, afaik): `a, b = b, a`, which avoids the temporary variable, as it first evaluates the right hand (`b, a`) completely and then does the assignment.
Konstantin Haase