I wrote a piece of Rack Middleware to automatically unzip compressed request bodies. The code seems to be working just fine, but when I plug it into my rails app, I get a failure "Invalid JSON" from ActionController::ParamsParser.
As a debugging mechanism, I'm writing both the zipped content, and the unzipped content to a file (to make sure that code is working properly) and I do receive my original JSON document (before the client zips it up).
The data I'm posting is JSON data, and the unzipped content is detected as valid JSON from http://jsonlint.com.
Any ideas what I'm doing wrong?
class CompressedRequests
def initialize(app)
@app = app
end
def call(env)
input = env['rack.input'].read
#output the zipped data we received
File.open('/Users/ben/Desktop/data.gz', 'w+') do |f|
f.write input
end
if env['REQUEST_METHOD'] =~ /(POST|PUT)/
if env.keys.include? 'HTTP_CONTENT_ENCODING'
new_input = decode(input, env['HTTP_CONTENT_ENCODING'])
env['rack.input'] = StringIO.new(new_input)
#output our decoded data (for debugging)
File.open('/Users/ben/Desktop/data.txt', 'w+') do |f|
f.write env['rack.input'].read
end
env.delete('HTTP_CONTENT_ENCODING')
end
end
env['rack.input'].rewind
status, headers, response = @app.call(env)
return [status, headers, response]
end
def decode(input, content_encoding)
if content_encoding == 'gzip'
Zlib::GzipReader.new(input).read
elsif content_encoding == 'deflate'
Zlib::Inflate.new.inflate new input
else
input
end
end
end
Here is the error that I'm getting from the console:
Contents::"2010-05-17T12:46:30Z","background":false},{"longitude":-95.38620785000001,"latitude":29.62815358333334,"estimated_speed":14.04305,"timestamp":"2010-05-17T12:46:36Z","background":false},{"longitude":-95.3862767,"latitude":29.62926725,"estimated_speed":39.87791,"timestamp":"2010-05-17T12:46:42Z","background":false},{"longitude":-95.38655023333334,"latitude":29.63051011666666,"estimated_speed":46.09239,"timestamp":"2010-05-17T12:46:49Z","background":false},{"longitude":-95.38676226666666,"latitude":29.63158775,"estimated_speed":47.34936,"timestamp":"2010-05-17T12:46:55Z","background":false},{"longitude":-95.38675346666666,"latitude":29.63219841666666,"estimated_speed":22.54016,"timestamp":"2010-05-17T12:47:03Z","background":false},{"longitude":-95.38675491666666,"latitude":29.63265714999999,"estimated_speed":14.03642,"timestamp":"2010-05-17T12:47:10Z","background":false},{"longitude":-95.38677551666666,"latitude":29.63358661666667,"estimated_speed":29.29489,"timestamp":"2010-05-17T12:47:17Z","background":false},{"longitude":-95.38679026666662,"latitude":29.63466445,"estimated_speed":38.34926,"timestamp":"2010-05-17T12:47:24Z","background":false},{"longitude":-95.38681656666668,"latitude":29.63590941666666,"estimated_speed":44.82093,"timestamp":"2010-05-17T12:47:31Z","background":false},{"longitude":-95.38683366666667,"latitude":29.63679638333334,"estimated_speed":40.21729,"timestamp":"2010-05-17T12:47:37Z","background":false},{"longitude":-95.38685133333333,"latitude":29.63815714999999,"estimated_speed":44.86543,"timestamp":"2010-05-17T12:47:44Z","background":false},{"longitude":-95.3868655
/!\ FAILSAFE /!\ Mon Oct 18 18:18:43 -0500 2010
Status: 500 Internal Server Error
Invalid JSON string
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/backends/yaml.rb:14:in `decode'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/decoding.rb:11:in `__send__'
/Library/Ruby/Gems/1.8/gems/activesupport-2.3.5/lib/active_support/json/decoding.rb:11:in `decode'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/params_parser.rb:42:in `parse_formatted_parameters'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/params_parser.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/session/cookie_store.rb:93:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/failsafe.rb:26:in `call'
/Users/ben/projects/safecell/safecellweb/lib/compressed_requests.rb:36:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `synchronize'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/lock.rb:11:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/dispatcher.rb:114:in `call'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/reloader.rb:34:in `run'
/Library/Ruby/Gems/1.8/gems/actionpack-2.3.5/lib/action_controller/dispatcher.rb:108:in `call'
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/rack/static.rb:31:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:46:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `each'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/urlmap.rb:40:in `call'
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/rack/log_tailer.rb:17:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/content_length.rb:13:in `call'
/Library/Ruby/Gems/1.8/gems/rack-1.0.1/lib/rack/handler/webrick.rb:50:in `service'
One last piece of information, I'm inserting this middleware after ActionController::Failsafe.
EDIT: Looks like it's not a truncation problem
After more digging, it looks like it's not a truncation problem after all. The logs are simply clipping the output so it looks like a truncation issue.
At this point I'm not sure why the JSON is coming in as invalid. Do I need to do any manual escaping?