is using threads and ruby mechanize safe ?

Question

Does anyone ever see alot of errors like this: Exception `Net::HTTPBadResponse' at /usr/lib/ruby/1.8/net/http.rb:2022 - wrong status line: SOME HTML CODE HERE

When using threads and mechanize? I'm relatively certain that this is some bad behavior between threads and the net/http library, but does anyone have any advice as far as the upper limit of threads you want to run at once when using mechanize/nethttp? and how can i capture this kind of exception vecause (rescue Net::HTTPBadResponse) doesnt work?

Answer 1

This could be something non-thread-safe in Mechanize, but I can think of other bugs that might cause the same problem. I'd start by disabling persistent connections, if you're using them. The next thing to do is to look at your code, and make sure that you're being careful with the objects you handle. If your application has multiple threads mucking about with common objects, that can break a library that would be otherwise thread-safe.

If there is a threading problem somewhere, the upper limit of threads you can use safely is 1. Any more, and you're just making a trade-off about how often you want the problem to occur, rather than whether it occurs or not.