Ruby Web API scraping / error handling with Hpricot

Question

I have written a simple ruby gem to scrape a set of websites, providing a simple API, inside the gem itself I have included a retry method... to attempt to use Hpricot 3 or more times on failure mostly due to timeouts.

def retryable(options = {}, &block)
  opts = { :tries => 1, :on => Exception }.merge(options)

  retry_exception, retries = opts[:on], opts[:tries]

  begin
    return yield
  rescue retry_exception
    retry if (retries -= 1) > 0
  end

  yield
end

So now, in my Rails app which uses this gem i have created I'm wondering how I should handle errors should the Gem itself fail to produce a result, for whatever reason...

models/Available.rb

data = Whatever.find_item_by_id options
unless hwdata
  raise "Web error "
end

I'm not quite sure how to handle this... at this point I don't really care about retrying, I only want a return a result, either a hash which the gem returns or returns false with some error?

Answer 1

For normal erros, Like 404, 500 etc, whatever mechanism you are using to fetch the website content, will throw the errors, for other reasons, the Gem should raise the other errors, and informs your rails app, where they can be handled, because its specific to that app.

The Gem should be as generic as possible, for reuse etc.