What is the maximum size of document to return from a REST API?

Question

I am considering creating a RESTful web service which will return a lot of data. What do people think is the maximum size of xml document that should be returned in one get from a web services API? I would estimate the size of the result set to be 100Mb and time to produce this would be about 2h.

How much time is a reasonable maximum for the server to be calculating a result set before returning the document?

The web server is Apache using django on mod_python.

The web service would only be called once or twice a day.

Are there any special configuration options I should enable to make this work on the Apache/mod_python/django stack.

Answer 1

According to this: http://lithosphere.lithium.com/t5/Growing-Successful-Communities/REST-API-search-results-older-than-lithium-search-results/m-p/3882%3Bjsessionid=26786A8E8B534275FE29DDC1B2B9F4D4

and this: http://confluence.atlassian.com/display/DOC/Prototype+REST+API

The maximum size is 50 if you don't set a default. If you do, it appears to be a limit of 1000 that you can set.

This seems to be consistent among multiple uses, so you could take that as a convention.

I would say do some testing, but if it's only twice per day, then go for the maximum and work your way down if you have/want to.

Answer 2

This is really a constraint of HTTP and nothing to do with REST. Personally, I regularly download content that is hundreds of megabytes over HTTP and don't have any problems.

Answer 3

If your web service is designed to server very large documents, then don't worry about a maximum. However, if you serve a mix of small and large documents, you might want to consider serving some form of summary, along with a URL to the full document. Your main concern should be to correctly configure the web server to handle this since serving a long document ties up a connection for a longer time period. The actual service process is not very CPU intensive though.

The REST model does not impose size limitations, and some people are using it to serve up .ISO images of CDs and DVDs. The DVD images are 2-6 gigabytes.