Ideas on Protecting Web App data sources

Question

I'm working on a new web app where a large amount of content (text, images, meta-data) is requested via an Ajax request.

No auth or login required for a user to access this.

My concern is that you could easily lookup the data source URL and hit it directly outside the app to get large data. In some ways, if you can do this you could probably scrape the static HTML pages elsewhere that also have this content.

Are there any suggestions on methods to obfuscate, hide, or otherwise make it very difficult to access the data directly?

Example: web app HTML page contains a key that is republished every 30 min. On the server side the data is obfuscated based on this key. In order to get the data outside the app you'd need to figure out the data source but also the extra step of scraping the page for a key every 30 min.

I realize there is no 100% way to stop someone, but I'm talking more about deterrence.

Answer 1

"Are there any suggestions on methods to obfuscate, hide, or otherwise make it very difficult to access the data directly?"

Answers your own question because if the data is worth getting it will be obtained because you are obfuscating is merely making it harder to find.

You could in the server side script processing the ajax and returning the data check where the request came from.

Answer 2

Use sessions in your webapp. Make a note (e.g. database entry or some other mechanism which your server-side code can access) when a valid request for the first page is received and include code in the second page to exclude the data when processing a request without a corresponding session entry.

Obviously the specifics on how to do this will vary between languages, but most robust web platforms will support sessions, largely for this type of reason.

Answer 3

If you are wanting to display real-time data and are concerned about scrapers...if this is a big enough concern, then I suggest doing it with flash instead of JS (AJAX). Have the data display withing a flash object. Flash can make real-time send/receive requests to the server just like AJAX. But the benefit of Flash is that the whole stage, data, code, etc.. are within a flash object, which cannot be scraped. Flash object makes the request, you output the stuff as a crypted string of code. Decrypt it within flash and display from there.