Making GWT RPC calls to an external service for which you don't have code

views:

223

answers:

+2 Q:

Making GWT RPC calls to an external service for which you don't have code

I would like to make a small program that scrapes information from a 3rd-party GWT-enabled website. Is it possible to somehow call this RPC directly? What would I need to reverse-engineer to do this (i.e. hopefully not the entire low-level protocol).

I am hoping I could somehow just call this from within my own server-side GWT or Servlet app.

I'm pretty sure GWT makes it difficult to do cross-site requests, for the sake of security.

Any solution you come up with will probably be hacky (and not flexible to future changes), and since you're presumably doing it without the consent of the site in question, probably a bad idea to begin with.

Is there some reason you can't ask the site to publish their data using a REST API?

Jason Hall 2010-04-19 15:32:05

This is for the Developer Dashboard on the Google Market. They don't mind me scraping it, but they don't have time to make a REST API. There is no robots.txt for this site, so scraping it should be absolutely okay (otherwise the search engines would be in trouble).

Artem 2010-04-19 16:39:38

There's a big difference between scraping and trying to make GWT-RPC requests to their services. GWT makes it hard to make a GWT-RPC service public, for security reasons. And when (not if) they change the API of these calls, your app will break. It *may* be possible, but it won't be a good idea.

Jason Hall 2010-04-19 17:12:27

I am sorry, I disagree there is a big difference. Google spiders routinely submit automated queries to all sorts of CGI backends and that's how they crawl the deep web. http://glinden.blogspot.com/2007/03/google-and-deep-web.html Please point me to some specific sources that say GWT-RPC is not to be contacted. That, and scraping is always possible through GreaseMonkey on Firefox. It's just a matter of doing it an easier way.

Artem 2010-04-21 04:44:32

It is possible.. but you would have to go through their code a bit to understand how the serialization/deserialization works.

Classes of interest are

RPC.java
ClientSerializationStreamWriter -> ServerSerializationStreamReader are the classes involved in making a GWT request.
ServerSerializationStreamWriter -> ClientSerializationStreamReader are the classes involved in creating and interpreting a responose.

I am trying out a similar thing as an academic project, will add more information if I am able to decipher these classes

sri 2010-04-19 20:43:08

But he doesn't have access to their code. He's just trying to make GWT-RPC requests against a service running on their server.

Jason Hall 2010-04-19 20:56:49

Jason, that would be general GWT code. Sri, that sounds great, please keep up the research.

Artem 2010-04-20 05:22:05

ansaurus

tags:

views:

answers:

Making GWT RPC calls to an external service for which you don't have code

related questions