Most Efficient Way of calling an external webservice in Java?

Question

In one of our applications we need to call the Yahoo Soap Webservice to Get Weather and other related info.

I used the wsdl2java tool from axis1.4 and generated th required stubs and wrote a client. I use jsp's use bean to include the client bean and call methods defined in the client which call the yahoo webservice inturn.

Now the problem: When users make calls to the jsp the response time of the webservice differs greatly, like for one user it took less then 10 seconds and the other in the same network took more than a minute.

I was just wondering if Axis1.4 queues the requests even though the jsps are multithreaded.

And finally is there an efficient way of calling the webservice(Yahoo weather). Typically i get around 200 simultaneous requests from my users.

Answer 1

I'd use Google tools to monitor how long the call to the web service is taking.

There are several things going on here:

1. Map Java beans to XML request.
2. Send XML request to web service.
3. Unmarshall XML request on web service side.
4. Web service processes request
5. Web service marshalles XML response
6. Web service sends XML response to Java client
7. Unmarshall XML response and display on client.

You can't see inside the Yahoo web service, but do break out what you can see on the client side to see where the time is spent.

Check memory as well. If Axis is generating .class files, maybe your perm space is being consumed. Visual VM is available to you with the JDK. Attach it to the PID on your client to see what's going on in memory on your app server.

Maybe this would be a good place for an AJAX call. This will be a good solution if you can get the weather in the background while users are doing other things.

Answer 2

Why don't you schedule one thread to get the weather every minute or so, and expose that to the JSP, in stead of letting each JSP get its own weather report?

That's a lot more efficient for both you and Yahoo, and JSP's only need to lookup a local object (almost instantaneous) in stead of connecting to a web service.

EDIT

Some new requirements in the comments of this answer suggest a different way of choosing solutions.

It seems that not only weather, which not only doesn't change that often but is also the same for every user, is requested by web service but also other data like flight data.

The requirements for flight data retrieval are very much different than for weather data. So I think you should define a few types of (remote) data and choose a different solution for each category.

As basis for the requirements I'd use something simple:

• Users like their information promptly, they do not like waiting
• The amount of data stored on the web server is finite
• Remote web services have an EULA of sorts and are probably not happy with 200 concurrent requests of the same data by the same source (you)

Fast data access to users is best achieved by having the data locally, be it transient (kept in a bean) or persistent (a local database). That can be done by periodically requesting data from the remote source, and using the cached data in the JSP. That would also keep you in the clear with the third point.

A finite amount of data stored on the web service means that not everything can be cached. Data which differs per user, or large data sets which can vary over small periods of time, cannot readily be cached. It's not really a good idea to load data on all flights of all airports in the US every minute or so. That kind of requests would be better served by running a specific web service query when necessary.

The trick is now to identify when caching data is feasible. If it is feasible, do that, otherwise run the web service query in the background. That can be done by presenting the JSP now and starting the web service query in the background. The JSP can have an AJAX script which queries your web server whether the data is ready, and insert that data in the page when ready.

Answer 3

I would recommend local caching and data pooling. Instead of sending out 200 separate requests for similar/same locations run a background thread which pulls the weather for only the locations your users are interested in and caches them locally, this cache updates every minute or so. When users request their personal preferences, the requests hit the cache and refetch if the location is new or the data in the cache is stale. This way the user will have a more seamless experience and you will not hit Yahoo throttles and get denied service.