How to make sure web services are kept stable from one release to the next?

Question

The company where I work is a software vendor with a suite of applications. There are also a number of web services, and of course they have to be kept stable even if the applications change. We haven't always succeeded with this, and sometimes a customer finds that a service is not behaving as before after upgrading.

We now want to handle this better. In general, web services shouldn't change, and if they have to, at least we will know about it and document the change.

But how do we ensure this? One idea is to compare the WSDL files with the previous versions at every release. That will make sure the interfaces don't change, but it won't detect that the behavior changes, for example if a bug is introduced in some common library.

Another idea is to build up a suite of service tests, for example using soapUI. But then we'll never know if we have covered enough cases.

What are some best practices regarding this?

Answer 1

I think, you can definitely be confident of the stability of the services If you keep updating your service tests with the latest changes in the service and I think this is one of the best practices people use before they deploy.

Also, In general, I think what would probably matter is how well the unit testing is being done by the developers who are writing the components(libraries) used by the services. Are those unit tests being updated with the changes in the components being used by the service.

Answer 2

There as two kinds of changes for a web service, breaking change and non-breaking change. Breaking change is like changing the signature of a web method or changing a datacontract schema. Non-breaking change is like adding a new web method or adding an optional member to a datcontract. In general your client should continue to work with a non-breaking change. I don't know which technology you are using but use versioing in service namespace and datacontract namespace following W3C recommendations. You can even continue to host different versions at different endpoints. This way your clients will break if they try to use a new version of your service without re-generating the proxy from the new version of WSDL or continue to use the old version.
Some WCF specific links are http://msdn.microsoft.com/en-us/library/ms731060.aspx http://msdn.microsoft.com/en-us/library/ms733832.aspx

I wouldn't consider behaviour change as a change in SOA sense. That is more like fixing defects.

Answer 3

We monitor our services with the open source tool Membrane SOA registry. It fetches the WSDL description of registered services every 5 or 10 minutes and compares the new WSDL with the one stored in the registry. Differences are displayed in detail. Have a look at the WSDL version history of a services registered at a public accessable show case registry. The registry is a Java Web application packaged into a WAR. But it also comes with a Web Server and a start batch. Just download and start monitoring services and WSDLs.

Answer 4

IMO, aside from monitoring the WSDL for changes (which is really only necessary if you have a willy-nilly implementation ("promote-to-production") strategy), the only way to really ensure that everythign is operational and stable, is to perform continuous, automated, periodic, functional testing with a test suite that provides complete coverage of both the WSDL and the underlying application functionality, including edge cases. The test cases should be version controlled just like the app and WSDL, and should be developed in parallel to new versions of the app (not afterward, as a reaction). This can all be automated with SoapUI. Ideally, logging results somewhere that can be accumulated and reported on some dashboard, so that if somethign breaks, you know when it broke, and hopefully correlate that to an event such as an application update, or something more benign such as a service pack being pushed, electrical work being performed, etc.. However... do as I say, not as I do. I have been unsuccessful in pushing this strategy at work. Your votes will tell me whether I should push harder or do something else!