End to End testing of Webservices

Question

Hi all, first time poster and TDD adopter. :-) I'll be a bit verbose so please bear with me.

I've recently started developing SOAP based web services using the Apache CXF framework, Spring and Commons Chain for implementing business flow. The problem I'm facing here is with testing the web services -- testing as in Unit testing and functional testing.

My first attempt at Unit testing was a complete failure. To keep the unit tests flexible, I used a Spring XML file to keep my test data in. Also, instead of creating instances of "components" to be tested, I retrieved them from my Spring Application context. The XML files which harbored data quickly got out of hand; creating object graphs in XML turned out to be a nightmare. Since the "components" to be tested were picked from the Spring Application Context, each test run loaded all the components involved in my application, the DAO objects used etc. Also, as opposed to the concept of unit test cases being centralized or concentrated on testing only the component, my unit tests started hitting databases, communicating with mail servers etc. Bad, really bad.

I knew what I had done wrong and started to think of ways to rectify it. Following an advice from one of the posts on this board, I looked up Mockito, the Java mocking framework so that I could do away with using real DAO classes and mail servers and just mock the functionality.

With unit tests a bit under control, this brings me to my second problem; the dependence on data. The web services which I have been developing have very little logic but heavy reliance on data. As an example, consider one of my components:

public class PaymentScheduleRetrievalComponent implements Command {
  public boolean execute(Context ctx) {
    Policy policy = (Policy)ctx.get("POLICY");
    List<PaymentSchedule> list = billingDAO.getPaymentStatementForPolicy(policy);
    ctx.put("PAYMENT_SCHEDULE_LIST", list);
    return false;
  }
}

A majority of my components follow the same route -- pick a domain object from the context, hit the DAO [we are using iBatis as the SQL mapper here] and retrieve the result.

So, now the questions:
- How are DAO classes tested esp when a single insertion or updation might leave the database in a "unstable" state [in cases where let's say 3 insertions into different tables actually form a single transaction]?
- What is the de-facto standard for functional testing web services which move around a lot of data i.e. mindless insertions/retrievals from the data store?

Your personal experiences/comments would be greatly appreciated. Please let me know in case I've missed out some details on my part in explaining the problem at hand.

-sasuke

Answer 1

I must say I don't really understand your exact problem. Is the problem that your database is left in an altered state after you've run the test?

Answer 2

First of all: Is there a reason you have to retrieve the subject under test (SUT) from the Spring Application context? For efficient unit testing you should be able to create the SUT without the context. It sounds like you have some hidden dependencies somewhere. That might be the root of some of your headache.

How are DAO classes tested esp when a single insertion or updation might leave the database in a "unstable" state [in cases where let's say 3 insertions into different tables actually form a single transaction]?

It seems you are worried about the database's constistency after you have running the tests. If possible use a own database for testing, where you don't need care about it. If you have such a sandbox database you can delete data as you wish. In this case I would do the following:

1. Flag all your fake data with some common identifier, like putting a special prefix to a field.
2. Before running the test drop a delete statement, which deletes the flagged data. If there is none, then nothing bad happens.
3. Run your single DAO test. After that repeat step 2. for the next test.

What is the de-facto standard for functional testing web services which move around a lot of data i.e. mindless insertions/retrievals from the data store?

I am not aware of any. From the question your are asking I can infer that you have on one side the web service and on the other side the database. Split up the responsibilities. Have separate test suites for each side. One side just testing database access (as described above). On the other side just testing web service requests and responses. In this case it pays of the stub/fake/mock the layer talking to the network. Or consider https://wsunit.dev.java.net/.

If the program is only shoving data in and out I think that there is not much behavior. If this is the case, then the hardest work is to unit test the database side and the web service side. The point is you can do unit testing without the need for "realistic" data. For functional testing you will need handrolled data, which is close to reality. This might be cumbersome, but if you already unit tested the database and web service parts intensively, this should reduce the need for "realistic" test cases considerably.

Answer 3

I must say I don't really understand your exact problem. Is the problem that your database is left in an altered state after you've run the test?

Yes, there are actually two issues here. First one being the problem with the database left in an inconsistent state after running the test cases. The second one being that I'm looking for an elegant solution in terms of end-to-end testing of web services.

For efficient unit testing you should be able to create the SUT without the context. It sounds like you have some hidden dependencies somewhere. That might be the root of some of your headache.

That indeed was the root cause of my headaches which I am now about to do away with with the help of a mocking framework.

It seems you are worried about the database's constistency after you have running the tests. If possible use a own database for testing, where you don't need care about it. If you have such a sandbox database you can delete data as you wish.

This is indeed one of the solutions to the problem I mentioned in my previous post but this might not work in all the cases esp when integrating with a legacy system in which the database/data isn't in your control and in cases when some DAO methods require a certain data to be already present in a given set of tables. Should I look into database unit testing frameworks like DBUnit?

In this case it pays of the stub/fake/mock the layer talking to the network. Or consider https://wsunit.dev.java.net/.

Ah, looks interesting. I've also heard of tools like SOAPUI and the likes which can be used for functional testing. Has anyone here had any success with such tools?

Thanks for all the answers and apologies for the ambiguous explanation; English isn't my first language.

-sasuke

Answer 4

I would stay well away from the "Context as global hashmap" 'pattern' if I were you.

Looks like you are testing your persistence mapping...

You might want to take a look at: testing persistent objects without spring

Answer 5

I would recommend an in-memory database for running your unit tests against, such as HSQL. You can use this to create your schema on the fly (for example if you are using Hibernate, you can use your XML mappings files), then insert/update/delete as required before destroying the database at the end of your unit test. At no time will your test interfere with your actual database.

For you second problem (end-to-end testing of web services), I have successfully unit tested CXF-based services in the past. The trick is to publish your web service using a light-weight web server at the beginning of your test (Jetty is ideal), then use CXF to point a client to your web service endpoint, run your calls, then finally shut down the Jetty instance hosting your web service once your unit test has completed.

To achive this, you can use the JaxWsServerFactoryBean (server-side) and JaxWsProxyFactoryBean (client-side) classes provided with CXF, see this page for sample code:

http://cwiki.apache.org/CXF20DOC/a-simple-jax-ws-service.html#AsimpleJAX-WSservice-Publishingyourservice

I would also give a big thumbs up to SOAP UI for doing functional testing of your web service. JMeter is also extremely useful for stress testing web services, which is particularity important for those services doing database lookups.