Writing test code for verifying database entries when testing an API

Question

I'm writing test code to test a client-server application. The application under test consists of

1. an application that runs on Tomcat or another Java EE application server, and
2. client jars that expose an API.

I'm basically writing test code that uses this client API to connect to the server.

In addition to extensively testing the API operations, my superiors have advised me to connect to the database on the server and verify that fields are being populated properly. I have done that for some of my test cases, but it hasn't really caught any bugs during regression.

Bugs are caught when a particular functionality fails, but that anyway gets revealed in the code that tests the API itself. It seems that DB data verification is not really useful, especially considering the extra effort required to write and maintain all that code.

My question is:

Is there any real benefit to write test code for connecting to the DB and verifying entries in this manner? Do the benefits pay off for the costs incurred in writing such code?

Answer 1

With such kind of tests you are verifying the content of the database. The other tests only verify the API operations but not the results into the DB.

Answer 2

Reading the database is unnecessary for such tests. You can achieve better results by testing that:

1. Save requests return a success status.
2. Get requests return the saved data.
3. The saved data is returned by a get request even after you reset client state ie. take client-side caching into consideration.

If you test the database contents, your test cannot be used to see if a change in the database works, because the tests expect a certain state in the database. If this is changed, your tests fail, even if the system works.

Answer 3

Looking at the database may find bugs, but it's not very likely, so I would keep the checking of the database to a minimum. As long as the API saves and restores data correctly, you don't really care how it has been stored.

You can check the database, but personally I would not do this in a systematic way.

You're asking the right questions, the purpose of testing is to find bugs. How many bugs have you found with this testing?

Automated regression testing is sometimes looked upon as a no-cost thing, but if you're continually having to update the tests then it isn't.

If you're unhappy at having to maintain these tests, I would record how much time you're spending doing it, and then you can argue that you could be doing more productive work instead, doing other forms of testing, or development.