What is CMS testing anyway?

Question

My company recently did a POC and have decided to use an commercial CMS. It is being implemented and we have been asked to Test it. What is there to be tested in a 3rd party CMS that has already been tested and being sold in the market?. Any direction would be great!

Answer 1

I would recommend adjusting your mind-set. What do you know about the test regime of this product? To start from the point of view that it's a commercially shipped product, it must have been tested, so I don't need to test it, is deeply flawed thinking.

First, all software has bugs.

Second, in testing the product you could reasonably focus on your proposed usage scenarios. You may choose patterns of use that were not anticipated by the development and test teams. At the very least you gain experience of the product's capabilities and limitations.

Third, installation into your environment may impact the system in unexpected ways. So at the very least the product must be exercised in your environment before you start to trust it. You need to explore the operational aspects, backups and restores for example. Now, before the system is live, is the time to find out how to recover from a disk crash.

I would ask the vendor is they have a regression suite you can run in your environment. if not I would devise a quick check-list of my own, trying to think about corner cases. Then also start to explore how your teams will use the product. Presumably there will be a "Build Master" role? Work with the people in that role and walk through some common scenarios. The likelyhood is that you will uncover some ways of working that are better than others.

Summary: testing isn't just about finding bugs (though you may well find some) but it's also about understanding the product better and learning how best to use it.