Looking for testing/QA idea for Python Web Application Project

Question

Hi, all,

I have the 'luck' of develop and enhance a legacy python web application for almost 2 years. The major contribution I consider I made is the introduction of the use of unit test, nosestest, pychecker and CI server. Yes, that's right, there are still project out there that has no single unit test (To be fair, it has a few doctest, but are broken).

Nonetheless, progress is slow, because literally the coverage is limited by how many unit tests you can afford to write.

From time to time embarrassing mistakes still occur, and it does not look good on management reports. (e.g. even pychecker cannot catch certain "missing attribute" situation, and the program just blows up in run time)

I just want to know if anyone has any suggestion about what additional thing I can do to improve the QA. The application uses WebWare 0.8.1, but I have expermentially ported it to cherrypy, so I can potentially take advantage of WSGI to conduct integration tests.

Mixed language development and/or hiring an additional tester are also options I am thinking.

Nothing is too wild, as long as it works.

Answer 1

Feather's great book is the first resource I always recommend to anybody in your situation (wish I had it in hand before I faced it my first four times or so!-) -- not Python specific but a lot of VERY useful general-purpose pieces of advice.

Another technique I've been happy with is fuzz testing -- low-effort, great returns in terms of catching sundry bugs and vulnerabilitues; check it out!

Last but not least, if you do have the headcount & budget to hire one more engineer, please do, but make sure he or she is a "software engineer in testing", NOT a warm body banging at the keyboard or mouse for manual "testing" -- somebody who's rarin' to write and integrate all sorts of automated testing approaches as opposed to spending their days endlessly repeating (if they're lucky) the same manual testing sequences!!!

I'm not sure what you think mixed language dev't will buy you in terms of QA. WSGI OTOH will give you nice bottlenecks/hooks to exploit in your forthcoming integration-test infrastructure -- it's good for that (AND for sundry other things too;-).

Answer 2

Automated testing seems to be as a very interesting approach. If you are developping a web app, you may be interested in WebDriver http://code.google.com/p/webdriver/

Answer 3

Since it is a web app, I'm wondering whether browser-based testing would make sense for you. If so, check out Selenium, an open-source suite of test tools. Here are some items that might be interesting to you:

• automatically starts and stops browser instances on major platforms (linux, win32, macos)
• tests by emulating user actions on web pages (clicking, typing), Javascript based
• uses assertions for behavioral results (new web page loaded, containing text, ...)
• can record interactive tests in firefox
• can be driven by Python test scripts, using a simple communication API and running against a coordination server (Selenium RC).
• can run multiple browsers on the same machine or multiple machines

It has a learning curve, but particularly the Selenium RC server architecture is very helpful in conducting automated browser tests.

Answer 4

Have a look at Twill, it's a headless web browser written in Python, specifically for automated testing. It can record and replay actions, and it can also hook directly into a WSGI stack.

Answer 5

Few things help as much as testing.

These two quotes are really important.

• "how many unit tests you can afford to write."

• "From time to time embarrassing mistakes still occur,"

If mistakes occur, you haven't written enough tests. If you're still having mistakes, then you can afford to write more unit tests. It's that simple.

Each embarrassing mistake is a direct result of not writing enough unit tests.

Each management report that describes an embarrassing mistake should also describe what testing is required to prevent that mistake from ever happening again.

A unit test is a permanent prevention of further problems.