What's the state of TDD and/or BDD in PHP?

Question

How widespread, supported, developed is testing in the PHP world? On par with Java? Up there with Ruby/Rails? I Googled and found that testing frameworks exist but I'm wondering if they're widely used.

Do the major PHP IDE's have built-in test runners the way Eclipse's Java tools do or NetBeans's Ruby/Rails tools do? Is testing built into PHP's MVC frameworks as with Rails?

I ask because a group where I work wants to hire someone to develop a PHP app for them. I'm concerned about quality and maintenance as I might be called on to support this thing.

Answer 1

There are at least two mature, stand-alone, JUnit style test suites available, named PHPUnit and SimpleTest, respectively.

As far the MVC Frameworks go, Symfony has its own testing framework named lime, Code Igniter has a unit_test library and CakePHP relies on the aforementioned SimpleTest.

I know that Zend Studio has built in support for PHPUnit tests, and both PHPUnit and SimpleTest have command-line runners so integration into any workflow is possible.

The tools are there in the PHP world if a developer wants to take advantage of them, and smart shops do take advantage of them.

The caveats are your par for the course PHP complaints. There are two PHP communities; PHP as a platform for building software, and PHP as a way to interact with a web server, web browser, and database to produce application-like things on the web. It's less a black and white thing and more a continuum; Among those more on the software developer side unit testing and TDD is supported and used as much as it is on any other platform. Among the "cobble together a bunch of stuff I don't understand but still get results people", it's unheard of.

There's a lot of non-framework/custom-framework legacy PHP code around that's difficult to get a useful test harness around. PHP also lends itself easily to patterns that rely on the existence of a browser environment to run. I don't have any evidence to back this up other than my own observations, but a lot of PHP shops that care about testing end up relying on acceptance testing (i.e. Selenium) as a substitute for actual Unit Testing, test-first, etc. development.

In your specific situation, interview the hell out of the developer your group is going to hire.

1. Ask them what unit testing framework they use

2. Ask them to describe, in general terms, a real world example of a time they developed a new feature and its supporting tests

3. Ask them to describe, in general terms, a real world example of a time their tests failed and what they did to resolve the situation

You're less interested in the specific situation they're going to describe and more interested in how comfortable they are discussing their knowledge of code testing in general.

Answer 2

In addition to the libraries/frameworks that Alan has already mentioned, you can make use of mod_perl's Apache::Test, which I what I use as a harness. It allows me to very simply integrate tests into my release process. The harness uses TAP output (Test Anything Protocol) to determine whether or not the tests pass or fail, using libraries like Test::Simple or Test::More (Perl and PHP).

Out of the box Apache::Test supports writing tests in both Perl and PHP. In my own projects, it took a little bit of trickery and a lot of reading to really get it working, but an implementation of Test::More in PHP is built-in to the harness. Running all tests written in both PHP and Perl is done through a single command and any failure along the way is captured Apache::Test, noting as best it can what went wrong.

The awesome part about all this is that you can even utilize PHPUnit, or Simple-Test alongside the previous two testing frameworks. By running tests in each respective library, you can use the PHP implementation of Test::More (or even Perl by testing stdout) and spit back out TAP for your harness to interpret.

Be sure to read the Apache::Test documentation and the mod_perl guide to running Apache::Test. Additionally, I found the article here a great help.

As a quick example, you could setup a test in Perl in very few lines of code that will run through all the pages on your site (that have links) and verify all result in '200 OK' responses and don't have any parsing errors:

#!perl

use strict;
use warnings;

use Apache::Test qw(:withtestmore);
use Apache::TestRequest;
use Test::More;
use Test::WWW::Mechanize;
use WWW::CheckSite::Validator;
use WWW::CheckSite::Spider;

plan 'no_plan';

my $config = Apache::Test::config();
my $host = "http://". Apache::TestRequest::hostport($config) || '';

my $s = WWW::CheckSite::Spider->new(
    uri => $host,
    ua_class => 'Test::WWW::Mechanize',
);
my $m = $s->current_agent;

while (my $page = $s->get_page) {
    is($m->status(), "200", $m->uri() ." retrieved successfully.");
    $m->content_lacks("Parse Error", $m->uri() ." does not contain syntax errors.");
}

Answer 3

On a past project, I've used the PHPUnit, and it has left me wanting. PHPUnit + Command line running of the tests, made it so that too much time was spent coding the tests, wasn't fast enough, and really seemed to constrain the style of the code in a way that I didn't like (Objects were easier to test, so it seemed to kinda favor objects).

Selenium was a solution that we talked about but never got around to getting into play, and I think we would really have benefited from that kind of output-level testing.

On this latest project, the lead programmer has taken a more functional programming approach as we've been revising software. When I mentioned that I'd like to code via TDD, he whipped up a custom solution in a day or less that I consider to have been as effective for me to use as PHPUnit. In addition, he really opened my eyes about the question of Object Oriented vs. Functional Programming.

First project, started from scratch, in at the ground floor, Object Oriented coding, large Unit Testing Framework, it became monolithic and bogged down quickly. Second project, well established CMS software with a 5 year history and old code, yet a functional programming paradigm and a simple testing framework (we actually often made use of php assert) made it get simpler instead of grow in complexity.

The second project, too, never got to the point of implementing Selenium (and I still think it would be beneficial), but the functional programming approach made it easier to deal with in-code testing.

Answer 4

Michael Booth's comparison of BDD testing features in both languages:

http://mechanicalrobotfish.com/posts/117-ruby-vs-php-bdd-beauty-contest-no-contest

concludes that PHP BDD tools and culture is underdeveloped at this point.

Certainly there is nothing comparable with what's available to a Ruby programmer, either in terms of knowledge (books, videos, articles, blog posts) or tools (Rspec, Shoulda, Factory Girl, Mocha, Cucumber).