There's no reason to avoid open source solutions if cost is an issue; these days, they're every bit as good as the best commercial products for any small installation (i.e., anything you'd spend less than hundreds of thousands of dollars on).
Myself, I'd go with FreeBSD, Apache, one of the good scripting languages (Python, Ruby, or Perl, in order of my own personal preference), and PostgreSQL.
On servers, FreeBSD seems a bit more stable than Linux, at a cost of having a slightly different feature set and being a tiny bit less pleasant to work with. Either one is fine though; Linux makes a better workstation, so there's choices all around.
Apache is a good server choice because there's nothing it can't do. You will never need another server if you go with Apache; even if it isn't the perfect fit (some other product might be a little faster or more natural for your current work), you know Apache will handle it and handle it well.
Any of the scripting languages are a good choice; PHP is okay, but Python/Ruby/Perl are such good alternatives these days that it's hard to see why anyone would choose PHP for a completely new project.
PostgreSQL is superior to MySQL in just about every respect (far better feature set, more stable, at least as fast). MySQL does, however, have the edge on replication. PostgreSQL can do it, but it won't be natively supported in the core until the 8.5 release at the earliest (expected in 2010). As well, if you expect to have complex queries, PostgreSQL is derived from Ingres, which was famous back in the day for its extremely powerful query parser, making it a natural fit for messy work.