Give the new developer instructions on how to set up his environment. It lets him start developing a good understanding of how the system works. If he's never exposed to the different third party components, it's much harder for him to get a good understanding of what's not working properly whenever things go bad.
Additionally, by maintaining the instructions the organization has a much better understanding of all the dependencies in both a product and a development environment.
Sometimes these dependencies include installing license keys, preparing connection strings, or other minutia that can easily get lost in just shipping a stock environment (whether a Ghost image or a VM). By explicitly setting these things up whenever there's a new hire (or new machine rollout) everyone gains.
Edit: Let me clarify (based on some other answers below) that the machine can come with the OS, Visual Studio, version control tools, database tools, etc. already installed. It's the 3rd party controls/add-ons/etc. (think COM+, GAC, etc.) that they should be installing/configuring.