If a phpinfo() dump is shown to an end user, what is the worst that a malicious user could do with that information? What fields are most unsecure? That is, if your phpinfo() was publicly displayed, after taking it down, where should you watch/focus for malicious exploits?
I'm wiki'ing this up, so that we can compile answers.
Hackers can use this information to find vulnerabilities and hack your site.
Besides the obvious like being able to see if register_globals is On, and where files might be located in your include_path, there's all the $_SERVER ($_SERVER["DOCUMENT_ROOT"] can give clues to define a relative pathname to /etc/passwd) and $_ENV information (it's amazing what people store in $_ENV, such as encryption keys)
Honestly, not much. Personally, I frequently leave phpinfo() pages up.
If you have some serious misconfigurations (e.g. PHP is running as root), or you're using old and vulnerable versions of some extensions or PHP itself, this information will be more exposed. On the other hand, you also wouldn't be protected by not exposing phpinfo(); you should have instead take care of having your server up-to-date and correctly configured.
Knowing the structure of your filesystem might allow hackers to execute directory traversal attacks if your site is vulnerable to them.
I think exposing phpinfo() on its own isn't necessarily a risk, but in combination with another vulnerability could lead to your site becoming compromised.
Obviously, the less specific info hackers have about your system, the better. Disabling phpinfo() won't make your site secure, but will make it slightly more difficult for them.
A well-configured, up-to-date system can afford to expose phpinfo() without risk.
Still, it is possible to get hold of so much detailed information - especially module versions, which could make a cracker's life easier when newly-discovered exploits come up - that I think it's good practice not to leave them up. Especially on shared hosting, where you have no influence on everyday server administration.
The biggest problem is that many versions make XSS attacks simple by printing the contents of the URL and other data used to access it.