The PHP mail() function only returns true or false, signifying whether the e-mail was successfully handed off to the mailer or not. It isn't possible to get any extra information, because PHP doesn't even handle the mailing. It hands it off to sendmail or whatever is configured in PHP.ini.
To determine the reason for failure, you must use another method. I'd like to second shamittomar's suggestion of Pear's Mail package. It is excellent.
If you use Mail::send() in Pear, it will either return true
or a PEAR_Error() object which you can call the getCode() method on and figure out if you have an SMTP authentication error.
See this for more details: http://pear.php.net/manual/en/package.mail.mail.send.php