ASP.net web sites do not use the configuration manager to determine if debug info is included in the compile. You must set it in the web.config file. Visual Studio will never change debug to "false" for you automactially, as far as I know.
Find this section in your web.config file and change it to "false":
<!--
Set compilation debug="true" to insert debugging
symbols into the compiled page. Because this
affects performance, set this value to true only
during development.
-->
<compilation debug="true">
Visual Studio will ask you if you want it changed from false to true if you are running your web site in the IDE, but unfortunately it does not do the reverse for publishing (which seems more important to me).
If you have multiple projects in your solution, and at least one of them supports a Release configuration (such as a DLL) - it will appear in the configuration drop-down list. Building with Release selected still does not affect the website, however.