You should look in to one of these several very good tutorials and articles about deploying ASP.NET applications.
Now to your thoughts on what should be uploaded and what not. If you understand the asp.net cycle and how the files are processed by the webserver, it would be easy to know that all files need to be uploaded. But since this is your first application you probably don't have that knowledge.
So, to save you some reading time, Upload everything. In older versions such as 1.1 you had to compile all your source-code into binaries, which you don't have to anymore, but you can, it's up to you.
I However prefere to upload all the files without compiling it to binaries, makes it easier to manage once they are on the server.
The webserver will upon request ( first request ) compile these and then use the compiled files on other requests, this is the short answer anyway, this is why the load-time is longer the first time and the other times it's ( suppose to ) go faster.
Also, worht to know is that if you change the web.config, the application will re-compile.
So just drag n' drop em' to your webb-location and start playing!