How can I have an embedded website inside a windows app and also have it so that the windows app would startup the web server and go to the default page of the website?
Can I use the webdev server that comes with .NET framework? Or is there something better out there?
Cassini (the embedded web server) combined with the WebBrowser control is probably your best bet.
You should be able to use the standard WebBrowser control to do what you want.
You can host full ASP.NET runtime inside pretty much any .NET application. See ApplicationHost:
Enables hosting of ASP.NET pages outside the Internet Information Services (IIS) application. This class enables the host to create application domains for processing ASP.NET requests.
To display a web page, you can drop a WebBrowser control onto your form, optionally provide an ObjectForScripting and you're all set.