I'm working on the upgrade of my c# visual express 2008 project to a newer version. How do I manage this without having to copy the whole project directory and work into the copied directory project?
Create a new branch in the source control system that you are using. You can read more about it at Branching and Merging Primer.
If you are not using a source control I suggest you start with subversion. VisualSVN SERVER is an easy way to get started. You can use TortoiseSVN to work with it or visual studio plugins such as ankhsvn or VisualSVN.
You do not really have to install svn server. You can create repository with Tortoise Svn too and use file protocol to connect to it. This means that you can put the repository on your flash drive and query it with you so that you can work from any computer.
Subversion is good, and easy to pick up, but you might also want to consider Mercurial or Git. These are almost as easy to pick up and give you some flexibility in that they are distributed - which I have found makes much more sense for an individual user. These also tend to take up less space in the long run as well.
In addition, for your needs, you will not necessarily need to branch. Get your chosen Source Control installed, and create your repository with your stable version in it. Once you have that, as long as you have a backup in place, you will always be able to get back to your stable version. If you ever have to do a bug fix on your stable version, that would be the time to branch from that version. For a small individual repository, it'll probably be less confusing to keep your main work in the trunk -
Lastly, since you are new to source control, get in the habit of doing one thing at a time, then checking that change in. Your source control tool (or commandline) should be open every time your IDE is, and you should use it often. Even small changes are important to track. Establishing this habit will take work, but will pay off immensely.
Good luck!