From my OpenGL experience, it seems that "targeting" a given version is just accessing the various extensions that have been added for that version. So the only reason you would "target" OpenGL version 3 is if you want to use some of the extensions that are new to version 3. If you don't really use version 3 extensions (if you're just doing basic OpenGL stuff), then you naturally aren't "targeting" version 3.
In Visual Studio, you will always link your application with opengl32.lib, and opengl32.lib doesn't change across different OpenGL versions. OpenGL instead uses wglGetProcAddress() to dynamically access OpenGL extensions/versions at run time instead of at compile time. Namely, if a given driver doesn't support an extension, then wglGetProcAddress() will return NULL at run-time when that extension's procedure is requested. So in your code you will need to implement logic that handles the NULL return case. In the simplest scenario, you could just print an error and say "this feature isn't available, so this program will behave ...". Or you can find other alternative methods for doing the same thing that doesn't use the extension. For the most part, you'll only get NULL returns from wglGetProcAddress if you application is running on old hardware/drivers that don't support the OpenGL version that added the extension you're looking for. However, in future years you'll want to keep abreast of those things that newer OpenGL versions decide to deprecate. I haven't read too much into the 3.1 spec, but apparently they're introducing a deprecation model where older technology/extensions may be deprecated, which will open the door for newer hardware/drivers to no longer support deprecated extensions, in which case wglGetProcAddress will again return NULL for those extensions. So if you put logic in for handling the NULL return on wglGetProcAddress(), you should still be fine even for deprecated extensions. It just might become necessary for you to implement better alternatives, or to make newer extensions default.
As far as the versioned API headers go, the changes to the headers are mostly just changes to allow access to new functions returned by wglGetProcAddress(). So if you include the API header for version 2, you're good to go as long as you only need the extensions for OpenGL 2. If you need to access functions/extensions that were added in version 3, then you just replace your version 2 header with the version 3 header, which just adds some additional function pointer typedefs associated with the new extensions, so that when you call wglGetProcAddress(), you can cast the return value to the right function pointer. Example:
PFNGLGENQUERIESARBPROC glGenQueriesARB = NULL;
...
glGenQueriesARB = (PFNGLGENQUERIESARBPROC)wglGetProcAddress("glGenQueriesARB");
In the above example, the typedef for PFNGLGENQUERIESARBPROC is defined in the API headers. glGenQueriesARB was added in 1.2 I believe, so I'd need at least the 1.2 API headers to get the definition of PFNGLGENQUERIESARBPROC. That's really all the headers do.
One more thing I want to mention about 3.1. Apparently with 3.1 they're deprecating a lot of OpenGL functionality that my company has used pretty ubiquitously, including display lists, the glBegin/glEnd mechanisms, and the GL_SELECT render mode. I don't know much about the details, but I don't see how they can do that without having to create a new opengl32.lib to link with, because it seems that most of that functionality is embedded into opengl32.lib, and not accessed through wglGetProcAddress. Additionally, is Microsoft going to include that new opengl32.lib in their Visual Studio versions? I don't have an answer for those questions, but I would think that, even though 3.1 deprecates it, this functionality is going to be around for a long time. If you keep linking with your current opengl32.lib, it should continue to work almost indefinitely, although you may lose hardware acceleration at some point. The vast majority of OpenGL tutorials available on the web use the glBegin/glEnd methods for drawing primitives. The same is true for GL_SELECT, although a lot of hardware no longer accelerates GL_SELECT render mode. Even opengl.org's tutorial's use the supposedly deprecated glBegin/glEnd methods. And I have yet to find a "getting started" tutorial that uses only 3.1 features, avoiding the deprecated functionality (certainly if someone knows of one, link me to it). Anyway, while it seems 3.1 has thrown away a lot of the old for all new stuff, I think the old stuff will still be around for quite a while.
Long story short, here's my advice. If your OpenGL needs are simple, just use the basic OpenGL functionality. Vertex arrays are supported on versions 1.1 - 3.1, so if you're looking for maximum lifetime and you're starting fresh, that's probably what you should use. However, my opinion is glBegin/glEnd and display lists are still going to be around for a while even though they are deprecated in version 3, so if you want to use them, I wouldn't fret too much. I would avoid GL_SELECT mode for picking in favor of an alternate method. Many of the hardware vendors have considered GL_SELECT deprecated for years now, even though it just got deprecated with version 3. In our application we get lots of issues where it doesn't work on ATI cards and integrated GMA cards. Subsequently we just implemented a picking method using occlusion queries which seems to fix the problem. So to do it right the first time, avoid GL_SELECT.
Good Luck.