Simple and sweet: Console applications (basic C programs using printf and such) are easily and cheaply done with the Tiny C Compiler - a no frills, no gui, complete C complier.
http://bellard.org/tcc/
However, C development is relatively simple on Visual Studio as well. The following instructions will set Visual C++ up as a good C compiler, and it will produce console applications at first, and yo can move up into more complex windows apps as you go.
- Get the Visual Studio C++ edition (express is fine)
- Start a new project - disable pre-compiled headers (maybe the wizard will let you do this, maybe you'll have to change the compiler settings once inside the project)
- Delete everything inside the project.
- Create a new "example.c" file with the hello world example
- Compile and away you go.
Alternately, get a linux virtual machine, or Cygwin. But as you already have Visual Studio, you might as well stick with what you know.
As an aside, this isn't Atwood learning C finally, is it? No ALTs! ;-D