Does anyone know why I would get an OutOfMemoryException when TaskMgr.exe says my process is only taking up ~1GB of memory?
There probably isn't enough contiguous free memory to do what you want.
See here.
Could it be that the IDE limits the amount of memory you can use? In which case, you can give run parameters to expand the amount of memory available. At least, that's how it is in eclipse.
If you have no swap space defined, increasing the Virual Memory size may help.
Also, realize that, when you're using .NET, the process memory limits on 32bit .NET code tend to be much lower than the theoretical limits. It's very common to start receiving out of memory errors somewhere between 1.2GB and 1.6GB of memory usage - well below the theoretical 2GB limit. (This is true even if allocating small chunks of memory.)
Moving to a 64bit platform would most likely eliminate this issue. (It's possible that your 1GB is not entirely accurate, either - TaskMgr does a very poor job of estimating memory allocation.)
If you want a better tool than Taskmanager download The SysInternals Suite from MS and use the Process Explorer Utility to monitor your app while its running. Select View->Select Columns->Process Memory and set the columns you want to monitor. It gives you a much better read on whats happening in memory.
That being said my bet is lack of contiguous memory available in your Heap is the problem.