64-Bit VB.NET Allocating > 2GB of RAM (.NET bug?)

Question

I have a 64 bit VB.NET application and want to allocate a buffer > 2GB in size.

In the following code both the "new" and the "ReDim" throw an "OverflowException."

How can I allocate buffers > 2GB when these functions only accept signed 32 bit values?

(Is this possible in C#?)

Edit - I am running WinXP 64 with 4GB of RAM.

Dim width As Long = 19005
Dim height As Long = 32768

Dim buffer() As Byte = New Byte((width * height * 4) - 1) {}

Dim size As Long = (width * height * 4) - 1
ReDim buffer(size)

Answer 1

The following works [in theory] (C# syntax):

Array.CreateInstance(typeof(int[]), 0L);

Edit: Make a type with a fixed-size allocated array of 1GB arrays. You can re-index in the Item property via a shift.

Answer 2

Apparently it is not possible to allocate more than 2GB even under 64 bit .net application running on a 64 bit OS.

I find this to be very disappointing and completely without regard for what 64 bit applications and OSs are made for. I am dealing with gigantic images and would like to be able to work with the raw bytes all in RAM at once. Now I have to implement paging algorithms to limit the chunks to 2GB.

Hey Microsoft, hows abouts you fix this in the coming .NET release? Yes, I said fix. That's because it's broken. How do you expect 64 bit applications to take off when you do stupid things like this. (Can you tell that I am annoyed.) Thanks for listening.

Link

http://blogs.msdn.com/joshwil/archive/2005/08/10/450202.aspx

Answer 3

I think the UnmanagedMemoryStream does what you need. MSDN doc for UnmanagedMemoryStream

I think it's a bad idea, to allocate a huge chunk of memory in a garbage collected environment, since most garbage collectors are optimized for small & short lived object. So using raw memory is generally a better and more performant solution for very large objects.

Answer 4

You may have to use memory mapped files for this, take a look at the MapViewOfFile function.