virtual memory allocation

views:

answers:

virtual memory allocation

what are my options when i need to allocate virtual memory beyond the limits of system memory?

paging file is unlimited(limited by available disk space) so why could not i use it to allocate memory beyond the limits of physical memory limit or OS memory limits? why did they limit the virtual memory with the boundaries of address space? i know CPU could work with physical memory but why not OS handles this for me when i am accessing some portion of virtual memory?

+2 A:

The OS does take care of this for you. You do not have to think about when physical memory is fully utilized because the virtual memory system hides this from you. The limit on the address space of an application is a result of the number of bits that are allocated for storing memory addresses in the architecture.

Edit Re: comments

I think you may be confusing the amount of memory available to a given process with the total amount of virtual memory being managed by the OS, which is shared among processes. There is a limit on the total amount of committed virtual memory by all processes, but this commit limit is NOT the same as the address space limit for an individual process. The total commit limit is loosely the amount of physical memory + the pagefile size. So it's possible to tune your pagefile size to increase or decrease this number.

bshields 2010-06-25 18:02:56

but when i allocate all physical memory why does not allocation go beyond that in virtual memory. why 32 bits(or 64 bits) limits me? why not page swapping works for the allocations of the process?

lockedscope 2010-06-25 18:12:52

It does go beyond that. If you have a machine with 512MB of RAM you can still allocate ~2GB of memory in a 32 bit application. And if you have a machine with 12GB of RAM you can still only allocate ~2GB of memory in a 32 bit application.

bshields 2010-06-25 18:18:29

so for a process how could i force some physical memory paged to disk(without freeing memory) to make that portion available for allocation? are there any api function for this? currently, i am using VirtualAlloc.

lockedscope 2010-06-25 19:08:28

VirtualAlloc is a very low-level API that should only be used in really rare cases. Unless you REALLY know what you're doing, you should use HeapAlloc or plain old new/malloc. This answer might have some info for you though: http://stackoverflow.com/questions/2447922/how-can-i-increase-my-application-memory-till-using-the-virtualalloc-function-w

bshields 2010-06-25 19:26:57

so to sum up, it is not possible to have more *virtual* memory than the system limits (2 or 3GBs on 32bit...) unless using memory mapped files, right?

lockedscope 2010-06-25 19:36:24

@lockedscope I updated the answer to address your comment.

bshields 2010-06-28 00:34:11

Bshields has the direct answer. I just wanted to add that if you're working on Windows, the esteemable Mark Russinovich has a very detailed and well written post on virtual memory here: http://blogs.technet.com/b/markrussinovich/archive/2008/11/17/3155406.aspx.

fatcat1111 2010-06-25 18:08:32

i have already read it, its very helpful. i am just talking some imaginary case, not for a solution of real scenario.

lockedscope 2010-06-25 18:14:12

Virtual memory and address spaces are two separate notions. Memory paging is yet another thing.

The size of the address space is limited to the range locations that can be addressed. This is a limitation of the CPU and the mode it is operating in. For example, a typical 32-bit application running on a 64-bit system with 32 GB of memory still has a 32-bit address space. The fact that more storage is available doesn't change the fact that (ignoring for the simplicity the existence of PAE) the 32-bit program only uses 32-bit addresses.

Virtual memory is realy just the notion that the operating system can control a set of mappings between virtual memory pages and physical memory pages.

Memory paging allows the operating system to make it so that some of the virtual pages are stored on disk instead of physical memory. This can allow more virtual memory to be allocated than there is physical memory on a system.

torak 2010-06-25 18:16:29

ansaurus

tags:

views:

answers:

virtual memory allocation

related questions