UNIX vs Windows memory deallocation

Question

My understanding is that in unix, when memory is freed, the memory doesn't get returned back to the operating system, it stays in the process to be used again for the next call to malloc.

On windows, I understand that the memory actually gets returned to the operating system.

Is there any big difference between these two ways of doing things or are they just two different ways of doing the same thing? And if there are any pros/cons to these two methods, what are they?

EDIT: Thanks for the clarification. I had always thought this was an OS thing (since processes never seem to decrease in size in UNIX-like systems, but do in windows).

Answer 1

From this Memory Management article

Malloc does not normally return the freed memory to the operating system; it remains owned by the process until it terminates. The process can reuse it the next time it requests more memory, but other programs will not have access to it, even if no other memory is available. As a corollary, then, the memory footprint of a program is the size of the largest allocation(s) made at any one time. Thus it is always wise to free objects you do not need, especially large ones as soon as possible, to minimize this footprint.

That article does suggest that in Windows, at least for C program, the memory is not returned to the OS.

So I am not sure about your generalization about windows memory deallocation.

That said, you can try and Emulating UNIX Memory Management Under Microsoft Windows, by implementing low level system calls sbrk and mmap/munmap on Windows.

Answer 2

I don't know about Windows but, on UNIX, the brk() call is used to bring more memory into the address space for use by the malloc() calls.

I've never seen this memory returned to the OS until the process terminates. You can usually see this with tools such as top.

I suspect the behavior would be the same for Windows but I do know that Windows has other allocation functions than malloc() which may do this (part of the Win32 API).

Answer 3

Note that I know much more about Windows than Unix in what follows ...

What actually happens with memory allocation and deallocation isn't quite what you describe, in any case. This is because there's two very different concepts at work here: the physical memory that the computer possesses, and the virtual address space of the program, the memory that your program thinks it can use.

When your program requests more memory from the operating system, what is really happening is that previously unavailable virtual address space in your program is being set up as accessible by the program. Modern operating systems don't work by just having a pool of "real" (that is, physical) memory that it hands out to processes when they make an allocation request: it maintains the virtual address space for every running program, and, when programs actually access parts of that virtual address space, ensures that this is mapped to some physical memory, possibly by swapping out some part of another program's address space to the swap file on disk.

As an example of this, on Windows each thread starts with (by default) a megabyte of stack space allocated for it. This doesn't mean that every thread consumes a megabyte of the machine's physical memory: it is simply that the address space is set up so that it is available for use. In this sense it doesn't really work to think about the operating system giving your program memory and then the program giving it back - it just doesn't work like that.

Answer 4

It all depends on which C runtime library you use. There is NO specific UNIX way or WINDOWS way. Each compiler vendor(HP, SUN, MS, GNU) ships with their own runtime library that contains the logic for malloc. each implementation of malloc will operate same/different depending on the OS. Neither UNIX/LINUX/Windows needs that a free "ACTUALLY RETURN" the memory back to the OS. That would be too expensive (since your alloc() would be in very small chunks)

Recently mozilla Firefox borrowed a malloc() implementation from *BSD OS. They chose to use a different malloc than what their compiler vendor (multiple in this case -- gcc and VC++) shipped. Since they wanted a certain behavior, they got what they wanted.

Answer 5

The only operating system where you can't easily give allocated memory back to the system is OS X - quoting Firefox 3 Memory Usage:

After extensive testing and confirmation from Apple employees we realized that there was no way for an allocator to give unused pages of memory back while keeping the address range reserved.. (You can unmap them and remap them, but that causes some race conditions and isn’t as performant.) There are APIs that claim to do it (both madvise() and msync()) but they don’t actually do anything.

Answer 6

As other mentioned, this is more linked to the malloc implementation than the OS per se. On linux, with glibc, the memory is actually always returned to the OS above a certain size: glibc malloc uses mmap for big allocations (controlled by MMAP_THRESHOLD), and in that case, free calls munmap, which frees automatically the reserved memory. Below that threshold, it uses brk, and free does not "return" the memory in that case.

Note that the above explanation is not exact: to be precise, you need to know the difference between physical memory, virtual memory, etc... This is well explained here:

http://blogs.msdn.com/oldnewthing/archive/2004/08/22/218527.aspx

Answer 7

Other posters have commented on the platform specific angle. But since you ask specifically about malloc, lets see what the C Standard says:

"The free function causes the space pointed to by ptr to be deallocated, that is, made available for further allocation."

Which seems a pretty clear requirement that the memory is not returned to the OS. You occasionally see programs relying on this behaviour:

int main(void)
{

    void *p = malloc(AS_MUCH_MEMORY_AS_I_WILL_EVER_NEED);

    if (p != 0)
    {
        free(p);
        /* malloc should always work for rest of program */
    }
}

However, when this question has come up on comp.lang.c, some posters have pointed out this section:

"The malloc function returns either a null pointer or a pointer to the allocated space."

This suggests that any call to malloc may fail. It seems that the intent of the Standard is that memory is not returned to the OS, but the issue is not 100% certain in the eyes of language lawyers.

Answer 8

There isn't much difference between Windows and Unix with respect to that.

In both, there are two levels of allocation. The operating system allocates memory to the process in large chunks (one page or more; on x86, the page size is usually 4096 bytes). The runtime libraries, running within the process, subdivide this space and allocate parts of it to your code.

To return the memory to the operating system, first all the memory allocated from one of these large chunks has to be released to the runtime library. The runtime library then can, if it wants, tell the operating system to release that chunk of memory.

On Linux, you have brk and mmap. brk controls the size of of a large chunk of memory allocated to your process; you can expand or shrink it, but only at one end. malloc traditionally expands this chunk of memory when it needs more memory to allocate from, and shrinks it when possible. However, shrinking is not easy; it takes a single one-byte ill-timed allocation at the end to make it unable to shrink even if everything before that allocation has been freed. This is the source of the "Unix doesn't release memory back" meme.

However, there's also anonymous mmap. Anonymous mmap requests a chunk of memory from the operating system, which can be placed anywhere in the process memory space. This chunk can be returned easily when it's not needed anymore, even if there are later allocations which weren't released yet. malloc uses also mmap (particularly for large allocations, where a whole chunk of memory can be easily returned after being freed).

Of course, on both Windows and Linux if you do not like the behavior of the memory allocator (or allocators) from the runtime libraries, you can use your own, asking memory from the operating system and subdividing it the way you want (or sometimes asking memory from another allocator, but in larger blocks). One interesting use is to have an allocator for all the memory associated with a task (for instance, a web server request), which is completely discarded at the end of the task (with no need to free all the pieces individually); another interesting use is an allocator for fixed-size objects (for instance, five-byte objects), which avoids memory fragmentation.