memory reserving and committing

Question

I am reading < Windows via C/C++ > and here's some quotation.

When a process is created and given its address space, the bulk of this usable address space is free, or unallocated. To use portions of this address space, you must allocate regions within it by calling VirtualAlloc. The act of allocating a region is called reserving.

To use a reserved region of address space, you must allocate physical storage and then map this storage to the reserved region. This process is called committing physical storage.

After you have reserved a region, you need to commit physical storage to the region before you can access the memory addresses contained within it. The system allocates physical storage committed to a region from the system's paging file.

Here are a couple of questions:

• Why do we need to follow the reserve-comit paradigm when using memory? i.e. Why do we need to follow this 2-step paradigm instead of allocating some physical memory directly and use it?

• If the the physical storage committed to a region is allocated from the system's paging file, why do we need the RAM (sounds ridiculous)? In my opinion, an address space region should be mapped to RAM (through the paging mechanism), and the RAM pages should be backed by the paging file.

Maybe this question could be answered by explaing the following 2 aspects:

• What does reserving do?

• What does committing do?

Update - 1 2:48 PM 11/23/2010

It is the following quotation from < Windows via C/C++ 5th edition > that makes me puzzled.

...It is best to think of physical storage as data stored in a paging file on a disk drive. So when an application commits physical storage to a region of address space by calling the VirtualAlloc function, space is actually allocated from a file on the hard disk.

After you have reserved a region, you need to commit physical storage to the region before you can access the memory addresses contained within it. The system allocates physical storage committed to a region from the system's paging file.

So, where is the RAM? What if I configure my machine to have no page file?

Boozy

Answer 1

The whole point of reserving pages is to ensure that contiguous address space is available for some task. For example, we want the stack to able to grow to 1MB, but we don't want to commit all of that memory because it won't actually be used yet. So we reserve 1MB of pages, but commit a small amount, like 64kB. By setting up a guard page at the end of the committed region, we can detect when we need to commit more memory.

Committing memory is the act of mapping some kind of storage to a page. This can be located in physical RAM, where it is part of the working set, or in the pagefile. It can also be mapped in or private memory. NtAllocateVirtualMemory/VirtualAlloc can reserve and commit at the same time, for convenience.

EDIT for updated question: When you commit pages, that is charged against the process pagefile quota / system-wide commitment limit. This limit is determined by the amount of physical RAM available and the size of the pagefile. This doesn't actually mean the pages are stored in, or written to the pagefile. They may be if memory is low, but otherwise pages are mostly kept in physical memory.

Scincoid

Answer 2

• You don't actually have to follow 2-stage reserve/commit allocation scheme.

  The point is that VirtualAlloc and VirtualFree may do several things. And sometimes it's really useful to do so. You don't have to however.

• Committed memory region is the one for which the system allocates physical storage.

  You don't have to worry about where exactly it's allocated: RAM or page file. This should be transparent to you (unless you're writing a kernel-mode device driver). Moreover, most of the memory pages can be swapped out to the page file and loaded into RAM on-demand.

  It's just some committed memory pages don't require binding with the page file, because they're already bound to another physical storage. Memory-mapped files are the example of this.

  In normal scenario when you commit a memory pages, use it for some time period and free it - most likely it won't reach the page file at all.

Sabbat

'Physical storage' isn't pagefile, it's either RAM or pagefile. The only way to get the OS to commit a memory region is to read/write to the page. – Abhorrent

Memory-mapped file could also be a kind of physical storage. – Boozy