NexusDB

Everything I'm saying now holds true for Windows. Most of it holds true for all operating systems running on the x86 platform in protected mode as most of it is dictated by hardware constraints. There are some things that I'll leave out of this description as they only complicate matters and have no major impact for user mode applications.

"Physical memory" refers to memory modules that are physically connected to the memory bus and which the CPU can access using physical addresses. Depending on the hardware such a "physical memory address" can have more then 32 bit. As user mode application developer you'll never get to see a physical address. Physical memory is separated into blocks of 4kb. The OS keeps track of what each of these 4kb blocks is used for. As user mode application developer you have no direct control over this at all.

Each process has it's own so called "virtual address space". As you are using 32bit pointers, there are 4294967296 possible addresses in a "virtual address space". 4GB. (Except in special cases) The upper 2GB of these addresses are reserved for the operating system, leaving the lower 2GB for the application. The address space is separated into blocks of 64kb. For each block the operating system keeps track if the block is currently free or if it's "reserved". Reserving virtual address space doesn't use any "physical memory". It only means that the OS now knows that a region of addresses is now in use. There are only 2 APIs that can reserve address space: VirtualAlloc and MapViewOfFile (other API functions may implicitly call these 2. e.g. LoadLibrary). In the end it all comes back to a central, per process, management structure in the kernel which efficiently prevents the same address region from being used for multiple purposes at the same time. It is *not possible* to get "address space" from the OS in sizes other then multiples of 64kb.

In addition, the address space of each process is divided into blocks of 4kb. For each of these 4kb blocks the OS keeps track of the following states:

• uncommitted; every block that is not in a reserved area of address space is implicitly also uncommitted.
• committed, empty; the block contains only 0's. There is no physical memory backing this block.
• committed, present in memory; the OS knows a mapping for this block to a "physical memory block".
• committed, present in file; the OS has a file handle and a file offset for this block (e.g. the swap file, or a file that has been mapped into memory as memory mapped file" this includes dll/exe images)

There are only 2 APIs that can commit address space: VirtualAlloc and MapViewOfFile (again, may be indirectly called). "Committing" address space *doesn't* use any "physical memory"!

Every time that the CPU has to access memory using a "virtual address" (which means basically any form of memory access in user mode applications) it has to look up a mapping from the virtual address to the actual physical address. Depending on the state of the 4kb page the address is in the following will happen:

• uncommitted; an access violation is raised.
• committed, empty; the OS takes a zeroed page of physical memory and associates it with this virtual page. State changes to "committed, present in memory". *This actually USES physical memory.*
• committed, present in memory; nothing to do, already have a mapping...
• committed, present in file; very similar to the "empty" case, just that the OS loads the contents from the file into the memory page. *This actually USES physical memory.*

So, what does all this have to do with the original question?

• there is 2GB address space available in a process. No matter if you have 32MB or 16GB of physical memory.
• address space can be reserved in blocks of 64kb. reserving address space doesn't use "memory"
• reserved address space can be "committed" in blocks of 4kb. committing address space doesn't use "memory".

A look at the code shows us:

const
  nxcl_1KB                        = 1024;
  nxcl_SystemAllocationSize       = 64 * nxcl_1KB;
 
constructor TnxMemoryStream.Create(aInitial : Integer);
begin
  if aInitial < nxcl_SystemAllocationSize then
    aInitial := 8 * nxcl_SystemAllocationSize;
...
and:
 
    Stream := TnxMemoryStream.Create(1024);

• 1024 is < 64kb
• 8 * 64 = 512kb
• 4016 * 512 kb = 2008MB = 1.9609375GB
• you are running out of address space.