The quest for the best memory handling is finally over?
Virtual Address Space Problems
Ever since ArmA was published, one of the biggest problems with stability and compatibility was caused by virtual address space exhaustion. The problem is the game needs to store a lot of data, using a lot of virtual addresses, and at the same time a large amount of virtual address space are used by other parts of the system, most notable of them being the graphics card driver. The problem started even before ArmA was published, back in the times of Operation Flashpoint, when the first graphics cards with 256 MB RAM or more appeared. Flashpoint was often even less stable than ArmA on the same system, and the reason is that it used memory mapping as a way to access files. While being fast and easy to use, memory mapping uses huge amounts of virtual space. As a workaround we provided a way to access files without mapping them (the well known -nomap command line). For ArmA we redesigned file access completely, not using file mapping at all, but using normal file API instead. At the same time we switched to Overlapped I/O, so that we can load the textures and other resources in the background. We (and you) learned the hard way that this was not enough.
Sky's the Limit?
Now what happens is that each application is given 2 GB of address space by the operating system. 2 GB sounds somewhat limiting, however it's not that bad - few people have more RAM in their computers anyway. Now the important thing to note is we are not talking about RAM (physical memory) here, we are talking about the address space. The application is limited to 2 GB address space, so if your system has 512 MB RAM or 4 GB - it does not matter here at all.
Those 2 GB are used for everything the application needs. Recent graphics cards drivers often eat several hundred of MB from this. To make it even worse, there are some system components which do not eat big parts, but eat them here and there, making the space fragmented into multiple small regions. My experience shows it is almost impossible for the game code to allocate more than around 700-900 MB of virtual space without the system becoming extremely unstable - and we are not talking one big contiguous allocation here, but rather several 10-100 MB sized chunks. To make it a nightmare, driver programmers seem not to have noticed the fact the address space is a limited resource, and the drivers often do not handle virtual allocation failure well. This mean once a game allocates too much of the virtual space, very strange things start to happen, like a triangle mess on the screen, or sometimes even a system reboot.
The Future - 64 bits
In the long term, the solution to this will be to move to a 64 bit operating system and to compile the game as a 64 bit application. Such a solution is not realistically possible for ArmA, partly because not enough people use a 64b OS yet, partly because the change is too big and would take too much time and resources.
We have performed various experiments and optimizations with mixed results in several patches (see this forum topic, or this into the new forum) , but none of them worked really well, with every solution either the performance or the stability suffered on some systems. Meanwhile, as we were continuing ArmA 2 development, we needed to allocate even more textures and more memory, and we were becoming more and more limited by this, up to a point where the game was not able to run stable on many systems at all .
Memory with No Address
And then one day, an idea came, as such ideas usually do, from nowhere. As usual with such ideas, once the system is implemented, it seems very obvious, but I never heard about it being used before, therefore I am quite proud to have invented it. I call this technique Non-addressable Memory Store.
The technique is based on using the File Mapping API. Yes, you hear well, the same API which caused problems with Flashpoint, but this time it is used a very different way, using it not to read the files, but as a way to allocate memory:
- on game initialization, enough memory is reserved and committed by using CreateFileMapping API. This consumes the physical memory, but no virtual addresses
- when storing or retrieving a page of data, temporary view is created using MapViewOfFile, which is destroyed again once access is finished. This uses very little of the virtual space (64 KB). Typically only a few pages are mapped into the memory this way at the same time.
Windows is handling this pattern very well, while the space can be backed up by the "page file" in theory, in practice it works in such a way that as long as there is enough of physical memory available, there is zero page file traffic and all information is handled in the physical memory only. The memory kept in this type of storage is not mapped to any virtual addresses at all, it is identified by the offset in the file mapping instead. Addresses are assigned only temporarily, when the content of the cache is read. This is possible thanks to the fact that the data stored in the file cache are location independent (do not contain any pointers).
The Barrier Is Broken
This technique in theory allows even for a 32b application to use more memory than 32b address space allows - the size is limited only by the possible size of the file mapping, and by the free RAM, therefore with 32b OS it would be possible to manage about 3 GB of data, and with 64b OS even more. This is of not something we do in ArmA, experiments have shown having a file cache size larger than 512 MB brings very little improvement, as we are not reading that much data anyway, but for some other applications it might be a useful option. It is not limited to file caching, but it can be used to store any content, with one significant limitation - the application must never point into a memory allocated this way using ordinary permanent pointers, as the pointers would become invalid once the memory is unmapped.
What this brings to ArmA is that the internal file cache, which is sized between 100 - 500 MB depending on how much RAM you have in your system (or depending on what -maxmem command line argument you use), is now using almost no virtual space at all. Those several hundreds MBs seem to be enough to keep the rest of the system happy. So far it seems to have fixed the infamous "Cannot allocate system memory surface" on almost all systems we have tested, without having to limit the memory used by the game.
Available to You in the Nearest Patch
This memory handling will be published in the upcoming 1.11 patch, and I hope you will like it as much as I do.