Preventing a heavy process from sinking in the swap file

Question

Our service tends to fall asleep during the nights on our client's server, and then have a hard time waking up. What seems to happen is that the process heap, which is sometimes several hundreds of MB, is moved to the swap file. This happens at night, when our service is not used, and others are scheduled to run (DB backups, AV scans etc). When this happens, after a few hours of inactivity the first call to the service takes up to a few minutes (consequent calls take seconds).

I'm quite certain it's an issue of virtual memory management, and I really hate the idea of forcing the OS to keep our service in the physical memory. I know doing that will hurt other processes on the server, and decrease the overall server throughput. Having that said, our clients just want our app to be responsive. They don't care if nightly jobs take longer.

I vaguely remember there's a way to force Windows to keep pages on the physical memory, but I really hate that idea. I'm leaning more towards some internal or external watchdog that will initiate higher-level functionalities (there is already some internal scheduler that does very little, and makes no difference). If there were a 3rd party tool that provided that kind of service is would have been just as good.

I'd love to hear any comments, recommendations and common solutions to this kind of problem. The service is written in VC2005 and runs on Windows servers.

Answer 1

As you mentioned, forcing the app to stay in memory isn't the best way to share resources on the machine. A quick solution that you might find that works well is to simply schedule an event that wakes your service up at a specific time each morning before your clients start to use it. You can just schedule it in the windows task scheduler with a simple script or EXE call.

Answer 2

In terms of cost, the cheapest and easiest solution is probably just to buy more RAM for that server, and then you can disable the page file entirely. If you're running 32-bit Windows, just buy 4GB of RAM. Then the entire address space will be backed with physical memory, and the page file won't be doing anything anyway.

Answer 3

A third approach could be to have your service run a thread that does something trivial like incrementing a counter and then sleeps for a fairly long period, say 10 seconds. Thios should have minimal effect on other applications but keep at least some of your pages available.

Answer 4

I'm not saying you want to do this, or that it is best practice, but you may find it works well enough for you. It seems to match what you've asked for.

Summary: Touch every page in the process, on page at a time, on a regular basis.

What about a thread that runs in the background and wakes up once every N seconds. Each time the page wakes up, it attempts to read from address X. The attempt is protected with an exception handler in case you read a bad address. Then increment X by the size of a page.

There are 65536 pages in 4GB, 49152 pages in 3GB, 32768 pages in 2GB. Divide your idle time (overnight dead time) by how often you want (attempt) to hit each page.

BYTE *ptr;

ptr = NULL;
while(TRUE)
{
    __try
    {
        BYTE b;

        b = *ptr;
    }
    __except(EXCEPTION_EXECUTE_HANDLER)
    {
        // ignore, some pages won't be accessible
    }

    ptr += sizeofVMPage;

    Sleep(N * 1000);
}

You can get the sizeOfVMPage value from the dwPageSize value in the returned result from GetSystemInfo().

Don't try to avoid the exception handler by using if (!IsBadReadPtr(ptr)) because other threads in the app may be modifying memory protections at the same time. If you get unstuck because of this it will almost impossible to identify why (it will most likely be a non-repeatable race condition), so don't waste time with it.

Of course, you'd want to turn this thread off during the day and only run it during your dead-time.

Answer 5

The other thing to ensure is that your data is localized.

In other words: do you really need all 300 MiB of the memory before you can do anything? Can the data structures you use be rearranged so that any particular request could be satisfied with only a few megabytes?

For example

• if your 300 MiB of heap memory contains facial recognition data. Can the data internally be arranged so that male and female face data are stored together? Or big-noes are separate from small-noses?

• if it has some sort of logical structure to it can be it sorted? so that a binary search can be used to skip over a lot of pages?

• if it's a propritary, in-memory, database engine, can the data be better indexed/clustered to not require so many memory page hits?

• if they're image textures, can commonly used textures be located near each other?

Do you really need all 300 MiB of the memory before you can do anything? You cannot service request without all that data back in memory?

Otherwise: scheduled task at 6 ᴀᴍ to wake it up.