High rate data stream and memory deficiency

views:

answers:

High rate data stream and memory deficiency

I have a program that accumulates traffic from network interface in 800 Mb/s. This program is developed by Delphi, also in 32bit platform(Delphi does not support 64bit architecture). I'm writing received date on the memory(RAM) and after a while (unknown and depends on received data), write a block of received data (unknown size and depends on received data) into hard disk and free the memory(RAM).

When I run the application, after some times, it throws "Out Of Memory" exception. What should I do to prevent this exception?

That's a very high data rate - I assume it's Mbits (100 MBytes/second). Only the fastest SSDs will be able to keep up, and that's only if your programming is 100% efficient. Check the table of sequential write speeds: http://www.anandtech.com/storage/showdoc.aspx?i=3702&p=9

If you can throw more RAM at the problem, I'd suggest going to 64-bit mode and allocating your entire RAM buffer all at once.

Mark Ransom 2010-01-25 23:19:10

Thanks. But as I mentioned my program is developed by Delphi 2009, and the Delphi does not support 64bit platform.Moreover that speeds was related to sequential read and write, while I need random access to hard disk.

salman 2010-01-26 07:15:19

salman 2010-01-26 07:23:05

@salman: If you need random access, you can see for yourself that the fastest available disk only has half the speed you require. My advice still stands, even for 32 bit - allocate as much RAM as you can without swapping, then fill it.

Mark Ransom 2010-01-26 13:54:27

How can I allocate RAM without swapping, and how can fill it after allocation? :)

salman 2010-01-27 05:08:50

ansaurus

tags:

views:

answers:

High rate data stream and memory deficiency

related questions