memory

Is there a more efficient way to reconcile large data sets?

I've been tasked with reconciling two big data sets (two big lists of transactions). Basically i extract the relevant fields from the two data sources into two files of the same format, then compare the files to find any records that are in A but not in B, or vice versa, and report on them. I wrote a blog entry on my best efforts achievi...

Order of local variable allocation on the stack

Take a look at these two functions: void function1() { int x; int y; int z; int *ret; } void function2() { char buffer1[4]; char buffer2[4]; char buffer3[4]; int *ret; } If I break at function1() in gdb, and print the addresses of the variables, I get this: (gdb) p &x $1 = (int *) 0xbffff380 (gdb) p...

Why is there a difference in memory address of environment variable when running a program.

I write a C program gets in an environment variable's name and print out it's memory address, simply use the getenv() function to do the job. Let's say I wanna have the address location of PATH --> ./test PATH. But when I debug that program in gdb, the memory location of that variable is different. Can you explain in detail why is there ...

PHP cli Memory usage optimization

Hello I am trying to code a custom url_rewriter for squid. & also with using some other url_rewriter programs like squidGuard so have to use a wrapper to able use both or any other program. when i try to loop with php. (that's the way how squid communicates with external programs. STDIN/STDOUT. it gives you a url & you have to sen...

Why does Java have such a large footprint?

Java - or at least Sun's Hotspot JVM - has long had a reputation for having a very large memory footprint. What exactly is it about the JVM that gives it this reputation? I'd be interested in a detailed breakdown: how much memory goes to the runtime (the JIT? the GC/memory management? the classloader?) anything related to "auxiliary" A...

MySQL Session Table Approach

Hi all, I am developing a multi-tenant web application using LAMP. All my user session data is currently stored in mysql with table type InnoDB. Is there any way that I can use MEMORY (used to be HEAP) table type to store the current sessions and use the garbage collector function of the session handler to move sessions between the Inn...

Allocating more than 1,000 MB of memory in 32-bit .NET process

I am wondering why I'm not able to allocate more that 1,000 MB of memory in my 32-bit .NET process. The following mini application throws an OutOfMemoryException after having allocated 1,000 MB. Why 1,000 MB, and not say 1.8 GB? Is there a process-wide setting I could change? static void Main(string[] args) { ArrayList list = new Ar...

Cost of passing an optional parameter to a method rather than computing it

I have a memory block that is divided into a series of location that can be retrieved and returned by client code. The method that returns locations back looks like this: void ReturnLocation(void *address) { int location = AddressToLocation(address); // I need the location here // some code DoSmthA(location); } void DoSmth...

Performance of list(...).insert(...)

I thought about the following question about computer's architecture. Suppose I do in Python from bisect import bisect index = bisect(x, a) # O(log n) (also, shouldn't it be a standard list function?) x.insert(index, a) # O(1) + memcpy() which takes log n, plus, if I correctly understand it, a memory copy operation for x[...

Out of dynamic memory in yy_create_buffer()

Has anyone seen this error when working with a PHP application "out of dynamic memory in yy_create_buffer()"? The error message that appears in the php error log is: Fatal error: out of dynamic memory in yy_create_buffer() in Unknown on line 0 I have not been able to identify a reproducible case. Increasing memory_limit has no affect o...

Why does my program crash when accessing a property with self. and a synthesized accessor?

I have data object class: @interface Item: NSObject { NSString *title; NSString *text; } @property (copy) NSString *title; @property (copy) NSString *text; @end @implementation Item @synthesize text; - (void)updateText { self.text=@"new text"; } - (NSString *)title { return title; } - (void)setTitle:(NSString *)aS...

(C#) Arrays, heap and stack and value types

int[] myIntegers; myIntegers = new int[100]; In the above code, is new int[100] generating the array on the heap? From what I've read on CLR via c#, the answer is yes. But what I can't understand, is what happens to the actual int's inside the array. As they are value types, I'd guess they'd have to be boxed, as I can, for example, pas...

Why does Windows reserve 1Gb (or 2 Gb) for its system address space?

Hi; It's a known fact that Windows applications usually have 2Gb of private addess space on a 32bit system. This space can be extended to 3Gb with the /3Gb switch. The operating system reserves itself the remaining of the 4Gb. My question is WHY? Code running in kernel mode (ie device driver code) has its own address space. Why, on t...

<list> throws unhandled exception when calling push_front()

I'm working on a GUI in SDL. I've created a slave/master class that contains a std::list of pointers to it's own slaves to create a heirarchy in the GUI (window containing buttons. Button a label and so on). It worked fine for a good while, until I edited a completely different class that doesn't effect the slave/master class directly. T...

A fail-safe way to prevent GD image library from running out of memory? (PHP)

Is there a way to prevent the PHP GD image library from running out of memory? If too large an image is uploaded, GD tends to run out of memory, terminating the script. I'd like it to throw a catchable exception or something to that extend, but alas it doesn't. Right now I'm using a cobbled-together script that first issues an ini_set('...

Why does a 32-bit OS support 4 GB of RAM?

Just reading some notes in a purdue lecture about OSs, and it says: A program sees memory as an array of bytes that goes from address 0 to 2^32-1 (0 to 4GB-1) Why 4 GB? ...

How can I calculate the free address space on Windows?

How can I calculate the amount of free virtual address space my process have on Windows? My application need to limit the amount of address space used. So I need to estimate how many memory I have consumed and how many virtual memory is left. If I have just a few hundred megabytes of address space left, my process begins to use a custo...

Memory problem on iPhone 3G/2G, sdk 3.0

I encountered a crash in my game, here's the crash log (it's a stress test): PID RPRVT RSHRD RSIZE Command 1 340K 224K 436K launchd 14 124K 160K 216K update 15 568K 164K 620K syslogd 16 792K 612K 1.16M lockdownd 17 2.22M 664K 3.04M mediaserverd 18 296K 160K 440K mDNSResponder 20 540...

-Xmx differences between applet and standalone Java process

I have an applet which needs more or less memory depending on how many data the client has. We usually recommend to use Java 1.6 latest version, but we actually support Java 1.5+ so we have a protection in the applet which shows a dialog box with a warning about "not enough memory" and instructions where to go to increase the memory. H...

What is the best way to access memory in Java, similar to mmap?

I'm working on a Java application that needs to communicate with a C application. The C application uses shared memory and mmap to communicate, and I need the Java application to have access to the same memory. My first attempt involved using JNI calls to retrieve data from the shared memory, but the overhead of each JNI call killed per...