memory

Python: quickly loading 7 GB of text files into unicode strings

I have a large directory of text files--approximately 7 GB. I need to load them quickly into Python unicode strings in iPython. I have 15 GB of memory total. (I'm using EC2, so I can buy more memory if absolutely necessary.) Simply reading the files will be too slow for my purposes. I have tried copying the files to a ramdisk and th...

Simple Memory Management Question In Objective-C

There's a memory leak in my program and because I'm not well versed in C (created a function in my Objective-C class to load strings from a SQLite database). Can anyone spot my blunder? Thank you: static int MyCallback(void *context, int lCount, char **values, char **columns){ NSMutableArray *sqlRawStrings = (NSMutableArray *)context; ...

What happens if you specify max heap size greater than available RAM

Asked in an interview. What happens if you specify max heap size (Xmx) greater than available RAM? I also wonder what happens if you specify min heap size (Xms) greater than available RAM? ...

Objc memory crash with autorelase

I have been hunting all over my code and can't find the source of this crash: I am trying to decode an object with an NSKeyedUnarchiver and it crashes on it every time and says: *** __NSAutoreleaseFreedObject(): release of previously deallocated object (0x1008ad200) ignored *** __NSAutoreleaseFreedObject(): release of previously deallo...

iPhone memory leak because of stringWithUTF8String

Hello all, i have the following code which shows memory leak for object favorite near the statement with stringWithUTF8String. i have declared favorites in the property -(NSMutableArray *) readFavoritesFromDatabase { // Check if database is present [self setDatabaseNameAndPath]; [self checkAndCreateDatabase]; // Setup the databas...

Tips for saving memory when coding under android ?

Hi all, Im currently developing a software under android and im getting quite quickly some OutOfMemoryException.... I did modified some part of my code to use more static variables instead of making new allocation with the "new" operator but is there any things else to do ? or any other tips ? Any advices would be welcome. Thanks. ...

Memory Optimization for child processes

I work on Linux for ARM processor for cable modem. There is a tool that I have written (as the job demands) that sends/storms customized UDP packets using raw sockets. I form the packet from scratch so that we have the flexibility to play with different options. This tool is mainly for stress testing routers. The details are here. I ac...

Windows protected mode - memory

Hello. I downloaded a disk and memory editor called HxD (available at http://mh-nexus.de/en/hxd/). I wonder how it is able to access (read and modify) virtual memory assigned to all applications running on my system (Windows XP Pro SP3). From what I know, Windows is running in protected mode, making such endeavours impossible. Yet it's n...

Why does CPU access memory on a word boundary?

Hi experts~ I heard a lot that data should be properly aligned in memory for better access efficiency. CPU access memory on a word boundary. So in the following scenario, the CPU has to make 2 memory accesses to get a single word. Supposing: 1 word = 4 bytes ("|" stands for word boundary. "o" stands for byte boundary) |----o----o...

Matlab: Loading files

Hey, If I use the load function by matlab I normally end up doing something like this: temp = load('filename.mat'); realData = temp.VarName; clear temp or realData = load('filename.mat'); realData = realData.VarName; is any of this methods superiour to the other, especially in terms of memory usage? Or is there a more direct app...

What is the byte alignment of the elements in a std::vector<char> ?

I'm hoping that the elements are 1 byte aligned and similarly that a std::vector<int> is 4 byte aligned ( or whatever size int happens to be on a particular platform ). Does anyone know how standard library containers get aligned? ...

Is it possible to use short ints (16-bit) in PHP?

Background: I have a large 2D array of integers that I need to load into memory in PHP for each Apache request. I want it to take up less memory. PHP stores ints in PHP_INT_SIZE bytes, which is 32 bits on most systems. All the integers are less than 2^16, which means they could be a short int (eg in C). Am I right in thinking that stori...

iPad crash and probably related memory problems

Hello folks, I am porting PC game to iPad which has ton of graphic assets (over 250MB) and I am about half-way through. I didn't have iPad until now so I tested it only on simulator and everything was fine. But when I run it on device for first time, it crashed. All I got from console is Program received signal: “0”. Data Formatters...

Boost memory map specifying readahead.

Is there an option in boost memory maps that forces it to start bringing the entire file into memory without pagefaulting first? Is that operating system dependent or is will it be mostly portable? I need to bring entire files into memory and while I don't care about the actual contents I am concerned by the time it takes. I kind of don'...

how does this address (0x00000400) = 1024

Im working in C++ and I have a #define VAL 0x00000400. when I set a variable equal to the define: int value = VAL; when I run through the debugger is shows the variable value = 1024. could someone explain how that turns into 1024? Maybe some links to memory address info, #define info, or something relevant. Thanks ...

Can't modify memory using 32bit assembler

Hello. I am using NASM to assemble my assembler code. The code I assembled looks like this: [BITS 32] [ORG 0] jmp 07c0h:start testvar db 0, 0, 0, 0, 0, 0, 4, 8, 15, 16, 23, 42 start: mov byte [testvar], 47 hang: jmp hang times 510-($-$$) db 0 dw 0AA55h I had problems with another piece of code, I noticed that I cou...

Use of static finals in Android Activity.

Why is the use of static final variables encouraged to declare constants over just final variables? The use of static sounds logical when there will be many instances of a class but is this argument correct when used for a Android activity. In fact, since the Class instance will be around even after the activity finishes and is eventuall...

Script/Tool that gives memory locations accessed ?

I have an application and I want to make a trace of all the data memory locations that it accesses. Is there any script or tool that can help me retrieve the memory locations accessed by my application ? ...

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes)

Possible Duplicate: Allowed memory size of 33554432 bytes exhausted (tried to allocate 43148176 bytes) in php Hi, In my php page, I got the error as follows, Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 71 bytes) I tried to set memory limit using ini_set('memory_limit', '128M'); But st...

Getter and setter and property retain in objective-c

Hi all, I have a class wich is initialized like this. // myclass.h @property(nonatomic,retain) NSMutableArray *daysOfWeek; // this is in .h file // myclass.m @synthesize daysOfWeek; -(id)init { if(self=[super init]) { // initialize days of week self.daysOfWeek = [NSMutableArray arra...