CALayer memory usage on draw - implicit cache?

Question

Hello everyone.

I'm having a peculiar problem with CoreGraphics/CoreAnimation on the iPhone. To better explain how the problem manifests itself, I'll walk you through my current setup and illustrate with code where appropriate.

I'm trying to draw a bunch of preloaded images in a UIView's CALayer, but whenever the image displays, the app's memory usage spikes and the memory is not reclaimed whenever the image changes.

Preloading of the images is done by reading them through UIImage's facilities, rendering them onto a Bitmap Context and extracting a CGImageRef out of that context. The purpose of this is to decompress and scale the images, so that these operations do not occur on every draw. A similar recommendation can be found in an Apple Q&A on the matter (search for CGContextDrawImage performance if you're curious). The context is set up with 8 bits per component and premultiplied alpha.

After the images are decompressed into a bitmap, they're stored in a NSArray and are later assigned (not retained) to a custom UIView subclass that does the drawing. I've tried various approaches to actually drawing the images, and by far the fastest method is directly setting the view's CALayer contents property. Other methods such as drawLayer:inContext: and drawRect: have varying impacts on the framerate, but they all exhibit the same memory behavior.

The problem is.. after the contents property changes, I see a spike in memory in Instruments and that memory doesn't go down even after the image is no longer being displayed. Object Allocations stay constant, so my only guess is that CoreAnimation is creating some implicit cache to speed up drawing. As I said however, that cache is not released when it should, and the gradual build-ups lead to a crash after only a couple of minutes of running.

The contents property retains the object and I don't explicitly release it because I want the original images to stay in memory for the duration of the application's execution; on top of that, a high retain count wouldn't account for the memory spikes I see.

When examining the stack, I see that CoreAnimation makes calls to functions such as CA::Render::copy_image, which leads me to believe it's duplicating the layer's contents somewhere out of reach. I suppose there's a valid reason for this, but not knowing how and when to clear this is currently a show-stopping bug.

So can anyone with intricate knowledge of CA please explain to me if I'm doing something wrong and how I can work around this behaviour.

Thanks.

Answer 1

I'm doing exactly what you have described - without examining leaks. Perhaps it would help when you provide your source.

What I do examine however, is that CA allocates system memory which are not recorded in Instruments "Object Allocations", but in Instrument's "Activity Monitor" respectively in Instrument's "Memory Monitor" - namely "Physical Memory Free". The "Physical Memory Free" decreases when an image is drawn in an offscreen context and it also decreases when that image is displayed.

The memory will not be freed (including that used for displaying) until the image is deallocated (though, it would be great to know how this memory can be freed explicitly).

In your case, if you have an array of UIImages as a cache, the system memory used for this images should be freed when the array of images is deallocated and when there are no further references to the UIImage objects.

Unless you provide your source, I would suspect your retain-count of your CGImageRefs are not balanced - and thus, they are leaking. Note that CGImageRef should not be put in an NSArray directly - these do not behave like Objective-C objects (aka toll free bridging to UIImage) - you need to call CGImageRetain/CGImageRelease explicitly.

Answer 2

@CouchDeveloper - thanks for taking the time to respond.

I did some further digging around and I'll answer my own question instead of replying directly, maybe the insight I've gained will help others having similar issues.

Your observation about allocations vs. memory usage is correct. The memory I'm looking at is the "real" (resident) memory used by the application in the Memory Monitor (Activity Monitor) instrument.

I think I've discovered the cause for these mysterious memory allocations. When CoreAnimation encounters an image that is not in the native pixel format it will copy the image in a buffer of its own and that seems to be the usage I'm seeing. I believe the right format is 32bit, host byte order, premultiplied alpha first (bitmap info flags kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Host).

In order to find out what CGImageRef instances are being copied and get a broader look into the framework's inner workings, you can use the Core Animation instrument on-device, or specify a few environment variables and launch the iPhone Simulator from the command-line. A list of all the flags can be found here. In my case, the most useful combination is CA_COLOR_COPY and CA_LOG_IMAGE_COPIES, which will give copied images a cyan overlay and log copies to stderr.

I'm still having some weird issues ("no data pointer", non-premultiplied alpha, etc.) but I think it's mostly issues with the way I create or pass images around, but I think I'm on the fast track to resolving all memory issues.

Hope this can be helpful to others wondering where their memory and performance issues are coming from!