How to properly release an AVCaptureSession

Question

I'm using the AV Foundation classes to capture the live video stream from the camera and to process the video samples. This works nicely. However, I do have problems properly releasing the AV foundation instances (capture session, preview layer, input and output) once I'm done.

When I no longer need the session and all associated objects, I stop the capture session and release it. This works most of the time. However, sometimes the app crashes with a EXEC_BAD_ACCESS signal raised in second thread that was created by the dispatch queue (and where the video samples are processed). The crash is mainly due to my own class instance, which serves as the sample buffer delegate and is freed after I've stop the capture session.

The Apple documentation mentions the problem: Stopping the capture session is an asynchronous operation. That is: it doesn't happen immediately. In particular, the second thread continues to process video samples and access different instances like the capture session or the input and output devices.

So how do I properly release the AVCaptureSession and all related instances? Is there a notification that reliably tells me that the AVCaptureSession has finished?

Here's my code:

Declarations:

AVCaptureSession* session;
AVCaptureVideoPreviewLayer* previewLayer;
UIView* view;

Setup of instances:

AVCaptureDevice* camera = [AVCaptureDevice defaultDeviceWithMediaType: AVMediaTypeVideo];
session = [[AVCaptureSession alloc] init];

AVCaptureDeviceInput* input = [AVCaptureDeviceInput deviceInputWithDevice: camera error: &error];
[session addInput: input];
AVCaptureVideoDataOutput* output = [[[AVCaptureVideoDataOutput alloc] init] autorelease];
[session addOutput: output];

dispatch_queue_t queue = dispatch_queue_create("augm_reality", NULL);
[output setSampleBufferDelegate: self queue: queue];
dispatch_release(queue);

previewLayer = [[AVCaptureVideoPreviewLayer layerWithSession: session] retain];
previewLayer.frame = view.bounds;
[view.layer addSublayer: previewLayer];

[session startRunning];

Cleanup:

[previewLayer removeFromSuperlayer];
[previewLayer release];
[session stopRunning];
[session release];

Answer 1

After AVCaptureSession allocation you may use:

NSNotificationCenter *notify =
[NSNotificationCenter defaultCenter];
[notify addObserver: self
            selector: @selector(onVideoError:)
            name: AVCaptureSessionRuntimeErrorNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionDidStartRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionDidStopRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionWasInterruptedNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionInterruptionEndedNotification
            object: session];

These are calling back the relevant methods upon session.stopRunning, session.startRunning etc.

There you should also implement some undocumented cleanup block:

AVCaptureInput* input = [session.inputs objectAtIndex:0];
[session removeInput:input];
AVCaptureVideoDataOutput* output = (AVCaptureVideoDataOutput*)[session.outputs objectAtIndex:0];
[session removeOutput:output];  

What I found confusing though is that upon calling seesion.stopRunning, onVideoStop: is called synchronously! despite Apple's asynchronous assumption on the case.

Its working but please let me know in case you see any trick. I would prefer working with it asynchronously.

Thanks

Answer 2

Solved! Perhaps it is the sequence of acions on initializing the session. This one works for me:

NSError *error = nil;

if(session)
    [session release];

// Create the session
session = [[AVCaptureSession alloc] init];


// Configure the session to produce lower resolution video frames, if your 
// processing algorithm can cope. We'll specify medium quality for the
// chosen device.
session.sessionPreset = AVCaptureSessionPresetMedium;

// Find a suitable AVCaptureDevice
AVCaptureDevice *device = [AVCaptureDevice
                           defaultDeviceWithMediaType:AVMediaTypeVideo];

// Create a device input with the device and add it to the session.
AVCaptureDeviceInput *input = [AVCaptureDeviceInput deviceInputWithDevice:device 
                                                                    error:&error];
if (!input) {
    // Handling the error appropriately.
}
[session addInput:input];

// Create a VideoDataOutput and add it to the session
AVCaptureVideoDataOutput *output = [[[AVCaptureVideoDataOutput alloc] init] autorelease];
[session addOutput:output];


// Configure your output.
dispatch_queue_t queue = dispatch_queue_create("myQueue", NULL);
[output setSampleBufferDelegate:self queue:queue];
dispatch_release(queue);

// Specify the pixel format
output.videoSettings = 
[NSDictionary dictionaryWithObject:
 [NSNumber numberWithInt:kCVPixelFormatType_32BGRA] 
                            forKey:(id)kCVPixelBufferPixelFormatTypeKey];

// If you wish to cap the frame rate to a known value, such as 15 fps, set 
// minFrameDuration.
output.minFrameDuration = CMTimeMake(1, 15);

previewLayer = [AVCaptureVideoPreviewLayer layerWithSession:session];
[delegate layerArrived:previewLayer];

NSNotificationCenter *notify =
[NSNotificationCenter defaultCenter];
[notify addObserver: self
            selector: @selector(onVideoError:)
            name: AVCaptureSessionRuntimeErrorNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionDidStartRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionDidStopRunningNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStop:)
            name: AVCaptureSessionWasInterruptedNotification
            object: session];
[notify addObserver: self
            selector: @selector(onVideoStart:)
            name: AVCaptureSessionInterruptionEndedNotification
            object: session];

// Start the session running to start the flow of data
[session startRunning];

Btw this sequence seems to resolve the synchronous notifications problem :)

Answer 3

Here's the best solution I've found so far. The basic idea is to use the finalizer of the dispatch queue. When the dispatch queue quits, we can be sure that there won't be any more action in the second thread where the sample buffers are processed.

static void capture_cleanup(void* p)
{
    AugmReality* ar = (AugmReality)p;
    [ar release];  // releases capture session if dealloc is called
}

...

dispatch_queue_t queue = dispatch_queue_create("augm_reality", NULL);
dispatch_set_context(queue, self);
dispatch_set_finalizer_f(queue, capture_cleanup);
[output setSampleBufferDelegate: self queue: queue];
dispatch_release(queue);
[self retain];

...

Unfortunately, I now have to explicitly stop capturing. Otherwise releasing my instance won't free it because the second thread now increments and decrements the counter as well.

A further problem is that my class is now released from two different threads. Is this reliable or is it the next problem causing crashes?

Answer 4

I've posted a very similar question in the Apple Developer Forum and got an answer from an Apple employee. He says it's a known problem:

This is a problem with the AVCaptureSession / VideoDataOutput in iOS 4.0-4.1 that has been fixed and will appear in a future update. For the time being, you can work around it by waiting for a short period after stopping the AVCaptureSession, e.g. half a second, before disposing of the session and data output.

He/she proposes the following code:

dispatch_after(
    dispatch_time(0, 500000000),
    dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), // or main queue, or your own
    ^{
        // Do your work here.
        [session release];
        // etc.
    }
);

I still like the approach with the dispatch queue finalizer better because this code just guesses when the second thread might have finished.