Using Audio Queue Services to play PCM data over a socket connection

Question

I'm writing a remote desktop client for the iPhone and I'm trying to implement audio redirection.
The client is connected to the server over a socket connection, and the server sends 32K chunks of PCM data at a time.

I'm trying to use AQS to play the data and it plays the first two seconds (1 buffer worth). However, since the next chunk of data hasn't come in over the socket yet, the next AudioQueueBuffer is empty. When the data comes in, I fill the next available buffer with the data and enqueue it with AudioQueueEnqueueBuffer. However, it never plays these buffers.

Does the queue stop playing if there are no buffers in the queue, even if you later add a buffer?

Here's the relevant part of the code:

void
wave_out_write(STREAM s, uint16 tick, uint8 index)
{

    if(items_in_queue == NUM_BUFFERS){
        return;
    }
    if(!playState.busy){
        OSStatus status;
        status = AudioQueueNewOutput(&playState.dataFormat, AudioOutputCallback, &playState, CFRunLoopGetCurrent(), NULL, 0, &playState.queue);

        if(status == 0){
            for(int i=0; i<NUM_BUFFERS; i++){
                AudioQueueAllocateBuffer(playState.queue, 40000, &playState.buffers[i]);

            }
            AudioQueueAddPropertyListener(playState.queue, kAudioQueueProperty_IsRunning, MyAudioQueuePropertyListenerProc, &playState);

            status = AudioQueueStart(playState.queue, NULL);
            if(status ==0){
                playState.busy = True;
            }
            else{
                return;
            }
        }
        else{
            return;
        }
    }
    playState.buffers[queue_hi]->mAudioDataByteSize = s->size;

    memcpy(playState.buffers[queue_hi]->mAudioData, s->data, s->size);

    AudioQueueEnqueueBuffer(playState.queue, playState.buffers[queue_hi], 0, 0);
    queue_hi++;
    queue_hi = queue_hi % NUM_BUFFERS;
    items_in_queue++;
}


void AudioOutputCallback(void* inUserData, AudioQueueRef outAQ, AudioQueueBufferRef outBuffer)
{
    PlayState *playState = (PlayState *)inUserData;
    items_in_queue--;
}

Thanks!

Answer 1

You may find the answer to this question useful: http://stackoverflow.com/questions/2564830/audioqueue-ate-my-buffer-first-15-milliseconds-of-it

Answer 2

Generally, when using circular audio buffers, you should prevent the buffers from underrunning. If you lack the necessary data (due to e.g. network congestion), try to pad your audio data with silence or pause the audio playback.

It could be that once your buffer chain underruns, you need to restart playback. I've never actually underrunned the AudioQueue buffer, but I remember from Win32 programming that this was the case, so please feel free to correct me if I'm wrong.

Answer 3

Using CoreAudio's audio queuing services is vastly simplified by ensuring that you always re-queue every buffer as soon as it's finished playback by doing so in the callback that is fired when playback is finished. The audio data should sit in a separate circular buffer as it is received from the network so that the network and audio code aren't directly coupled.

To ensure that you don't drop audio, queue up a fixed number of buffers with data; this acts as a jitter buffer. Do not start playback until all the buffers are queued. As soon as each buffer is finished playback, re-queue it immediately with the next packet of data. If no data is available, just queue a buffer of silence; since the arriving audio packets will eventually catch up, this just has the effect of reducing dropped audio at the expense of extra latency.