How can I mix a melody track over the top of my background track?

Question

For simplicity, let's say I have 3 tracks. One always plays, then occaisionally one of the others will also play:

[----------Background track----x------]
[------------"Good" track------x------]
[------------"Evil" track------x------]

So at time "x" you will be hearing background+(good/evil), depending on what the player is doing. The tracks are all the same length and will loop.

A naive approach is to have all the non-background tracks playing silently, then adjust their volume when you want to mix them in/out. I don't want to do this.

My current approach is to allow sounds to be played "relative to" a previous sound. So given the first sound is playing ("background"), the code to play a 2nd file relative to it would;

1. Use ov_time_tell or ov_pcm_tell to find the position in seconds/pcm samples of the first file
2. Seek to that point in the new OggVorbis_file using ov_time_seek or ov_pcm_seek
3. Load the seeked 2nd file into OpenAL buffers, attach those buffers to a new source, play

My problem is somewhere in this process a ~0.5s lag is introduced, so the 2nd file is out of synch with the first.