Change the pitch (and speed) of audio during playback in Python

Question

I'm working on a Python program that plays music. One feature will be a slider that the user can drag up or down to change the pitch of the music as it plays.

For example, if the pitch is set to 2, then the music will sound one octave higher, it will play twice as fast, and it will last half as long. All I'm really changing is the playback speed, but I need to do so interactively in real-time.

A good example of this functionality implemented in flash can be found here. (It takes a little bit to load, be patient.)

I've looked into many python audio packages, but I haven't found one that can change the pitch of a sound that is currently playing. I have multiple versions of Python, so there is no requirement for what version the package supports. I'm developing this on Windows 7.

Any suggestions?

Answer 1

It sounds as though you want to resample the audio on-the-fly.

Perhaps you could try using the scikits.samplerate module. It uses the Secret Rabbit Code library.

Answer 2

You might want to look at using wxPython to create a media player, and investigate the SetPlaybackRate() function. wxWidget docs here.

That SetPlaybackRate() function is not supported on all platforms, and I've not tried it myself to see whether it does exactly what you want, and how well it works or not.

Answer 3

With Craig McQueen's help, I have created a proof-of-concept program.

This program plays a mono wav file called "music.wav" (located in the same folder as the program) and displays a short and wide window. The pitch of the music changes when you click and drag in the window. The left side of the window is two octaves lower, and the right side is two octaves higher.

There is some strange behavior here that I'm not sure how to fix. If the pitch is currently low, then there's about a 2 second delay before the pitch changes. However, the pitch changes in real-time for high pitches. (The delay increases smoothly as the pitch gets lower). I only add more sound to the buffer if soundOutput.getLeft() < 0.2. That is to say, if the amount of sound left on the buffer is less than 0.2 seconds. Therefore there should be no delay. For troubleshooting, I included code that writes soundOutput.getLeft() to a file. It tends to stay at or very near 0 all the time.

Decreasing the frames read to waveRead.readframes(100) decreases the delay, but also makes the sound choppy. Increasing the frames read significantly increases the delay.

import os, sys, wave, pygame, numpy, pymedia.audio.sound, scikits.samplerate

class Window:
    def __init__(self, width, height, minOctave, maxOctave):
        """
        width, height: the width and height of the screen.
        minOctave, maxOctave: the highest and lowest pitch changes. 0 is no change.
        """
        self.minOctave = minOctave
        self.maxOctave = maxOctave
        self.width = width
        self.mouseDown = False
        self.ratio = 1.0 # The resampling ratio
        waveRead = wave.open(os.path.join(sys.path[0], "music.wav"), 'rb')
        sampleRate = waveRead.getframerate()
        channels = waveRead.getnchannels()
        soundFormat = pymedia.audio.sound.AFMT_S16_LE
        soundOutput = pymedia.audio.sound.Output(sampleRate, channels, soundFormat)
        pygame.init()
        screen = pygame.display.set_mode((width, height), 0)
        screen.fill((255, 255, 255))
        pygame.display.flip()
        fout = open(os.path.join(sys.path[0], "musicdata.txt"), 'w') # For troubleshooting
        byteString = waveRead.readframes(1000) # Read at most 1000 samples from the file.
        while len(byteString) != 0:
            self.handleEvent(pygame.event.poll()) # This does not wait for an event.
            fout.write(str(soundOutput.getLeft()) + "\n") # For troubleshooting
            if soundOutput.getLeft() < 0.2: # If there is less than 0.2 seconds left in the sound buffer.
                array = numpy.fromstring(byteString, dtype=numpy.int16)
                byteString = scikits.samplerate.resample(array, self.ratio, "sinc_fastest").astype(numpy.int16).tostring()
                soundOutput.play(byteString)
                byteString = waveRead.readframes(500) # Read at most 500 samples from the file.
        waveRead.close()
        return

    def handleEvent(self, event):
        if event.type == pygame.QUIT or (event.type == pygame.KEYUP and event.key == pygame.K_ESCAPE):
            sys.exit()
        if event.type == pygame.MOUSEBUTTONDOWN and event.button == 1:
            self.mouseDown = True
            self.setRatio(event.pos)
        if event.type == pygame.MOUSEBUTTONUP and event.button == 1:
            self.mouseDown = False
        if event.type == pygame.MOUSEMOTION and self.mouseDown:
            self.setRatio(event.pos)
        return None

    def setRatio(self, point):
        self.ratio = 2 ** -(self.minOctave + point[0] * (self.maxOctave - self.minOctave) / float(self.width))
        print(self.ratio)

def main():
    Window(768, 100, -2.0, 2.0)

if __name__ == '__main__':
    main()

It's a pain to try to get all the packages I use to work well together. I'm using Python 2.6.6, PyGame 1.9.1 for python 2.6, NumPy 1.3.0 for python 2.6, PyMedia 1.3.7.3 for python 2.6, and scikits.samplerate 0.3.1 for python 2.6. Note that scikits.samplerate conflicts with NumPy 1.4 or greater, and one of the packages (I forget which one) requires setuptools