Play Sounds of Varying Varying Pitch Asynchronously and Concurrently

Question

My goal is to use Python to play sounds with the following requirements in a computer game context.

1. Take some input WAV file and randomly Vary the pitch to +/- 50% of original. Changing the sample rate seems to be an easy way to do this with PyDub.

2. Play the sound.

3. Be able to call this function rapidly so that long and short duration sounds overlap in actual playback.

I have spent over 24 work-hours searching for a way to meet all these requirements. I have done this before in Visual Basic and I was surprised at how difficult it is in Python.

Here is what I know so far:

1. PyGame.Mixer can play overlapping sounds concurrently, but it must play them all at the same sample rate. There doesn't appear to be a way to vary pitch.

2. PyDub can vary pitch by changing samplerate, but it can't play overlapping sounds with its basic playback. And, I have to write the output sound to file then immediately load it back, which feels wasteful.

3. WinSound can play PyDub's varying-samplerate sounds, but not with concurrent playback, not even with threading.

4. Playsound package does not work with python 3.6.

5. PyAudio can play PyDub's varying-samplerate sounds with concurrent playback if I use Threading, however, any more than a couple times and it causes horrible memory problems that quickly make Python crash.

My question: How can I achieve my 3 goals above without causing problems?

Here is the best result that I have so far (this is the PyAudio version which causes a crash if tested more than once or twice):

from pydub import AudioSegment
from random import random, seed
from time import sleep
import os
import threading
import pyaudio
import wave

def PlayAsyncWithRandPitch(WavPath):
    MyBaseFilename = os.path.basename(WavPath)
    sound = AudioSegment.from_file(WavPath, format="wav")
    seed()
    octaves = ((random()-0.50))
    print("random octave factor for this sound is: "+str(octaves))
    print("current sound frame rate:"+str(sound.frame_rate))
    new_sample_rate = int(sound.frame_rate * (2.0 ** octaves))
    print("new sound frame rate:"+str(new_sample_rate))
    newpitchsound = sound._spawn(sound.raw_data, overrides={'frame_rate': new_sample_rate})
    MyTotalNewPath = os.getcwd()+"\\Soundfiles\\Temp\\Mod_"+MyBaseFilename
    newpitchsound.export(MyTotalNewPath, format="wav")
    SoundThread = threading.Thread(target=PAPlay, args=(MyTotalNewPath,))
    SoundThread.start()
#=======================================================================================


#This function is just code for playing a sound in PyAudio
def PAPlay(filename):
    CHUNK = 1024
    wf = wave.open(filename, 'rb')
    p = pyaudio.PyAudio()
    stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
                    channels=wf.getnchannels(),
                    rate=wf.getframerate(),
                    output=True)
    data = wf.readframes(CHUNK)
    while data != '':
        stream.write(data)
        data = wf.readframes(CHUNK)
    stream.stop_stream()
    stream.close()
    p.terminate()
    return


if __name__ == "__main__":
    #Example sounds to test if more than one can play at once
    PlayAsyncWithRandPitch(os.getcwd()+'\\Soundfiles\\RifleMiss.WAV')
    sleep(0.2)
    PlayAsyncWithRandPitch(os.getcwd()+'\\Soundfiles\\splash.wav')
    sleep(0.2)
    PlayAsyncWithRandPitch(os.getcwd()+'\\Soundfiles\\sparkhit1.WAV')
    sleep(5.0)

Thank you in advance for your kind help!

Lavonia

preprocess your audio clips into the various sample rate permutations so you render them directly during run time – Omalley

I don't know how to do that. But thank you for the reply, I will consider it :) – Lavonia

Answer 1

Thanks to another hour of googling, I was able to solve it by finding an obscure note about PyDub. There is a way to actually change the samplerate, but "not actually" change the sample rate. It's called the chipmunk method.

https://github.com/jiaaro/pydub/issues/157#issuecomment-252366466

I really don't pretend to understand the nuance here, but it seems the concept is "take a sound, set the samplerate to some modified value, then convert the sample rate back to the traditional 44,100 HZ value."

They give this example which works very well:

from pydub import AudioSegment
sound = AudioSegment.from_file('./test/data/test1.mp3')
# shift the pitch up by half an octave (speed will increase proportionally)
octaves = 0.5
new_sample_rate = int(sound.frame_rate * (2.0 ** octaves))
# keep the same samples but tell the computer they ought to be played at the 
# new, higher sample rate. This file sounds like a chipmunk but has a weird sample rate.
chipmunk_sound = sound._spawn(sound.raw_data, overrides={'frame_rate': new_sample_rate})
# now we just convert it to a common sample rate (44.1k - standard audio CD) to 
# make sure it works in regular audio players. Other than potentially losing audio quality (if
# you set it too low - 44.1k is plenty) this should now noticeable change how the audio sounds.
chipmunk_ready_to_export = chipmunk_sound.set_frame_rate(44100)

It doesn't make much sense to me, but it does work :) Hope this helps someone out there.

Lavonia

Answer 2

This method seems kind of dubious. I explained how I do varispeed using Java to a C++ person on the following link.

The main idea is to use linear interpolation to get values from between the samples, and to progress through the sample data at a rate other than 1 by 1. If you are going 150% and need sample 0, then sample 1.5 (half way between 1 and 2), interpolate the value.

Pneumatic

Thanks for the reply! The truth is I have no idea how to use Java or c++. The method I have below works in Python with good speed so I'll use that until I hear better. – Lavonia

The gist of the algorithm there is not code dependent. – Pneumatic