How can I use AVAudioPlayer to play audio faster *and* higher pitched?

Question

Statement of Problem:

I have a collection of sound effects in my app stored as.m4a files (AAC format, 48 KHz, 16-bit) that I want to play at a variety of speeds and pitches, without having to pre-generate all the variants as separate files.

Although the .rate property of an AVAudioPlayer object can alter playback speed, it always maintains the original pitch, which is not what I want. Instead, I simply want to play the sound sample faster or slower and have the pitch go up or down to match — just like speeding up or slowing down an old-fashioned reel-to-reel tape recorder. In other words, I need some way to essentially alter the audio sample rate by amounts like +2 semitones (12% faster), –5 semitones (33% slower), +12 semitones (2x faster), etc.

Question:

Is there some way fetch the Linear PCM audio data from an AVAudioPlayer object, apply sample rate conversion using a different iOS framework, and stuff the resulting audio data into a new AVAudioPlayer object, which can then be played normally?

Possible avenues:

I was reading up on AudioConverterConvertComplexBuffer. In particular kAudioConverterSampleRateConverterComplexity_Mastering, and kAudioConverterQuality_Max, and AudioConverterFillComplexBuffer() caught my eye. So it looks possible with this audio conversion framework. Is this an avenue I should explore further?

Requirements:

1. I actually don't need playback to begin instantly. If sample rate conversion incurs a slight delay, that's fine. All of my samples are 4 seconds or less, so I would imagine that any on-the-fly resampling would occur quickly, on the order of 1/10 second or less. (More than 1/2 would be too much, though.)

2. I'd really rather not get into heavyweight stuff like OpenAL or Core Audio if there is a simpler way to do this using a conversion framework provided by iOS. However, if there is a simple solution to this problem using OpenAL or Core Audio, I'd be happy to consider that. By "simple" I mean something that can be implemented in 50–100 lines of code and doesn't require starting up additional threads to feed data to the a sound device. I'd rather just have everything taken care of automatically — which is why I'm willing to convert the audio clip prior to playing.

3. I want to avoid any third-party libraries here, because this isn't rocket science and I know it must be possible with native iOS frameworks somehow.

4. Again, I need to adjust the pitch and playback rate together, not separately. So if playback is slowed down 2x, a human voice would become very deep and slow-spoken. And if playback is sped up 2–3x, a human voice would sound like a fast-talking chipmunk. In other words, I absolutely do not want to alter the pitch while keeping the audio duration the same, because that operation results in an undesirably "tinny" sound when bending the pitch upward more than a couple semitones. I just want to speed the whole thing up and have the pitch go up as a natural side-effect, just like old-fashioned tape recorders used to do.

5. Needs to work in iOS 6 and up, although iOS 5 support would be a nice bonus.

Kurman

Perhaps you could look into using AUSampler. – Outsert

This is a good question. Looking at AVAudioPlayer, there is a sample rate key in the settings, but it is read-only. If you are able to adjust the sample rate, you would be able to change the pitch. Here is an interesting post on the topic: forums.macrumors.com/showthread.php?t=636846 – Rascon

Answer 1

The forum link Jack Wu mentions has one suggestion, which involves overriding the AIFF header data directly. This may work, but you will need to have AIFF files since it relies on a specific range of the AIFF header to write into. This also needs to be done before you create the AVAudioPlayer, which means that you can't modify the pitch once it is running.

If you are willing to go to the AudioUnits route, a complete simple solution is probably ~200 lines (note that this assumes the code style that has one function take up to 7 lines with one parameter on each line). There is an Varispeed AudioUnit, which does exactly what you want by locking pitch to rate. You would basically need to look at the API, docs and some sample AudioUnit code to get familiar and then:

1. create/init the audio graph and stream format (~100 lines)
2. create and add to the graph a RemoteIO AudioUnit (kAudioUnitSubType_RemoteIO) (this outputs to the speaker)
3. create and add a varispeed unit, and connect the output of the varispeed unit (kAudioUnitSubType_Varispeed) to the input of the RemoteIO Unit
4. create and add to the graph a AudioFilePlayer (kAudioUnitSubType_AudioFilePlayer) unit to read the file and connect it to the varispeed unit
5. start the graph to begin playback
6. when you want to change the pitch, do it via AudioUnitSetParameter, and the pitch and playback rate change will take effect while playing

Note that there is a TimePitch audio unit which allows independent control of pitch and rate, as well.

Malmsey

This sounds like the best bet so far. I'll try this out hopefully soon. I'm awarding the bounty for this answer since it sounds like a great lead, but not (yet) setting it as the accepted answer, due to uncertainties. If it ends up turning out to be what does the trick, I'll mark it as the accepted answer and post some follow-up comments. – Kurman

Answer 2

For iOS 7, you'd want to look at AVPlayerItem's time-pitch algorithm (audioTimePitchAlgorithm) called AVAudioTimePitchAlgorithmVarispeed. Unfortunately this feature is not available on early systems.

Fawcett

how would you use AVAudioTimePitchAlgorithmVarispeed? I'd love to know more as I'm trying to do a similar thing. – Prussianism

hi matt, don't mean to spam you but wondering if you could give this answer a second look. Could really use the help. Thanks. – Prussianism

@Auser What do you not understand about it? – Fawcett

thanks for responding. I don't understand how to set this property: #23879694 – Prussianism

@Prussianism Let's move over there then! I've put a comment asking you to rephrase: you should ask about AVAudioTimePitchAlgorithmVarispeed in particular and I'll show you how to learn where you can use it. – Fawcett