How to decode AAC compressed frames to PCM using AudioConverterFillComplexBuffer iOS

Question

I want to implement SIP calls in my application, and first problem, that I need to solve, is converting audio from compressed AAC format with ADTS header to linear PCM.

My input data is an NSArray of ADTS frames with different framesize. Each frame is typeof NSMutableData. Each frame is of the same format and sample rate, only difference is framesize.

I tried to implement sample code, suggested by Igor Rotaru for this issue, but can't make it work.

Now my code looks like this. First of all, I configure the AudioConverter:

- (void)configureAudioConverter {
    AudioStreamBasicDescription inFormat;
    memset(&inFormat, 0, sizeof(inFormat));
    inputFormat.mBitsPerChannel = 0;
    inputFormat.mBytesPerFrame = 0;
    inputFormat.mBytesPerPacket = 0;
    inputFormat.mChannelsPerFrame = 1;
    inputFormat.mFormatFlags = kMPEG4Object_AAC_LC;
    inputFormat.mFormatID = kAudioFormatMPEG4AAC;
    inputFormat.mFramesPerPacket = 1024;
    inputFormat.mReserved = 0;
    inputFormat.mSampleRate = 22050;

    AudioStreamBasicDescription outputFormat;
    memset(&outputFormat, 0, sizeof(outputFormat));
    outputFormat.mSampleRate       = inputFormat.mSampleRate;
    outputFormat.mFormatID         = kAudioFormatLinearPCM;
    outputFormat.mFormatFlags      = kLinearPCMFormatFlagIsSignedInteger;
    outputFormat.mBytesPerPacket   = 2;
    outputFormat.mFramesPerPacket  = 1;
    outputFormat.mBytesPerFrame    = 2;
    outputFormat.mChannelsPerFrame = 1;
    outputFormat.mBitsPerChannel   = 16;
    outputFormat.mReserved         = 0;

    AudioClassDescription *description = [self
                                      getAudioClassDescriptionWithType:kAudioFormatMPEG4AAC
                                      fromManufacturer:kAppleSoftwareAudioCodecManufacturer];

    OSStatus status =  AudioConverterNewSpecific(&inputFormat, &outputFormat, 1, description, &_audioConverter);

    if (status != 0) {
        printf("setup converter error, status: %i\n", (int)status);
    }
}

After that I wrote the callback function:

struct MyUserData {
    UInt32 mChannels;
    UInt32 mDataSize;
    const void* mData;
    AudioStreamPacketDescription mPacket;
};

OSStatus inInputDataProc(AudioConverterRef inAudioConverter,
                         UInt32 *ioNumberDataPackets,
                         AudioBufferList *ioData,
                         AudioStreamPacketDescription **outDataPacketDescription,
                         void *inUserData)
{
    struct MyUserData* userData = (struct MyUserData*)(inUserData);

    if (!userData->mDataSize) {
        *ioNumberDataPackets = 0;
        return kNoMoreDataError;
    }

    if (outDataPacketDescription) {
        userData->mPacket.mStartOffset = 0;
        userData->mPacket.mVariableFramesInPacket = 0;
        userData->mPacket.mDataByteSize = userData->mDataSize;
        *outDataPacketDescription = &userData->mPacket;
    }

    ioData->mBuffers[0].mNumberChannels = userData->mChannels;
    ioData->mBuffers[0].mDataByteSize = userData->mDataSize;
    ioData->mBuffers[0].mData = (void *)userData->mData;

    // No more data to provide following this run.
    userData->mDataSize = 0;

    return noErr;
}

And my function for decoding frames looks like this:

- (void)startDecodingAudio {
    if (!_converterConfigured){
        return;
    }

    while (true){
        if ([self hasFramesToDecode]){
            struct MyUserData userData = {1, (UInt32)_decoderBuffer[_currPosInDecoderBuf].length, _decoderBuffer[_currPosInDecoderBuf].bytes};

            uint8_t *buffer = (uint8_t *)malloc(128 * sizeof(short int));
            AudioBufferList decBuffer;
            decBuffer.mNumberBuffers = 1;
            decBuffer.mBuffers[0].mNumberChannels = 1;
            decBuffer.mBuffers[0].mDataByteSize = 128 * sizeof(short int);
            decBuffer.mBuffers[0].mData = buffer;

            UInt32 numFrames = 128;

            AudioStreamPacketDescription outPacketDescription;
            memset(&outPacketDescription, 0, sizeof(AudioStreamPacketDescription));
            outPacketDescription.mDataByteSize = 128;
            outPacketDescription.mStartOffset = 0;
            outPacketDescription.mVariableFramesInPacket = 0;

            OSStatus status = AudioConverterFillComplexBuffer(_audioConverter,
                                                              inInputDataProc,
                                                              &userData,
                                                              &numFrames,
                                                              &decBuffer,
                                                              &outPacketDescription);

            NSError *error = nil;

            if (status == kNoMoreDataError) {
                NSLog(@"%u bytes decoded", (unsigned int)decBuffer.mBuffers[0].mDataByteSize);
                [_decodedData appendData:[NSData dataWithBytes:decBuffer.mBuffers[0].mData length:decBuffer.mBuffers[0].mDataByteSize]];
                _currPosInDecoderBuf += 1;
            } else {
                error = [NSError errorWithDomain:NSOSStatusErrorDomain code:status userInfo:nil];
            }
        } else {
            break;
        }
    }
}

Each time, AudioConverterFillComplexBuffer returns status 1852797029 which is, according to Apple API, kAudioCodecIllegalOperationError. If somebody succeded in converting with such formats, please, share some examples, or advice.

Wagoner

Have you already solved your problem? – Bannerol

@VladislavRudskoy yes, see my answer below. – Wagoner

Answer 1

Finally, I decoded my bytes with StreamingKit library (original reposiory can be found here).

Wagoner

Hey @Wagoner have you tried to convert AAC files? I'm getting AAC chunks in byte arrays from a realtime socket and can't figure out if this lib will have any use to me – Aurelea

@OzShabat I wrote custom datasource for using this library with socket streams. Let me see, if I still have this code locally, I will update my answer above. If not, I am afraid I've forgotten this case already. – Wagoner

Hey again, @OzShabat, I found my fork of library at my github. See the class STKInputStreamDataSource. Will be very glad, if it'll help. – Wagoner

I'm working on your fork for more then a day now. Very clever solution! Unfortunately, the audio packets I'm getting aren't configured to support AAC with adts so I don't think this library will help me – Aurelea

I still don't know how you get pcm data from StreamingKit? I can play aac files using StreamingKit, but I can't find pcm data – Grimaud