How to get uncompressed byte array from DirectShow?

Question

I am trying to use DirectShow https://learn.microsoft.com/en-us/windows/win32/directshow/directshow in order to get uncompressed byte array from .mp3 stream. I have an implementation that can playback .mp3 byte stream

bool coAudioPlayer::LoadImp(SoundDataType dataType, std::string const & filename, unsigned char const * pData, int64_t dataLen, bool bOnlyIfFilenameChanged)
{
...
    m_pMemReader = new CMemReader(m_pMemStream, m_pMediaType, &hr);

    m_pMemReader->AddRef();


    hr = CoCreateInstance(CLSID_FilterGraph,
        NULL,
        CLSCTX_INPROC_SERVER,
        IID_IGraphBuilder,
        (void **)&this->m_pigb);

    hr = m_pigb->AddFilter(m_pMemReader, NULL);
    if (FAILED(hr))
    {
        return false;
    }

    m_pigb->QueryInterface(IID_IMediaControl, (void **)&m_pimc);
    m_pigb->QueryInterface(IID_IMediaEventEx, (void **)&m_pimex);
    m_pigb->QueryInterface(IID_IBasicAudio, (void**)&m_piba);
    m_pigb->QueryInterface(IID_IMediaSeeking, (void**)&m_pims);

    /*  Render our output pin */
    hr = m_pigb->Render(m_pMemReader->GetPin(0));
    if (!SUCCEEDED(hr))
    {
        return false;
    }

    HRESULT hr = m_pimc->Run();

    return m_bReady;
}

But I need to extend this functional and add approach to get uncompressed byte array (sound frames). As far as I understand under the hood DirectShow decodes it, but I don't see any way to retrieve this decoded array.

Is there a way to do it?

Daviddavida

Write a custom DirectShow filter that does something with the frames and attach it to the memreader's output pin. The Microsoft documentation on this is excellent. You can start from the CBaseRenderer filter. – Encrata

@Encrata What do you think about this approach learn.microsoft.com/en-us/windows/win32/api/strmif/… ? Looks relevant, not? – Daviddavida

That interface is associated with an input pin belonging to a filter; A CBaseRenderer filter exposes one or more CBaseRendererInputPin pins that implement this interface. – Encrata

Answer 1

"uncompressed byte array" is quite a wrong definition of desired data. There is no such thing as media data in byte array format. MP3 audio would typically be decompressed by MP3 Decoder media object wrapper into DirectShow filter into audio of MEDIASUBTYPE_PCM format with certain properties (sampling rate, channel count, bits per sample). Specifically, selected bit depth (and this decoder does support multiple bit depth options!) defines directly the representation of audio data as byte array.

You don't need or want to access the data when you build a playback pipeline, such as in scenario with Render method you mention and do you don't have the data.

A typical way to access the content is to build a pipeline around Sample Grabber Filter. There is a huge amount of code and questions explaining the approach. The relevant keyword is SampleCB standing for ISampleGrabberCB::SampleCB method. For example: ffmpeg audio frame from directshow sampleCB imediasample.

Cruel