How to render and save captured Video/Audio into a custom file/filter format in DirectShow?

Question

Basiclly, I want to capture audio/video. Run it through a mp4 muxer and save it to a file on disk. Before I used ICaptureGraphBuilder2, but it seems unusuable when saving to custom formats.

My code so far,

I enumerate video/audio devices. In this sample I only try to capture audio. I get the correct device and use GetPin to enumerate the filters pins to get its output pin.

hr = pMoniker2->BindToObject(0, 0, IID_IBaseFilter, (void**)&pSrc2);
hr = pGraph->AddFilter(pSrc2, L"AudioCap");

hr = GetPin(pSrc2, PINDIR_OUTPUT, &outPin);

This is the custom filter, a MP4 muxer. It properly loads and I can get the input pin and connect it with my output pin. So far so good.

HRESULT hr = CreateObjectFromPath(TEXT("c:\\filters\\mp4mux.dll"), clsid, &pUnk);
if (SUCCEEDED(hr))
{
    IBaseFilterPtr pFilter = pUnk;
    HRESULT hr = pGraph->AddFilter(pFilter, L"Private Filter");
    hr = GetPin(pFilter, PINDIR_INPUT, &inPin);
}

hr = pGraph->Connect(outPin, inPin);

This is where I get lost, I can't find out how to take the next steps to Render and save the output to a file on disk. So any help with the next steps would be greatfully appreciated, thanks in advance!

EDIT: Filesink code

AM_MEDIA_TYPE mType;

mType.majortype = MEDIATYPE_Video;
mType.subtype = MEDIASUBTYPE_H264;
mType.bFixedSizeSamples = FALSE;
mType.bTemporalCompression = TRUE;
mType.lSampleSize = 0;
mType.formattype = FORMAT_None;
mType.pUnk = NULL;
mType.cbFormat = 0;
mType.pbFormat = NULL;
//Not 100% sure about the setup of the media format.

IBaseFilter * iFiltera = NULL; 
IFileSinkFilter* iFilter = NULL; 
IGraphBuilder *pGraph;

hr = pMoniker2->BindToObject(0, 0, IID_IBaseFilter, (void**)&pSrc2); //audio capture
hr = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER,  IID_IGraphBuilder, (void**)&pGraph);
hr = CoCreateInstance(CLSID_FileWriter, NULL, CLSCTX_INPROC_SERVER, IID_IBaseFilter, (void**)&iFiltera);

hr = pBuild->SetFiltergraph(pGraph);

hr = pGraph->AddFilter(pSrc2, L"AudioCap");
hr = GetPin(pSrc2, PINDIR_OUTPUT, &outPin); //ADDED

hr = pGraph->AddFilter(iFiltera, L"FileWriter");
hr = iFiltera->QueryInterface(IID_IFileSinkFilter, (void**)&iFilter);

iFilter->SetFileName((LPCOLESTR)"c:\\wav\\tester.mp4", NULL); //UPDATED mType set to NULL

HRESULT hr = CreateObjectFromPath(TEXT("c:\\filters\\mp4mux.dll"), clsid, &pUnk);

IBaseFilterPtr pFilter = pUnk;
if (SUCCEEDED(hr))
{

    HRESULT hr = pGraph->AddFilter(pFilter, L"Private Filter");
    hr = GetPin(pFilter, PINDIR_INPUT, &inPin); //mux in

    hr = GetPin(pFilter, PINDIR_OUTPUT, &mOutPin); //mux out
    hr = GetPin(iFiltera, PINDIR_INPUT, &filePin); // filewriter in
}

hr = pGraph->Connect(outPin, inPin); //connect audio out and mux in
hr = pGraph->Connect(mOutPin, filePin); //connect mux out and file in; Error 0x80040217(VFW_E_CANNOT_CONNECT?) //works now

//ADDED code
IMediaControl *pMC = NULL;
hr = pGraph->QueryInterface(IID_IMediaControl, (void **)&pMC);

hr = pMC->Run();
Sleep(4000);
hr = pMC->Stop();  

Ardellardella

Answer 1

You need to have an idea what filter graph topology you need for specific task. You are doing capture, here - fine. So you have audio capture filter you provided code snippet for. Then you either compress audio (where your preferred choice should be AAC AKA MPEG-4 Part 3, provided that you are to produce MP4 file), or keep audio uncompressed PCM. Then you connect MPEG-4 multiplexer as you do. The multiplexer produces output stream and you are expected to complete the pipeline with File Writer Filter.

You can manually build the chain in GraphEdit SDK application (or there are alternate richer tools). Your filter graph looks like this:

Note that you can expose filter graph in your application, and connect to it remotely and inspect the topology. This makes debugging much easier. Starting/stopping the filter graph (IMediaControl::Run, ::Stop from code) creates you the file.

My understanding is that you get lost immediately after adding multiplexer. Now you need to find its output pin, add File Writer, query its IFileSinkFilter, set the destination file name using it, find its input pin, connect the two unconnected pins (mux output, writer input). Your pipeline is ready to run.

Bobsleigh

If im understanding this right. Connect microphone outpin to MP4 multiplexer inpin. If thats right then im on the right way. Then im stuck. So I'm supposed to create a filesink, find the input pin and connect it to the MP4 multiplexer output? Then everything is set and I just use mediacontrols on my filtergraph as before with start stop? If I'm right and if you dont mind me asking, do you have a suggestion for finding pins of a IFileSinkFilter? Also thanks a lot for that picture, it made everything a lot clearer i believe :) – Ardellardella

Temp.mp4 on my screenshot is File Writer which has both IBaseFilter and IFileSinkFilter. That is you add it connect its pin to the input, set file name and you are good to go. – Bobsleigh

Please forgive my inexperience with COM and Directshow. I'm not 100% sure about how to implement a Filewriter with both filters. I added some code to my post that does what I think you meant, however it fails at connecting the filewriter input with the mpeg4mux output. If you can take a look at it whenever you have time and point out what I'm doing wrong id be very thankful. – Ardellardella

Your code is about right. There is no need for media type in SetFileName call, and after all you provide video media type there while your question is about recording audio. Is there any hr error on the way? Consider using smart pointers CComPtr for better reference management and readability of code. – Bobsleigh

Ok, good and bad news. It seems you were right about the media type, if I set it to NULL everything seems to run fine even if i change capture source to video. But in both cases even though there are no errors there is no output file created. I put if(FAILED(hr)){printf("error");} after every call but no errors pops. Could it be the media type? Any ideas? I also updated the changes and added the last part of my code with the media control in case i messed that up somehow. Again, thanks for your persistance and patience. – Ardellardella

Hey, you have a string constant issue, your file path is invalid. (LPCOLESTR)"c:\\wav\\tester.mp4" - this is not going to work out, you should L"c:\\wav\\tester.mp4" instead. – Bobsleigh

Ah, correct. Cool! It creates output files now both for audio and video with data. But with mType as NULL, media players cant read the files. For example, the files are bigger than 0kb but the lenght is displayed as 0 seconds. If i change NULL to my current mType produces files but no data. So I guess I have to figure out how to correctly set mType? I feel bad for asking but incase you know how to set mType for audio/video it would save me a lot of time, tinkering with it. Thanks for all the help though! – Ardellardella

You basically built the capture graph correctly. I think the reason might be that Multiplexer filter in this scenario is not quite doing well with raw/uncompressed/PCM audio data. .MP4 format basically assumes that you put AAC audio on the input, not uncompressed. I remember I had to change something on Multiplexer source for it to start working properly. To compress into AAC however you need a third party compressor, so I have no quick advice for you what to do. The same applies to video part, while the mux takes raw video, it works correctly with MPEG-4 video formats (incl. H.264). – Bobsleigh

So my issue has nothing to do with mType? Because for example VLC displayed the error "no support for '(empty)' format" So i guessed it was because there was data but no file/data/header information in the file, describing the data. If that is true, you mean that for Audio I need an AAC muxer between the audio capture output and the MPEG4 input? The video part I didn't quite follow, what do I need to do to make the video work? – Ardellardella

MP4 format is a part of MPEG-4 series specifications (Parts 12, 14). And as such, it works best with MPEG-4 Video (Parts 2, 10 aka AVC, H.264) and Audio (Part 3 aka AAC). The mux produces best files, players exhibit less compatibility issues with these encodings. Right now you are putting non MPEG-4 data into MP4 and hence the problems. It is not about mType at all. AAC compressor would convert PCM audio into AAC audio and the MP4 files would be good. – Bobsleigh

Ok thanks for explaining that. I understand the audio part. So my last question is, can I solve the video part? Do I need an AVC muxer before the MPEG4 muxer to make the video capture work and when I say video capture I mean only video, no sound. – Ardellardella

On video part you capture from camera, right? So video again is uncompressed. You similarly need H.264 compressor/encoder to come up with H.264 data on the mux input. That's it, nothing else is necessary. MP4 file will be good on the output. – Bobsleigh

Thank you for clarifying that for me. I can't tell how grateful I am for your help over the past few days. I will look into those compressors and see if I can complete the project :). – Ardellardella