Real-time video encoding in DirectShow

Question

I have developed a Windows application that captures video from an external device using DirectShow. The image resolution is 640x480 and the videos saved without compression have very huge sizes (approx. 27MB per second).

My goal is to reduce this size as much as possible, so I am looking for an encoder which will allow me to compress the video in real-time. It could be H.264, MPEG-2 or anything else. It must allow me to save the video to disk and it would be best if I also could stream it in real-time over network (Wi-Fi, so the size should be around 1MB per second, or less). The significant quality loss would be unacceptable.

I have found out that getting an appropriate DirectShow filter for this task is very difficult. It can be assumed that the client machine is reasonably modern (fast 2-core CPU) and can utilize CUDA/OpenCL. There are a few apps that allow to encode video using CUDA and offer good performance, however I have not found an appropriate DirectShow filter or an API which could be used to develop one. The NVIDIA nvcuvenc.dll seems to have private API so I am unable to use it directly. Any CPU-based encoders I have found are too slow for my requirements, but maybe I have missed some.

Could anybody recommend me a solution, i.e. an encoder (paid or free, that can be used in an closed-source app) that can achieve a good performance, regardless whether it is using CPU/CUDA/OpenCL or DirectCompute? Or maybe I should use some external hardware video encoder?

Best regards,

madbadger

Nippers

Don't know if you found a solution for this, but in a meanwhile I developed Transport Stream encoding for my product - maybe such solution would also be acceptable to you since you are talking about streaming over the WiFi – Striker

Answer 1

Since you're using Directshow, by far the easiest thing to do would be to use WMV9 inside an ASF container. This is easier because it's available on almost all Windows machines (very few install time dependencies), decently fast (you should have no issues using it on a reasonably modern machine) and the quality is reasonable. But considering your limit is 8 mbit/sec (1 MB/sec), quality isn't an issue for you. 2 mbit/sec, VGA-resolution WMV9 should look quite good.

It's not nearly as good as a decent implementation of H264, but from an implementation standpoint, you're going to save yourself a lot of time by going this route.

See this:

http://msdn.microsoft.com/en-us/library/dd375008%28v=VS.85%29.aspx

Rattish

It seems quite tricky to create a profile and configure the ASFWriter for the WMV9 codec. Haven't found any good guide to that either, just MicroSoft's less-than-helpful docs. – Turves

Answer 2

Which filters have you tried?

If you're only dealing with 640x480, then any reasonable-quality commercial software-based encoder should be fine as long as you choose a realistic bitrate. Hardware acceleration using Cuda or OpenCL shouldn't be required. H264 takes a bit more horse-power and would require more CPU cores, but Mpeg2 or any of the h263-era codecs (xvid, wmv9, divx, etc) should have no problems even on a modest CPU. Streaming it over the network at the same time takes a little more effort, but should still be possible.

It's not DirectShow-based, but VLC Media Player can do all this. It's based on the FFMpeg open-source project. Some versions of it are LGPL-licensed, so the library could incorporated into your project without many restrictions.

If you just want a set of DirectShow filters that will handle all this for you, I've had good results with MainConcept's products before. They're at the expensive end of the spectrum though.

Kukri

Thanks, I've registered for evaluation of MainConcept SDKs. I'll write about the results later. – Nippers

Just as expected, MainConcept's CUDA H.264 encoder is fast enough and well-documented. By "fast enough" I mean I can achieve real-time VGA 30fps encoding on an average modern machine with no visible quality loss and the resulting files are 50-100 times smaller than the original non-compressed ones. The only downside is the high price of the components. – Nippers

I've written commercial solutions using MainConcept's filters. I found that their H.264 mux filter leaked memory and their support is very lacking. Their encoder and decoder filters are great though. – Lamdin

Answer 3

You dont specify what filters you've tried, or what 'significant' quality loss means, so about the best I suspect we can do is suggest some encoders to try to see if they meet your requirements.

Two good ones are the Theora and WebM video encoder filters (you can get them from a single installer at xiph.org). They're both high quality encoders which can be tweaked to balance performance vs quality. WebM can use multiple processors when encoding, which might help in your situation. Both are also used w/ HTML5 video, so that might be an extra plus for you.

Blower

Answer 4

Forget about WMV encoding for realtime streaming. WMV works well for realtime low quality streams, but it doesn't do high quality encoding in realtime.

I suggest that you take a look at MainConcept's SDK. They do a series of DirectShow filters for encoding H.264. I've implemented realtime streaming and muxing of streams encoded in H.264 using MainConcept's codec and DirectShow filters, and it's great.

Hope this helps

Carmel

Answer 5

I am using Windows Media Encoder for real-time encoding, and it works well even in resolution 720x576. One such example of it's usage is in VideoPhill Recorder.

It is written in pure .NET with DirectShow.NET for capturing and windowsMedia.NET for encoding.

Using those two I am able to achieve real-time encoding with 24/7 stability.

And both libraries are free to use on Windows, so you won't have to pay any licenses except for OS.

Analysis

Answer 6

ffdshow tryouts leverage ffmpeg's x264 stuff, which is said to be pretty fast (I think so anyway). Also libjpeg-turbo might help, or choosing some other codec made for high throughput like camstudio's or what not.

update: ffmpeg can take directshow input now: http://ffmpeg.zeranoe.com/forum/viewtopic.php?f=3&t=27

Symphonious

Answer 7

Have you seen this yet?

http://www.streamcoders.com/products/rtpavsource.html http://www.streamcoders.com/products/rtpavrender.html

Adora

Answer 8

If you can stay at or below 1280x1024, Micorsofts MPEG-2 encoder (included in Vista and up) is quite good.

I haven't gotten it to work for 1080p content at all though. I suspect the encoder just aborts on that. Shame.

Turves

Answer 9

Here is one option : http://www.codeproject.com/Articles/421869/H-264-CUDA-Encoder-DirectShow-Filter-in-Csharp

It uses about 10% of my cpu (p4 3ghz) to encode a SD video to h264 in graph edit.

Lepper

Answer 10

See the CaptureDS C# sample that comes with AVBlocks. It shows how to build a video recorder with AVBlocks and DirectShow. DirectShow is used for video capture and AVBlocks is used for video encoding:

Greeneyed

Could you please post the core part of the code from the link? – Prothalamium