Delta Compression in .NET

Question

Has anyone been able to perform compression in a .NET environment to generate deltas between files. I'd like to use this functionality if at all possible, perhaps by using the functionality in msdelta.dll. I'd also be interested in how to generate deltas using other libraries (open source preferably).

Polymeric

xdelta is the open source one I've seen used most. Not sure how easy it is to call from C#, though. – Vulgarian

Thanks. I've checked out your link and found another link (pobox.com/~skeet/csharp/miscutil) which does have c# decoder version. However, I will need an encoder also. – Polymeric

Answer 1

I hope this isn't too much of a shameless plug, but I've written a wrapper library around both PatchAPI and MSDelta for my own purposes.

The library is dual-licensed under the MS-PL and DBAD-PL and available on GitHub.

I'm entertaining the notion of publishing the project on NuGet, but for the moment you can download the source and both create and apply deltas.

Creating a delta should be self-explanatory:

var compression = new MsDeltaCompression(); /* or PatchApiCompression(); */
compression.CreateDelta(sourcePath, destinationPath, deltaPath);

And equally self-explanatory (hopefully) is applying a delta:

var compression = new MsDeltaCompression(); /* or PatchApiCompression(); */
compression.ApplyDelta(deltaPath, sourcePath, destinationPath);

Tested on x86, but the P/Invoke signatures should be equally valid for x64 and ia64.

If you've haven't decided on whether you're using PatchAPI or MSDelta, my project's README.md tries to suggest (briefly) which one you should use, but otherwise the documentation for Microsoft's Delta Compression has this to say about MSDelta vs. PatchAPI:

MSDelta ... can create much smaller compressed files than those produced by other methods. Shipping with Windows Vista, it is the next generation of the technology previously released as PatchAPI (which will continue to be supported).

Emphasis mine.

Tendinous

Great library ta.speot.is. I was curious if there's any possibility you could expand on it to make signature files, that way if I'm working over the network I don't have to transfer the whole file over the network to get the delta, and instead just transfer the delta. – Berryberryhill

@Jeremy Thanks. Are you asking about some sort of delta patch server, where you send it a hash of the current version of your file and it sends you back a patch to bring it up-to-date with the latest one? – Tendinous

yes that is correct. I know MS has Remote Differential Compression (RDC), but MSDelta seems easier to use. I hear that RDC also relies on it's RDC service to be running in order to use it, something I don't want to rely on. – Berryberryhill

@Jeremy Well the purpose of this library was to provide a part of something similar to that. Although I never went on to make the server part of it, I don't think it's harder than using this library in conjunction with <your favourite web service framework here>. If you were looking at functionality similar to RDC I must admit I've never seen RDC before and never intended for this to complement or supplant it. – Tendinous

Hello. I am new in .Net. Can u explain me where can i find msdelta.dll OR mspatcha.dll. I want to use those dll inside my app. But couldn't find. I intsalled Microsoft SDK 7.1 but there have only mspatchc.dll not mspatcha.dll. And there have msdelta.lib not dll. – Matzo

@Matzo They're part of Windows. You shouldn't need to "find" them or to distribute them. – Tendinous

Answer 2

Fossil SCM has a delta compression algorithm implemented in C, and I've made a port of it on C# here: https://github.com/endel/FossilDelta

To create the delta, you must provide a byte[] of the origin and target. It is returned as byte[], which you can apply later.

byte[] origin = System.IO.File.ReadAllBytes ("old-file");
byte[] target = System.IO.File.ReadAllBytes ("new-file");
byte[] delta = Fossil.Delta.Create(origin, target);

Having the delta, you can apply the changes in the original file like this:

byte[] applied = Fossil.Delta.Apply(origin, delta);

I think it worth mentioning that the author of this algorithm is the same author of SQLite - so it has some credibility.

Blastocyst