Windows C# implementation of linux dd command

Question

I'm writing a C#.Net app to run on windows that needs to take an image of a removable disk and chuck it onto a Linux Live USB. The Live USB is the inserted into the target machine and boots, on start up it runs a script which uses the dd command like so to flash it onto another drive:

dd if=/path/to/file/from/csharp/program of=/dev/sdX

The problem I am having is creating the image on the windows side. I have tried my Live Linux out with files I have created on a Linux system using dd and that works fine, but I need to be able to create these files from within a C#.Net application on Windows. I'd rather not have to rely on cygwin or some other dependency so tried to use the Win32 CreateFile function to open the physical device.

CreateFile is called with the first arg set to "\.\F:" (if F: is the drive I want to image), like so:

SafeFileHandle TheDevice = CreateFile(_DevicePath, (uint)FileAccess.Read, (uint)(FileShare.Write | FileShare.Read | FileShare.Delete), IntPtr.Zero, (uint)FileMode.Open, (uint)FILE_ATTRIBUTE_SYSTEM | FILE_FLAG_SEQUENTIAL_SCAN, IntPtr.Zero);
if (TheDevice.IsInvalid) 
{ 
    throw new IOException("Unable to access drive. Win32 Error Code " + Marshal.GetLastWin32Error()); 
}
FileStream Dest = System.IO.File.Open(_SaveFile, FileMode.Create);
FileStream Src = new FileStream(TheDevice, FileAccess.Read);
Src.CopyTo(Dest);
Dest.Flush();
Src.Close();
Dest.Close();

But when the output file is dd'd back onto a disk using the Live Linux USB the result is not as expected (the disk isn't bootable etc, but from examining the output file in a hex editor, it looks like there is an MBR at the beginning etc).

Is this a problem with endianess or should I using something other than a FileStream to copy the data into the file.

Alternatively is there an example of dd for Windows source code (C# or C++, i've looked at the Delphi for http://www.chrysocome.net/dd and don't totally understand it or have a decent Delphi IDE to pick the code apart) so I can see how that works?

UPDATE/EDIT:

Here is a hex string of the first 512 Bytes that the dd output contains:

33 C0 FA 8E D8 8E D0 BC 00 7C 89 E6 06 57 8E C0 FB FC BF 00 06 B9 00 01 F3 A5 EA 1F 06 
00 00 52 52 B4 41 BB AA 55 31 C9 30 F6 F9 CD 13 72 13 81 FB 55 AA 75 0D D1 E9 73 09 66 
C7 06 8D 06 B4 42 EB 15 5A B4 08 CD 13 83 E1 3F 51 0F B6 C6 40 F7 E1 52 50 66 31 C0 66 
99 E8 66 00 E8 21 01 4D 69 73 73 69 6E 67 20 6F 70 65 72 61 74 69 6E 67 20 73 79 73 74 
65 6D 2E 0D 0A 66 60 66 31 D2 BB 00 7C 66 52 66 50 06 53 6A 01 6A 10 89 E6 66 F7 36 F4 
7B C0 E4 06 88 E1 88 C5 92 F6 36 F8 7B 88 C6 08 E1 41 B8 01 02 8A 16 FA 7B CD 13 8D 64 
10 66 61 C3 E8 C4 FF BE BE 7D BF BE 07 B9 20 00 F3 A5 C3 66 60 89 E5 BB BE 07 B9 04 00 
31 C0 53 51 F6 07 80 74 03 40 89 DE 83 C3 10 E2 F3 48 74 5B 79 39 59 5B 8A 47 04 3C 0F 
74 06 24 7F 3C 05 75 22 66 8B 47 08 66 8B 56 14 66 01 D0 66 21 D2 75 03 66 89 C2 E8 AC 
FF 72 03 E8 B6 FF 66 8B 46 1C E8 A0 FF 83 C3 10 E2 CC 66 61 C3 E8 62 00 4D 75 6C 74 69 
70 6C 65 20 61 63 74 69 76 65 20 70 61 72 74 69 74 69 6F 6E 73 2E 0D 0A 66 8B 44 08 66 
03 46 1C 66 89 44 08 E8 30 FF 72 13 81 3E FE 7D 55 AA 0F 85 06 FF BC FA 7B 5A 5F 07 FA 
FF E4 E8 1E 00 4F 70 65 72 61 74 69 6E 67 20 73 79 73 74 65 6D 20 6C 6F 61 64 20 65 72 
72 6F 72 2E 0D 0A 5E AC B4 0E 8A 3E 62 04 B3 07 CD 10 3C 0A 75 F1 CD 18 F4 EB FD 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 19 16 9F 29 00 00 80 01 01 00 06 FE 3F 0E 3F 00 00 00 61 C8 03 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA

and here is what my code produces:

EB 76 90 4D 53 44 4F 53 35 2E 30 00 02 04 04 00 02 00 02 00 00 F8 F2 00 3F 00 FF 00 3F 
00 00 00 61 C8 03 00 80 00 29 7A E8 21 04 4E 4F 20 4E 41 4D 45 20 20 20 20 46 41 54 31 
36 20 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 
00 00 00 00 E9 05 01 B4 0E 53 33 DB CD 10 5B C3 8A 07 3C 00 74 06 E8 EE FF 43 EB F4 C3 
0D 4E 6F 20 42 50 42 3A 20 43 61 6E 27 74 20 62 6F 6F 74 20 75 73 69 6E 67 20 43 48 53 
20 66 75 6E 63 74 69 6F 6E 73 00 50 B0 2E E8 BC FF 58 33 DB 8E 06 E4 01 F6 06 DC 01 02 
75 42 F6 06 DC 01 04 75 07 80 3E E8 01 80 72 34 53 53 52 50 06 53 55 6A 10 8B F4 52 50 
8A 16 E8 01 B8 00 42 F9 CD 13 8A EC 58 5A 8D 64 10 72 14 80 FD 00 75 0F 03 C5 83 D2 00 
C3 BB 91 00 E8 78 FF F4 EB FD 83 3E 18 00 00 74 F0 52 50 8B CD F7 36 18 00 8B F2 03 D1 
3B 16 18 00 76 06 8B 0E 18 00 2B CE 33 D2 F7 36 1A 00 88 16 E9 01 8B F8 8B D7 51 8A C1 
8D 4C 01 C0 E6 06 0A CE 8A EA 8B 16 E8 01 B4 02 CD 13 59 73 15 80 FC 09 75 0A 49 EB DE 
8A C4 04 30 E8 18 FF B4 00 CD 13 EB D1 58 5A 03 C1 83 D2 00 2B E9 74 07 C1 E1 09 03 D9 
EB 94 C3 00 00 00 00 FA FC E8 00 00 5E 81 EE 85 01 2E 8B 84 E4 01 8E D8 8E C0 8E D0 2E 
C7 84 7C 01 AF 01 2E 89 84 7E 01 B9 00 01 BF 00 00 F3 2E A5 2E FF AC 7C FF BC 00 0A FB 
80 3E E8 01 FF 75 04 88 16 E8 01 83 06 E4 01 20 A1 E0 01 8B 16 E2 01 BD 02 00 E8 E9 FE 
50 52 EB 74 90 00 00 00 00 00 00 00 00 00 00 00 D3 20 00 00 00 30 80 00 FF 00 68 41 00 
40 09 FF 40 5A AC 04 00 00 AC 04 00 00 00 00 12 00 55 AA

This was taken from exactly the same CF card without any editing/writing etc happening, so i'm confused as to why they are so different, but both end with the correct 55 AA bytes too. Does Windows mangle the MBR's on cards when they're accessed this way or is some other weird under the hood stuff happening that I'm not aware of?

Caffey

I'm assuming using calling "dd" from cygwin from your C# code is out of the question? – Insinuation

It would be a last resort, the problem I think have is converting the //./F: path into something that dd under cygwin understands. So the problem (for me) then becomes how to within the c# application convert my //./F: path into /dev/sdX in cygwin. – Caffey

@Kragen I think what is happening here is that the windows version is getting the Volume Boot Record not the Master Boot Record. This is probably because I'm passing to it //./F: and F: is just the first partition, not the Physical Disk. Would be nice if someone else could confirm if I'm correct or not before I get into the next question about how to access the Physical Disk (instead of partition/volume). – Caffey

+1 for well-researched, clear question including sample input and output :) ♡ – Isoagglutinin

@Caffey That sounds spot on to me - looking at the two samples the first (produced by dd) looks like a MBR (the first 512 bytes of the disk), wheras the second looks like a boot record (the first 512 bytes of a partition). Both look like valid and completely different boot sectors - there is no way you could accidentally mangle one into the other from a programming mistake. – Trillby

Answer 1

I think what you have should work - I've tried this myself using a bootable floppy disk image (mounted as a virtual drive using ImDisk) and the resulting file is binary identical to the original image.

For completeness here is the code I used (in its entirity):

using System;
using System.IO;
using System.Runtime.InteropServices;
using Microsoft.Win32.SafeHandles;

namespace ConsoleApplication1
{
    public class Program
    {
        const int FILE_ATTRIBUTE_SYSTEM = 0x4;
        const int FILE_FLAG_SEQUENTIAL_SCAN = 0x8;

        [DllImport("Kernel32.dll", SetLastError = true, CharSet = CharSet.Auto)]
        public static extern SafeFileHandle CreateFile(string fileName, [MarshalAs(UnmanagedType.U4)] FileAccess fileAccess, [MarshalAs(UnmanagedType.U4)] FileShare fileShare, IntPtr securityAttributes, [MarshalAs(UnmanagedType.U4)] FileMode creationDisposition, int flags, IntPtr template);

        [STAThread]
        static void Main()
        {
            using (SafeFileHandle device = CreateFile(@"\\.\E:", FileAccess.Read, FileShare.Write | FileShare.Read | FileShare.Delete, IntPtr.Zero, FileMode.Open, FILE_ATTRIBUTE_SYSTEM | FILE_FLAG_SEQUENTIAL_SCAN, IntPtr.Zero))
            {
                if (device.IsInvalid)
                {
                    throw new IOException("Unable to access drive. Win32 Error Code " + Marshal.GetLastWin32Error());
                }
                using (FileStream dest = File.Open("TempFile.bin", FileMode.Create))
                {
                    using (FileStream src = new FileStream(device, FileAccess.Read))
                    {
                        src.CopyTo(dest);
                    }
                }
            }
        }
    }
}

If this doesn't work then it seems to indicate that:

1. There is a problem with the original image.
2. The problem is with whatever is using the disk image that you've just written.
3. There is some subtle differences in dealing with the specific device you are accessing (although I can't think what)

The most likely culprit is step 2. What exactly is it that you are doing with the resulting disk image?

---

Update: This is written in the comments, but for completeness I thought I'd add it to my answer - it looks like whats happening is that the contents of the first partition of the disk is being written, when instead what is wanted is the contents of the entire disk.

When you take a look at the second hex string (the one produced by sample code) in something like HxD we see this:

ëv.MSDOS5.0..........øò.?.ÿ.?...aÈ..€.)zè!.NO NAME    FAT16   ..
........................................................é..´.S3Û
Í.[Ê.<.t.èîÿCëôÃ.No BPB: Can't boot using CHS functions.P°.è¼ÿX
3ÛŽ.ä.ö.Ü..uBö.Ü..u.€>è.€r4SSRP.SUj.‹ôRPŠ.è.¸.BùÍ.ŠìXZ.d.r.€ý.u.
.ŃÒ.û‘.èxÿôëýƒ>...tðRP‹Í÷6..‹ò.Ñ;...v.‹...+Î3Ò÷6..ˆ.é.‹ø‹×QŠÁ.
L.Àæ..Ίê‹.è.´.Í.Ys.€ü.u.IëÞŠÄ.0è.ÿ´.Í.ëÑXZ.ÁƒÒ.+ét.Áá..Ùë”Ã....
úüè..^.î…..‹„ä.ŽØŽÀŽÐ.Ç„|.¯..‰„~.¹..¿..ó.¥.ÿ¬|ÿ¼..û€>è.ÿu.ˆ.è.ƒ.
ä. ¡à.‹.â.½..èéþPRët............Ó ...0€.ÿ.hA.@.ÿ@Z¬...¬.......Uª

This looks to me like the boot sector of a FAT16 partition - the presence of the strings "MSDOS5.0", "NO NAME" and "FAT16" near the start is a dead giveaway.

Compare this to the output of the first hex string (the one produced by dd):

3ÀúŽØŽÐ¼.|‰æ.WŽÀûü¿..¹..ó¥ê....RR´A»ªU1É0öùÍ.r..ûUªu.Ñés.fÇ...´B
ë.Z´.Í.ƒá?Q.¶Æ@÷áRPf1Àf™èf.è!.Missing operating system...f`f1Ò».
|fRfP.Sj.j.‰æf÷6ô{Àä.ˆáˆÅ’ö6ø{ˆÆ.áA¸..Š.ú{Í..d.faÃèÄÿ¾¾}¿¾.¹ .ó¥
Ãf`‰å»¾.¹..1ÀSQö.€t.@‰ÞƒÃ.âóHt[y9Y[ŠG.<.t.$.<.u"f‹G.f‹V.f.Ðf!Òu.
f‰Âè¬ÿr.è¶ÿf‹F.è ÿƒÃ.âÌfaÃèb.Multiple active partitions...f‹D.f.
F.f‰D.è0ÿr..>þ}Uª.….ÿ¼ú{Z_.úÿäè..Operating system load error...^
¬´.Š>b.³.Í.<.uñÍ.ôëý......................................Ÿ)..€.
...þ?.?...aÈ..................................................Uª

And we see something that looks to me a lot like a master boot record. Why? Because in the MBR all of the first 440 bytes is boot code, unlike a FAT boot sector which contains the distinctive bios parameter block (it looks like garbage above, but if you put that through a disassembler you get something that looks like valid 16 bit code).

Also, both of those look like valid and completely different boot sectors (complete with error messages). There is no way that a programming error could have "mangled" one to look like the other - it must just be that the wrong thing is being read.

---

In order to get CreateFile to return the disk instead of the partition it looks like you just need to pass it a different string, for example @"\\.\PhysicalDrive0" opens the first physical disk.

See:

• Low Level Disk Access
• INFO: Direct Drive Access Under Win32

Trillby

Some background info, that will explain what I'm doing in more detail. Our old systems used to boot from a FAT formatted CF card, so to update the software on the system, you'd just reflash the CF card using a windows app. The new system doesn't have any support for booting from CF cards, but it does have a hard drive and a USB port. So the plan was, like before use the windows app to flash the CF card, then take a an image of that CF card (in windows with my C# app) copy it onto the Live USB that would then flash the hard drive, from the Live Linux environment using dd, with the CF card image – Caffey

I don't think it is step 2 as if I put a image that i have created from the CF using a Linux VM and dd onto the Live USB it does work. – Caffey

@Caffey 7 Ah I see - so the image is eventually written to a hard disk. In that case I'm not sure why its not working - as long as the first sector (512 bytes) ends with the "valid bootsector" signature bytes (0x55, 0xAA) the BIOS should just go ahead and boot using that boot sector. My guess would be that the BIOS is in fact using that boot sector, but hardware differences mean that the boot loader (that previously worked on your old system) is failing on the new system. – Trillby

+1 for complete working code w/namespaces everything so no hunting around necessary. ♡ – Isoagglutinin

@Kragen Cool, thanks for re-assuring me about that, I guess I need to ask another question (after a bit o googling) about how to use WMI (or something similar) to get mappings from F:\ to \\.\PhysicalDriveX. – Caffey

Answer 2

This is what i've written to do get the \.\PhysicalDriveX path for a given drive letter. If Pass the drive letter into this and take the return value and pass into CreateFile as the first Param I should now get something similar to dd under Linux.

using System.Management; //Add in a reference to this as well in the project settings
public static string GetPhysicalDevicePath(char DriveLetter)
{
    ManagementClass devs = new ManagementClass( @"Win32_Diskdrive");
    {
        ManagementObjectCollection moc = devs.GetInstances();
        foreach(ManagementObject mo in moc)
        {
            foreach (ManagementObject b in mo.GetRelated("Win32_DiskPartition"))
            {
                foreach (ManagementBaseObject c in b.GetRelated("Win32_LogicalDisk"))
                {
                    string DevName = string.Format("{0}", c["Name"]);
                    if (DevName[0] == DriveLetter)
                        return string.Format("{0}", mo["DeviceId"]); 
                }
            }
        }
    }
    return "";
}

Caffey