Why does WriteFile call ReadFile and how do I avoid it?
Asked Answered
G

1

7

When sequentially overwriting (in e.g. 8192 bytes aligned and sized blocks) in existing parts of a file (sometimes/always?) WriteFile causes file reads from disk, which totally trashes performance. I don't quite understand why this happens and how to avoid it.

Using Process Monitor I get the following information:

Event.Class: File System
Event.Operator: ReadFile
Event.Offset: 1 875 288 653 824
Event.Length: 4 096
Event.I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O
Event.Priority: Normal 

0   FLTMGR.SYS  FltDecodeParameters + 0x1ab1    0xfffff8016e7f4621  C:\WINDOWS\System32\drivers\FLTMGR.SYS
1   FLTMGR.SYS  FltDecodeParameters + 0x166c    0xfffff8016e7f41dc  C:\WINDOWS\System32\drivers\FLTMGR.SYS
2   FLTMGR.SYS  FltDecodeParameters + 0x1093    0xfffff8016e7f3c03  C:\WINDOWS\System32\drivers\FLTMGR.SYS
3   FLTMGR.SYS  FltDecodeParameters + 0xe8e 0xfffff8016e7f39fe  C:\WINDOWS\System32\drivers\FLTMGR.SYS
4   ntoskrnl.exe    IoPageRead + 0x258  0xfffff8015b8ba598  C:\WINDOWS\system32\ntoskrnl.exe
5   ntoskrnl.exe    IoPageRead + 0x4d4  0xfffff8015b8ba814  C:\WINDOWS\system32\ntoskrnl.exe
6   ntoskrnl.exe    IoGetBaseFileSystemDeviceObject + 0x7c8 0xfffff8015b8b8ae8  C:\WINDOWS\system32\ntoskrnl.exe
7   ntoskrnl.exe    PfFileInfoNotify + 0x6ead   0xfffff8015b8a823d  C:\WINDOWS\system32\ntoskrnl.exe
8   ntoskrnl.exe    setjmpex + 0x224d   0xfffff8015b9dfdbd  C:\WINDOWS\system32\ntoskrnl.exe
9   ntoskrnl.exe    memcpy + 0x250  0xfffff8015b9e2590  C:\WINDOWS\system32\ntoskrnl.exe
10  ntoskrnl.exe    MmMapLockedPagesSpecifyCache + 0x132e   0xfffff8015b89950e  C:\WINDOWS\system32\ntoskrnl.exe
11  ntoskrnl.exe    MmMapLockedPagesSpecifyCache + 0x949    0xfffff8015b898b29  C:\WINDOWS\system32\ntoskrnl.exe
12  ntoskrnl.exe    CcCopyWriteEx + 0x1ca   0xfffff8015b965f9a  C:\WINDOWS\system32\ntoskrnl.exe
13  NTFS.sys    NTFS.sys + 0xb7518  0xfffff8016f907518  C:\WINDOWS\System32\Drivers\NTFS.sys
14  FLTMGR.SYS  FltIsCallbackDataDirty + 0xb15  0xfffff8016e7f58b5  C:\WINDOWS\System32\drivers\FLTMGR.SYS
15  FLTMGR.SYS  FltDecodeParameters + 0x3ad 0xfffff8016e7f2f1d  C:\WINDOWS\System32\drivers\FLTMGR.SYS
16  FLTMGR.SYS  FltParseFileName + 0x159c   0xfffff8016e822eec  C:\WINDOWS\System32\drivers\FLTMGR.SYS
17  ntoskrnl.exe    NtWriteFile + 0x432 0xfffff8015bca8532  C:\WINDOWS\system32\ntoskrnl.exe
18  ntoskrnl.exe    setjmpex + 0x37f3   0xfffff8015b9e1363  C:\WINDOWS\system32\ntoskrnl.exe
19  ntdll.dll   ZwWriteFile + 0xa   0x7ff9eb34357a  C:\WINDOWS\SYSTEM32\ntdll.dll
20  KERNELBASE.dll  WriteFile + 0x88    0x7ff9e84cd458  C:\WINDOWS\system32\KERNELBASE.dll
21  capture.exe capture.exe + 0x6f40b   0x7ff6b971f40b  C:\ingest\server\capture.exe
22  capture.exe capture.exe + 0x5998    0x7ff6b96b5998  C:\ingest\server\capture.exe
23  capture.exe capture.exe + 0x13285   0x7ff6b96c3285  C:\ingest\server\capture.exe
24  nxtcapture.exe  nxtcapture.exe + 0x2b89 0x7ff6b96b2b89  C:\ingest\server\capture.exe
25  capture.exe capture.exe + 0x136be1  0x7ff6b97e6be1  C:\ingest\server\capture.exe
26  KERNEL32.DLL    BaseThreadInitThunk + 0x22  0x7ff9eab02d92  C:\WINDOWS\system32\KERNEL32.DLL
27  ntdll.dll   RtlUserThreadStart + 0x34   0x7ff9eb2b9f64  C:\WINDOWS\SYSTEM32\ntdll.dll

The following information from CcCopyWriteEx might be relevant:

If Wait is TRUE, CcCopyWriteEx is guaranteed to complete the copy request and return TRUE. If the required pages of the cached file are already resident in memory, the data will be copied immediately and no blocking will occur. If any needed pages are not resident, the caller will be put in a wait state until all required pages have been made resident and the data can be copied.

If I understand this correctly this works similar to Memory Mapping where the pages that are overwritten first need to read from the disk, even if the entire page is to be overwritten. Any way to avoid this? Possibly some Cache Manager Routine?

FILE_FLAG_WRITE_THROUGH makes a difference. It still calls ReadFile but not as often and the amount read from disk seems to be much lower. Though I don't fully understand what is happening and the disk activity and queue length is still much higher than when writing to non-initialized pages in the target file.

0   FLTMGR.SYS  FltDecodeParameters + 0x1ab1    0xfffff8018a9f4621  C:\WINDOWS\System32\drivers\FLTMGR.SYS
1   FLTMGR.SYS  FltDecodeParameters + 0x166c    0xfffff8018a9f41dc  C:\WINDOWS\System32\drivers\FLTMGR.SYS
2   FLTMGR.SYS  FltDecodeParameters + 0x1093    0xfffff8018a9f3c03  C:\WINDOWS\System32\drivers\FLTMGR.SYS
3   FLTMGR.SYS  FltDecodeParameters + 0xe8e 0xfffff8018a9f39fe  C:\WINDOWS\System32\drivers\FLTMGR.SYS
4   ntoskrnl.exe    IoPageRead + 0x258  0xfffff803f5e4d598  C:\WINDOWS\system32\ntoskrnl.exe
5   ntoskrnl.exe    IoPageRead + 0x4d4  0xfffff803f5e4d814  C:\WINDOWS\system32\ntoskrnl.exe
6   ntoskrnl.exe    IoGetBaseFileSystemDeviceObject + 0x7c8 0xfffff803f5e4bae8  C:\WINDOWS\system32\ntoskrnl.exe
7   ntoskrnl.exe    PfFileInfoNotify + 0x6ead   0xfffff803f5e3b23d  C:\WINDOWS\system32\ntoskrnl.exe
8   ntoskrnl.exe    setjmpex + 0x224d   0xfffff803f5f72dbd  C:\WINDOWS\system32\ntoskrnl.exe
9   ntoskrnl.exe    memcpy + 0x250  0xfffff803f5f75590  C:\WINDOWS\system32\ntoskrnl.exe
10  ntoskrnl.exe    MmMapLockedPagesSpecifyCache + 0x132e   0xfffff803f5e2c50e  C:\WINDOWS\system32\ntoskrnl.exe
11  ntoskrnl.exe    MmMapLockedPagesSpecifyCache + 0x949    0xfffff803f5e2bb29  C:\WINDOWS\system32\ntoskrnl.exe
12  ntoskrnl.exe    CcCopyWriteEx + 0x1ca   0xfffff803f5ef8f9a  C:\WINDOWS\system32\ntoskrnl.exe
13  NTFS.sys    NTFS.sys + 0x8ce8   0xfffff8018c068ce8  C:\WINDOWS\System32\Drivers\NTFS.sys
14  NTFS.sys    NTFS.sys + 0x7dc7   0xfffff8018c067dc7  C:\WINDOWS\System32\Drivers\NTFS.sys
15  FLTMGR.SYS  FltIsCallbackDataDirty + 0x424  0xfffff8018a9f51c4  C:\WINDOWS\System32\drivers\FLTMGR.SYS
16  FLTMGR.SYS  FltDecodeParameters + 0xea6 0xfffff8018a9f3a16  C:\WINDOWS\System32\drivers\FLTMGR.SYS
17  ntoskrnl.exe    NtReadFile + 0xa1b  0xfffff803f623c72b  C:\WINDOWS\system32\ntoskrnl.exe
18  ntoskrnl.exe    NtWriteFile + 0x680 0xfffff803f623b780  C:\WINDOWS\system32\ntoskrnl.exe
19  ntoskrnl.exe    setjmpex + 0x37f3   0xfffff803f5f74363  C:\WINDOWS\system32\ntoskrnl.exe
20  ntdll.dll   ZwWriteFile + 0xa   0x7ff9a313357a  C:\WINDOWS\SYSTEM32\ntdll.dll
21  KERNELBASE.dll  WriteFile + 0x88    0x7ff9a059d458  C:\WINDOWS\system32\KERNELBASE.dll
22  capture.exe capture.exe + 0x6f40b   0x7ff7eef0f40b  C:\ingest\server\capture.exe
23  capture.exe capture.exe + 0x5998    0x7ff7eeea5998  C:\ingest\server\capture.exe
24  capture.exe capture.exe + 0x13285   0x7ff7eeeb3285  C:\ingest\server\capture.exe
25  capture.exe capture.exe + 0x2b89    0x7ff7eeea2b89  C:\ingest\server\capture.exe
26  capture.exe capture.exe + 0x136c71  0x7ff7eefd6c71  C:\ingest\server\capture.exe
27  KERNEL32.DLL    BaseThreadInitThunk + 0x22  0x7ff9a0a92d92  C:\WINDOWS\system32\KERNEL32.DLL
28  ntdll.dll   RtlUserThreadStart + 0x34   0x7ff9a30a9f64  C:\WINDOWS\SYSTEM32\ntdll.dll

Also interesting with write-through is that it seems to be reading single sectors (note Length: 4096 in ReadFile calls).

13:09:00,8579774    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 519 567 872, Length: 3 112 960, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:00,9399289    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 522 680 832, Length: 2 957 312, I/O Flags: Write Through, Priority: Normal
13:09:00,9418288    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 522 680 832, Length: 2 957 312, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,0466551    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 525 638 144, Length: 3 145 728, I/O Flags: Write Through, Priority: Normal
13:09:01,0486614    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 525 638 144, Length: 3 145 728, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,1476023    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 528 783 872, Length: 2 490 368, I/O Flags: Write Through, Priority: Normal
13:09:01,1492754    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 528 783 872, Length: 2 490 368, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,2594166    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 531 274 240, Length: 2 981 888, I/O Flags: Write Through, Priority: Normal
13:09:01,2613028    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 531 274 240, Length: 2 981 888, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,3693932    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 534 256 128, Length: 2 678 784, I/O Flags: Write Through, Priority: Normal
13:09:01,3705444    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 534 256 128, Length: 90 112, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,3708446    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 534 346 240, Length: 2 588 672, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,4846832    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 536 934 912, Length: 3 096 576, I/O Flags: Write Through, Priority: Normal
13:09:01,4862724    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 536 934 912, Length: 3 096 576, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,5905083    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 540 031 488, Length: 3 211 264, I/O Flags: Write Through, Priority: Normal
13:09:01,5923326    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 540 031 488, Length: 3 211 264, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,6951367    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 543 242 752, Length: 2 670 592, I/O Flags: Write Through, Priority: Normal
13:09:01,6967169    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 543 242 752, Length: 2 670 592, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,8028430    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 545 913 344, Length: 2 342 912, I/O Flags: Write Through, Priority: Normal
13:09:01,8045821    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 545 913 344, Length: 2 342 912, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:01,9234657    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 548 256 256, Length: 3 055 616, I/O Flags: Write Through, Priority: Normal
13:09:01,9254169    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 548 256 256, Length: 3 055 616, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,0273731    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 551 311 872, Length: 3 424 256, I/O Flags: Write Through, Priority: Normal
13:09:02,0288307    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 551 311 872, Length: 3 424 256, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,1316979    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 554 736 128, Length: 2 678 784, I/O Flags: Write Through, Priority: Normal
13:09:02,1327686    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 554 736 128, Length: 2 678 784, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,2327822    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 557 414 912, Length: 2 605 056, I/O Flags: Write Through, Priority: Normal
13:09:02,2341425    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 557 414 912, Length: 2 605 056, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,3390269    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 560 019 968, Length: 2 711 552, I/O Flags: Write Through, Priority: Normal
13:09:02,3402005    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 560 019 968, Length: 2 711 552, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,4392960    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 562 731 520, Length: 3 334 144, I/O Flags: Write Through, Priority: Normal
13:09:02,4411620    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 562 731 520, Length: 3 334 144, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,5463951    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 566 065 664, Length: 2 277 376, I/O Flags: Write Through, Priority: Normal
13:09:02,5474362    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 566 065 664, Length: 1 835 008, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,5484791    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 567 900 672, Length: 442 368, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,6509821    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 568 343 040, Length: 3 342 336, I/O Flags: Write Through, Priority: Normal
13:09:02,6524223    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 568 343 040, Length: 3 342 336, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,7516238    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 571 685 376, Length: 2 555 904, I/O Flags: Write Through, Priority: Normal
13:09:02,7529352    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 571 685 376, Length: 2 555 904, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,8526049    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 574 241 280, Length: 2 138 112, I/O Flags: Write Through, Priority: Normal
13:09:02,8541937    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 574 241 280, Length: 2 138 112, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:02,9589201    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 576 379 392, Length: 3 039 232, I/O Flags: Write Through, Priority: Normal
13:09:02,9602760    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 576 379 392, Length: 3 039 232, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,0738157    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 579 418 624, Length: 2 785 280, I/O Flags: Write Through, Priority: Normal
13:09:03,0752279    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 579 418 624, Length: 2 785 280, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,1890830    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 582 203 904, Length: 3 383 296, I/O Flags: Write Through, Priority: Normal
13:09:03,1908153    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 582 203 904, Length: 3 383 296, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,3075209    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 585 587 200, Length: 3 055 616, I/O Flags: Write Through, Priority: Normal
13:09:03,3091505    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 585 587 200, Length: 3 055 616, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,4250265    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 588 642 816, Length: 3 006 464, I/O Flags: Write Through, Priority: Normal
13:09:03,4266098    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 588 642 816, Length: 3 006 464, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,5379099    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 591 649 280, Length: 3 293 184, I/O Flags: Write Through, Priority: Normal
13:09:03,5392216    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 591 649 280, Length: 3 293 184, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,6386064    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 594 942 464, Length: 2 596 864, I/O Flags: Write Through, Priority: Normal
13:09:03,6403585    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 594 942 464, Length: 2 596 864, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,7476785    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 597 539 328, Length: 2 498 560, I/O Flags: Write Through, Priority: Normal
13:09:03,7495491    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 597 539 328, Length: 2 498 560, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,8598891    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 600 037 888, Length: 3 522 560, I/O Flags: Write Through, Priority: Normal
13:09:03,8613834    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 600 037 888, Length: 1 417 216, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,8622310    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 601 455 104, Length: 2 105 344, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:03,9748379    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 603 560 448, Length: 3 006 464, I/O Flags: Write Through, Priority: Normal
13:09:03,9761048    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 603 560 448, Length: 3 006 464, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,0970842    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 606 566 912, Length: 3 104 768, I/O Flags: Write Through, Priority: Normal
13:09:04,0986363    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 606 566 912, Length: 3 104 768, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,2026181    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 609 671 680, Length: 2 342 912, I/O Flags: Write Through, Priority: Normal
13:09:04,2047880    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 839 040, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2059156    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 908 672, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2062328    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 912 768, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2124125    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 916 864, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2125033    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 920 960, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2125692    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 925 056, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2127042    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 929 152, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2130060    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 933 248, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2132903    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 610 937 344, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2134340    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 092 992, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2136517    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 097 088, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2139954    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 101 184, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2140859    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 105 280, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2142153    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 109 376, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2143550    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 113 472, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2144234    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 117 568, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2144772    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 611 121 664, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:04,2151351    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 609 671 680, Length: 2 342 912, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,3066104    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 612 014 592, Length: 2 940 928, I/O Flags: Write Through, Priority: Normal
13:09:04,3085623    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 612 014 592, Length: 2 940 928, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,4154163    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 614 955 520, Length: 3 186 688, I/O Flags: Write Through, Priority: Normal
13:09:04,4177492    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 614 955 520, Length: 3 186 688, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,5174478    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 618 142 208, Length: 2 547 712, I/O Flags: Write Through, Priority: Normal
13:09:04,5192064    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 618 142 208, Length: 2 547 712, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,6315595    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 620 689 920, Length: 2 891 776, I/O Flags: Write Through, Priority: Normal
13:09:04,6328619    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 620 689 920, Length: 2 891 776, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,7344473    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 623 581 696, Length: 3 055 616, I/O Flags: Write Through, Priority: Normal
13:09:04,7358909    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 623 581 696, Length: 3 055 616, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,8373090    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 626 637 312, Length: 2 621 440, I/O Flags: Write Through, Priority: Normal
13:09:04,8385106    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 626 637 312, Length: 2 621 440, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:04,9471593    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 629 258 752, Length: 3 047 424, I/O Flags: Write Through, Priority: Normal
13:09:04,9485457    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 629 258 752, Length: 3 047 424, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:05,0705718    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 632 306 176, Length: 3 022 848, I/O Flags: Write Through, Priority: Normal
13:09:05,0730288    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 632 306 176, Length: 2 703 360, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:05,0748379    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 635 009 536, Length: 319 488, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:05,1867648    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 635 329 024, Length: 3 383 296, I/O Flags: Write Through, Priority: Normal
13:09:05,1885589    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 635 329 024, Length: 3 383 296, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:05,3091527    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 638 712 320, Length: 2 637 824, I/O Flags: Write Through, Priority: Normal
13:09:05,3108466    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 638 712 320, Length: 2 637 824, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:05,4259338    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 641 350 144, Length: 3 506 176, I/O Flags: Write Through, Priority: Normal
13:09:05,4277572    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 641 350 144, Length: 3 506 176, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Write Through, Priority: Normal
13:09:05,5460495    capture.exe 3336    WriteFile   L:\data.loop    SUCCESS Offset: 1 927 644 856 320, Length: 3 309 568, I/O Flags: Write Through, Priority: Normal
13:09:05,5479142    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 596 544, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5641906    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 633 408, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5679702    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 637 504, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5680940    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 641 600, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5684703    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 645 696, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5687985    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 649 792, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5688542    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 653 888, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5688983    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 657 984, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
13:09:05,5692249    capture.exe 3336    ReadFile    L:\data.loop    SUCCESS Offset: 1 927 647 662 080, Length: 4 096, I/O Flags: Non-cached, Paging I/O, Synchronous Paging I/O, Priority: Normal
Gyrostat answered 26/8, 2015 at 10:35 Comment(4)
You apparently haven't configured Process Monitor to use a symbol server, so your stack traces are misleading. The functions names listed in the backtrace are just the nearest exported symbol. In most cases its actually a non-exported function that has been called. Configuring procmon to use the Microsoft symbol server should provide more informative backtraces: blogs.msdn.com/b/vijaysk/archive/2009/04/02/…Synaesthesia
It would help to show minimal code that reproduces the issue.Dorotheadorothee
Is the destination drive compressed or encrypted?Pickens
@Cauterite: No. It actually turns out the file cache uses memory mapping internally which causes hard page faults on 256Kb alignments.Gyrostat
C
3

The reads that you are seeing very likely are the operating system cache read-ahead. The FILE_FLAG_WRITE_THROUGH flag is an instruction to the disk driver to write data directly to drive bypassing the hardware disk system cache (although it is notoriously ignored by many less-than-enterprise-grade SATA drives). That flag does not prevent caching data by either the OS or the disk system cache, nor does it disable the OS cache read-ahead.

I am assuming that your intention is to write the file at a high data rate almost continuously and rarely if ever read it, like in a video or instrumentation capture system. It it is, then what you are looking for is another flag, FILE_FLAG_NO_BUFFERING, which disabled both OS caching of file data on either reading or writing. Obviously, read-ahead is also disabled to satisfy the promise that reads do not come from the system cache.

Do not specify FILE_FLAG_WRITE_THROUGH together with FILE_FLAG_NO_BUFFERING unless you want the guarantee that the data has been actually turned into magnetic bits by the time your call to WriteFile returns. This is intended for transaction processing (like database servers) and normally reduces application's disk I/O performance, in some cases by a large factor.

Cinerator answered 6/1, 2016 at 23:21 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.