why NTFS filesystem is so slow compared to EXT3 filesystem when copying large amount of small files? [closed]
Asked Answered
W

0

9

I've run the following test I've created a folder containing 15'000 files of 400 bytes using this batch :

@ECHO off
SET times=15000

FOR /L %%i IN (1,1,%times%) DO (
    fsutil file createnew filename%%i.txt 400
)

then I copy past it on my Windows Computer using this command :

robocopy LargeNumberOfFiles\ LargeNumberOfFiles2\

After it has completed I can see that the transfer rate was 915810 Bytes/sec this is less than 1 MB/s. It took me several seconds to copy 7 MBytes Please note that this is very slow.

I've tried the same with a folder with a single file of 50 Mbytes and the transfer rate is 1219512195 Bytes/sec. (yeah GB/s) instantaneous.

Why copying large number of files take so much time - ressources on a windows filesystem ?

Please note that I've tried to do the same on a linux system which runs on the same computer in a virtual machine (vmware player) with ext3 filesystem.

I use the cp command and the copy is instantaneous !

Please also note the following :

  • no antivirus
  • I've tested that behaviour on multiple windows computers (always ntfs) i always get comparable results (transfer rate under 1MB/s avg 7-8 seconds to copy 7 MBytes)
  • I've tested on multiple linux ext3 system the copy is always instantaneous for that amount (15000 files of 400 bytes)
  • The question is about understanding what makes windows filesystem so slow to copy large number of files compared to a linux one for instance.
Wendelin answered 12/12, 2012 at 11:3 Comment(2)
Yes the result is the same if i copy past through windows explorer or using copy cmdWendelin
I suspect it's not the copying of a large number of files that is experiencing bad performance, but the fact that all those files are in one directory - 15000 files in one directory on NTFS is going to be slow (actually that's quite a lot for most file systems, but some handle it better than others).Subcelestial

© 2022 - 2024 — McMap. All rights reserved.