Performance problems counting the files with System.IO.Directory.GetFiles()

Question

I use in a C# program

System.IO.Directory.GetFiles(dirname, "*.*", System.IO.SearchOption.AllDirectories).Length;

to count to number of files in a directory(and its subdirectories) on a share. I do this for many directories within a loop. The problem is now, that after 20-30 calls of GetFiles(...).Length my program is suddenly frozen or becomes very slow. What could be the reason for that and how can this problem be solved! Are there workarounds?

Brunella

After 20-30 calls are you sure you don't (as a result of your looping) target a directory that has a much higher number of files, a much slower network connectivity (if remote), security restrictions, etc? I.e. The performance hit isn't a result of repeatedly calling the function, but as a result of the udnerlying file structure? – Venal

It might be occupying memory to the extent that paging gets started! – Hexameter

@smudge202 - too bad I can't upvote your comment. – Seeger

I agree with Abdul, check to see if your program has any recorder hard faults/sec which would indicate a lot of page swapping. Remember that reading file pointers into memory still involves reading from the hard disk so there might be scattered files or drive contention. – Coiffeur

@Seeger you can...(if you hover your mouse over a comment you should see an up arrow & flag icon appear to the left, you can click on the up arrow to upvote the comment) I upvoted your comment as an example... – Gemperle

Is the call Directory.EnumerateFiles(dirname, ".",System.IO.SearchOption.AllDirectories).Count() slow too? – Hubbell

@Jehof: Unfortunately the progam becomes very slow too after 50 calls of Count()! – Brunella

You say this is counting files on a share? You network doesn't enforce quota's of any kind does it? – Accredit

+1 to smudge and one more to Jon for being so gosh darn helpful to the noob. – Seeger

Answer 1

Take a look at http://www.codeproject.com/KB/files/FastDirectoryEnumerator.aspx. Mybe this article will help you.

Workman

Answer 2

Are you sure you aren't somehow storing the results of previous calls making your application starve on memory? Check the memory allocation for your application in processes tab of Task Manager.

Dorsad