PHP: Can I prevent slow work of fopen in Windows (NTFS)?
Asked Answered
P

0

0

I have a script in PHP that reads several files and I run it on Windows 10. When I run it for the first time, it works slowly (tens of seconds) and next times, it works quickly (less than a second). After some time elapses (can't say how much exactly yet), the problem repeats, so it looks like there's some kind of cache involved.

To narrow down the problem, I've added some "manual profiling" and found that the problematic bit is the fopen (compared to it, flock, fgets take no time):

$profile_log = '';
$time = microtime(true);

// I'll remove the second line that presumably doesn't matter
// once I re-run the profiling and confirm that
$fileHandle = fopen($file_full_path_and_name, 'r');
if(!$fileHandle) return null;

$new_time = microtime(true);
$profile_log .= "open time: " . ($new_time - $time) . "\n";
$time = $new_time;

// ...

file_put_contents('./debug.txt', "$profile_log\n\n", FILE_APPEND);

gives me "open time" of 0.7 seconds and more for each of my files (of the size in the range of 600 KB - 10 MB).

What could be wrong and how can I fix this?


I've also asked AI for suggestions and from what it replied I suppose that

  1. Windows has both file buffering and caching,

  2. it's invoked by fopen (as far as I understand, this causes pre-reading of the whole file instead of just opening – while the script reads just several first line of each file),

  3. using

    $fileHandle = fopen($file_full_path_and_name, 'r', false, stream_context_create([
        'options' => [
            'write_buffer' => 0,
            'read_buffer' => 0,
            'buffer' => 0,
            'flags' => FILE_FLAG_NO_BUFFERING,
        ]
    ]));
    

    should disable that.

However, the above snippet doesn't seem to reduce the run time [TODO: retest once more].

Which of 1-2-3 are correct? Any other ideas how to deal with this issue or at least debug it? There's also a problem with reproducing the issue since I'm not aware yet of ways to invalidate the cache.

Pyle answered 5/5, 2023 at 19:22 Comment(5)
This is really odd, it shouldn't read anything when you just open it. First thing I'd try would be to run your timing test with a really tiny file and really huge file, to see how they compare. Is this directory a network mounted share maybe?Corroboration
@AlexHowansky nice suggestion, first experiment resulted in no "delay" at all, but I'll rerun it later: may be when one creates and edits files via other tools, the cache is heated, too. No, it's not a network mounted share, just some files on hard drive; the only irregularity is, they are not inside the folder that's hosted by WAMP itself (to be precise, it's a similar package, not WAMP itself).Pyle
@AlexHowansky yes, there's a difference based on the file size: for a 214 bytes file the time was 0.006 seconds, for 10.8 MB files (several with same content) the time was in the range 0.3-0.45 seconds, but for a 404 KB file it was, unexpectedly, 0.26 seconds (too close to the "huge" files)Pyle
@TangentiallyPerpendicular this ban is about answering and has nothing to do with some "may be ..?" in questions. GPT is good at suggesting approaches and narrowing down the search, it's just its answers should be verified before being used as statements (which I'm not doing here anyway).Pyle
This seems related, at least it mentions the ~1000 B "slope" which seems to be where the speed of fopen changes considerablyPyle

© 2022 - 2024 — McMap. All rights reserved.