Is perl's rename doing something odd on APFS?

Question

I think this is related to iCloud Drive in some way, but I have not investigated.

---

I'm trying to track down an issue with Perl's rename on macOS using the Apple Filesystem (APFS). I've been able to replicate this with perls back to at least 5.12.3 but all of mine are compiled with Apple LLVM version 9.1.0 (clang-902.0.39.1). Those same perls do not have this problem with FAT or HFS+ filesystems. I haven't noticed this problem anywhere else.

• Run it the first time. I end up with a Changes and a Changes.bak. That's exactly what I expected.

• Run it again. You end up with Changes and Changes 3 file. There is no Changes.bak. This is odd.

• Run it again. I end up with a Changes file, Changes.bak, and Changes 3.

• Run it again. I end up with a Changes file, Changes 3, and Changes 4. Again, there's no Changes.bak.

• If I remove the print line I can't get this to present ("Doctor, it hurts when I move my arm like this").

• I re-ordered the file handle opens and closes but that didn't seem to fix anything.

I figure there's something happening at the filesystem level. So I really have two questions:

• Is this a bug and at what level? Is the rename not guaranteed to finish whatever it needs to do before I start messing with file handles?

• I want to read the old file and create the new one that inserts some data in the middle. Copy the header, insert the new lines, output all the old line into the new file. I could write to a temp file and move that later, but am I doing anything else stupid?

If you can reproduce this behavior but don't know, leave a comment. Maybe there's something else odd about my system.

---

my $changes = "Changes";
my $bak     = $changes . ".bak";

rename $changes, $bak or die "Could not backup $changes. $!\n";

open my $in, '<', $bak or die "Could not read old $changes file! $!\n";
open my $out, ">", $changes;

# comment this print line and there's no problem
print {$out} 'Hello';

close $out;
close $in;

Belt

rename(3pl) simply calls rename(2). Win32 and VMS override this (as you can see here), but not MacOS as far as I can see. Check the man page for rename(2)? – Zena

I'd read rename(2) but it was not illuminating. – Belt

That search I linked didn't tell the whole picture. PerlIO adds a layer of indirection, and while I'm missing a few pieces, I don't see anything but Netware and Win32 overriding there. – Zena

Do you get the same behaviour from a C program? – Zena

Does not happen on my Mac. – Handknit

Works as expected for me. OSX 10.14.6 and perl 5.18 with apfs filesystem. – Handknit

Make sure your second open worked, it might be failing. $! might have a clue even if it succeeded, try printing it after each file operation. – Atrabilious

This is an issue with iCloud Drive and synchronization. I haven't developed a full demonstration yet. – Belt

Maybe you should unlink($bak) before the rename. – Heisel

The name in $bak doesn't exist yet. As I said, this is a remote sync issue with iCloud. – Belt

Answer 1

I know this is an old question, but this might be been related to a bug Apple's filesystem handling had about a year ago. We ran into some problems with the file metadata (mtime?) not getting set correctly in certain situations.

When you run into problems like this with perl, python, node, etc., try doing the same operation in a different language to see if the same behavior obtains. If so, that's likely an OS bug (most of these scripting languages often are thin wrappers around the c libraries anyway).

Cheers.

Primo

Thanks. After talking the directory out of iCloud Drive (~/Desktop, really), I haven't had this problem. Now it's on a Synology shared volume and the problem hasn't reappeared. – Belt