UNIX atomically swap or replace directories?

Question

I have the following problem. I have two directories which I'll refer to as dir1 and dir2. They are both populated with both files and subdirectories. They are both on the same filesystem. The contents of dir1 is old, and must be replaced with the contents of dir2. The contents of dir1 can be discarded. However, I need this replacing to happen atomically. My current solution is the following.

mv dir1 dir1_backup; mv dir2 dir1; rm -rf dir1_backup

If the first command fails, the second command will not run. This is good. However, what if the second command fails? dir1 does not exist anymore, and I'm left in an invalid state.

What is the best way to accomplish this? Is it possible?

Bewitch

It might be too late - the easy way to do atomic directory swaps is via symbolic link. – Halfpenny

possible duplicate of Moving a directory atomically – Halfpenny

don't you mean && instead of ; to separate your cmds? Unfortunately, while && should work, I'm not sure I'd ever want to "bet the store" like that. I would build something that does validations, especially before the final rm step. Good luck. – S

I have seen - #307937 - before. The solutions presented within it fail my edge case. – Bewitch

Would care to explain why using symlinks, and doing a mv -T point_to_dir2 point_to_dir1 wouldn't work in your edge case. – Thermionic

Answer 1

Linux provides the renameat2 system call which with the RENAME_EXCHANGE flag will swap 2 objects:

renameat2(AT_FDCWD, "dir1", AT_FDCWD, "dir2", RENAME_EXCHANGE);

Jocelyn

If someone wants a wrapper executable on renameat2, I wrote one. You can download it from github.com/AbhyudayaSharma/exchange – Kristlekristo

Answer 2

I don't think that your assertion above that the second command will fail if the first fails is true. If mv dir1 dir1_backup fails, then dir1 will still exist, and mv dir2 dir1 will make dir2 a subdirectory of dir1. Also, since you are passing -f to rm in the third command, it will silently fail to delete a non-existant directory.

For reference, three bash operators (assuming bash here) and what they do:

• ; simply executes the following command after the first exits, regardless of the exit status.
• && executes the following command only if the previous command exited cleanly (ie, had an exit status of 0.)
• || executes the following command only if the previous command exited uncleanly (ie, had a non-zero exit code.)

If you want each command's execution to be dependent upon the previous command's successful execution, then you might want to consider something like the following:

    mv dir1 dir1_backup && mv dir2 dir1 && rm -rf dir1_backup

If the moving of a directory is a failure mode you expect, then it may be reasonable to test the return value of the move or to test for the existence of the expected directory before proceeding to remove the old directory contents. If the test fails, then move the old directory contents back. The overall operation will have failed, but at least you will not be in an invalid state. For example:

    mv dir1 dir1_backup && \
    mv dir2 dir1 && \
    rm -rf dir1_backup || mv dir1_backup dir1

Since we know that && executes the following command only if the previous command exits successfully, mv dir2 dir1 will only execute if mv dir1 dir1_backup succeeds.

Further, rm -rf dir1_backup will only execute if the move of dir2 to dir1 succeeds. However, if the last exit code (the code returned by mv dir2 dir1 if it has failed) is non-zero, the || operator causes mv dir1_backup dir1 to execute.

Hope that helps! Let me know if it needs clarification.

Colonial

cfunk, thanks for clearing that up. I thought that ; and && did the opposite of what they actually do. Your solution is closer to what I want, but there is a corner case. First command completes successfully. Second command fails, triggering the last command. (mv dir1_backup dir1) Imagine if the LAST command fails. I am still left with in invalid state: dir1 does not exist. – Bewitch

If you are able to rename the directory in the first case, what is the failure mode you expect to prevent you from moving it back? If that's expected, you might want to investigate the use of symlinks instead of moving directories around. If possible, it might be best to have the application accessing the directory look at the new location and not worry about shuffling directories at all. – Colonial

Answer 3

On Linux you might be able to get away with bind mounting the new directory on top of the old. However you would still have issues with processes that had an open handle on a directory inside the old directory(eg their current working directory).

mount --bind dir2 dir1

Similar tricks would allow you to gain access to the old directory for cleanup purposes.

If this filesystem is exported over NFS then you would probably need to ensure the export options were set to allow traversing file system boundaries and do the bind mount on the server.

Other *nix have similar features but they aren't standardised.

Eveliaevelin