Race condition while moving files on Linux

Question

Suppose I have two scripts. The first one puts (with mv command) some files into a directory, the second one checks the directory once in a while and processes the files. The situation I'm concerned about is when the second script starts processing of the file which is only partly moved at the moment. Can this happen in real life on XFS file system?

Blasted

Answer 1

It depends on where you're moving the files from. mv WITHIN a single filesystem is atomic, otherwise it must do a copy which is not atomic (followed by a delete of the original file), and is prone to the kind of race condition you mention.

FWIW, this is normal POSIX semantics, nothing particular to XFS.

Signorina

The moral of the story is that you can make sure you're safe by the simple expedient of making sure the first script puts items into the directory from the same filesystem; if this is in doubt you could mv them into a subdirectory of the destination directory first and then move them out to the destination directory to be sure that their arrival is atomic. – Whittling

Answer 2

Race condition would not occure in your case in XFS file system. However XFS allows multiple processes to read and write a file at once by using flexible locking scheme in contrast to Unix file systems single threaded inode lock. XFS tack care of serializing the writes on the same region by multiple processes .

XFS uses direct I/O for accessing the file.Direct I/O allows an application to specify that its data not to be cached in the buffer cache.

When using normal, buffered I/O, multiple readers can access the file concurrently, but only a single writer is allowed access to the file at a time. When using direct I/O, multiple readers and writers can access the file simultaneously.

Anecdotage