Can mmap and O_DIRECT be used together?

Question

As I understand it, when you mmap a file you are basically mapping the pages from the page cache for that file directly into your process, and when you use O_DIRECT you are bypassing the page cache. Does it ever make sense to use the two together? If my understanding is right how would it even work? mmap seems to rely on the file being in the page cache and O_DIRECT seems to prevent it from going there (assuming nothing else on the system has the file open). I found this question but the answerer seems to think it's perfectly normal to do.

Westsouthwest

That other question is using O_DIRECT with a different file than the one being mapped. – Bolen

Have you tried it? My guess is that mmap() would simply ignore the fact that the file is opened in O_DIRECT mode -- that option just affects how the read() and write() calls behave. – Bolen

Related: #19909112 – Bolen

I don't think "you are basically mapping the pages from the page cache" is correct. If the file is in the page cache, then mmap will use it. But if it's not, it maps it from the disk location. – Bolen

@Bolen You say if it is not green, it is make it green, but it is not green. if the file (inode data) is not in page it maps it form the dist location. But what is map to dist location? It is allocation of page of memory and connection it is to disk blocks, this is exactly what page cache is. So if it is not in page cache it fill page cache. Of course it is about Linux kernel. – Thaddeusthaddus

The page cache and virtual memory are two ways of getting disk blocks into memory. They're not the same thing, although they can be used together. – Bolen

@Bolen -- I don't think that makes sense. Say it 'maps it from the disk location', I assume that means reading the page would then trigger reading from the disk. But then the page has to be in RAM for subsequent accesses , unless you expect every read from that page to cause a page fault, so I think the OS must put put that read page in RAM somewhere, and that might as well be the page cache. I'm not even sure mapping disk blocks is how disk drivers usually work -- during the whole 32-bit era disks were much bigger than address space, so I assume data was sent to/from the drive by queues. – Westsouthwest

Anyway, your question was whether it works. Try it and see. The design is just a trivial side issue. – Bolen

Maybe I'm not understanding the gist of your question. I thought you were asking what happens if a process opens a file in O_DIRECT mode, and then calls mmap on that descriptor. Are you actually asking what happsn if process A maps the file, and process B opens the file in O_DIRECT mode -- will they see each other's changes to the file? – Bolen

@Bolen -- If it works I still want to understand how. My question was the first one -- if you open a file in O_DIRECT mode and then call mmap on it. I still think it would have to use the page cache, which is what prompted the whole question. – Westsouthwest

How it works would be a better question for unix.stackexchange.com. It doesn't affect programs that use it. – Bolen

Also interesting: yarchive.net/comp/linux/o_direct.html – Yager

Answer 1

I think it would not have a lot of sense.

O_DIRECT means that all I/O should be reflected in storage, as soon as possible (without cache).

The mapped pages is a copy of storage (file) in the memory. Reflecting every read from and write to memory would have to do some I/O and this would be huge performance hit.

Crellen