How to write mmap input memory to O_DIRECT output file?

Question

why doesn't following pseudo-code work (O_DIRECT results in EFAULT)

in_fd = open("/dev/mem");
in_mmap = mmap(in_fd);
out_fd = open("/tmp/file", O_DIRECT);
write(out_fd, in_mmap, PAGE_SIZE);

while following does (no O_DIRECT)

in_fd = open("/dev/mem");
in_mmap = mmap(in_fd);
out_fd = open("/tmp/file");
write(out_fd, in_mmap, PAGE_SIZE);

I guess it's something with virtual kernel pages to virtual user pages, which cannot be translated in the write call?

Best regards,

Friedrich

Steen

O_DIRECT writes need to pin the pages; maybe that is the problem... What are you actually trying to do? If this is a memory-mapped hardware device, you might be better off adding splice support to the driver... Also, please provide a more complete example (including all flags to all system calls). Ideally one that anybody could compile and run to reproduce. – Wesla

Answer 1

Using mmap() with O_DIRECT is tricky. There are some restrictions. The output to the file should be block aligned. For example, if you set offset in mmap() to 0 your code will work. You have to check the block size of your filesystem to set that value properly.

Pebrook

Answer 2

There are two approaches:

1. Use CMA and vm_insert_pages. Detailed instructions can be found in my another Stack Overflow answer

2. Use no-map reserved memory combined with my patch series. This series includes a sample that demonstrates how to implement this approach easily.

Conglobate