Is O_LARGEFILE needed just to write a large file?

Question

Is the O_LARGEFILE flag needed if all that I want to do is write a large file (O_WRONLY) or append to a large file (O_APPEND | O_WRONLY)?

From a thread that I read titled "Cannot write >2gb index file" on the CLucene-dev mailing list, it appears that O_LARGEFILE might be needed to write large files, but participants in that discussion are using O_RDWR, not O_WRONLY, so I am not sure.

Mclane

Answer 1

O_LARGEFILE should never be used directly by applications. It's to be used internally by the 64-bit-offset-compatible version of open in libc when it makes the syscall to the kernel (Linux, or possibly another kernel with this 64-bit-offset-mode-is-a-second-class-citizen nonsense). Just make sure to always include -D_FILE_OFFSET_BITS=64 in your CFLAGS and you'll never have to worry about anything.

Cassock

Do you have any source for that O_LARGEFILE should not be used? – Findlay

The best source is the fact that grepping POSIX for it turns up zero results. – Cassock

So you mean, O_LARGEFILE is something like a cap ? It must be some reason why this flag exists. – Estell

On 32-bit systems, the version of open used with -D_FILE_OFFSET_BITS=64 passes O_LARGEFILE to the kernel transparently. The version used with 32-bit off_t does not. This in turn determines "the offset maximum established in the open file description", in the language of POSIX, which causes a number of functions to report errors when they would result in a file offset that can't be represented in the 32-bit off_t. The maximum is associated with the open file description rather than the process since open file descriptions can be shared between processes. – Cassock

In any case, there's absolutely no reason to poke at O_LARGEFILE yourself. If you're using 64-bit off_t on a 32-bit system, it will be set automatically for you, and otherwise (on 64-bit systems, or on 32-bit systems with 32-bit off_t), it should not be set. – Cassock

Does POSIX mention _FILE_OFFSET_BITS? Can you link to it? – Projective

No, it's a glibc thing. Sane systems always use 64-bit offsets. POSIX allows multiple environments and the getconf command can tell you the CFLAGS for a particular one. – Cassock

Answer 2

IIRC if you do

#define _LARGEFILE_SOURCE
#define _FILE_OFFSET_BITS 64

before all other includes you do not need to pass this flag.

additionally see

Eve

_LARGEFILE_SOURCE is not needed, only _FILE_OFFSET_BITS. – Cassock