ftell at a position past 2GB

Question

On a 32-bit system, what does ftell return if the current position indicator of a file opened in binary mode is past the 2GB point? In the C99 standard, is this undefined behavior since ftell must return a long int (maximum value being 2**31-1)?

Steamship

If, as you said, it's undefined behavior, there's no way to (safely or reliably) tell. It could just return the right number, -1, 0, a random number, it could segfault, or it could send in a swat team. – Pavlov

Answer 1

on long int

long int is supposed to be AT LEAST 32-bits, but C99 standard does NOT limit it to 32-bit. C99 standard does provide convenience types like int16_t & int32_t etc that map to correct bit sizes for a target platform.

on ftell/fseek

ftell() and fseek() are limited to 32 bits (including sign bit) on the vast majority of 32-bit architecture systems. So when there is large file support you run into this 2GB issue.

POSIX.1-2001 and SysV functions for fseek and ftell are fseeko and ftello because they use off_t as the parameter for the offset.

you do need to define compile with -D_FILE_OFFSET_BITS=64 or define it somewhere before including stdio.h to ensure that off_t is 64-bits.

Read about this at the cert.org secure coding guide.

On confusion about ftell and size of long int

C99 says long int must be at least 32-bits it does NOT say that it cannot be bigger

try the following on x86_64 architecture:

#include <stdio.h>

int main(int argc, char *argv[]) {
    FILE *fp;
    fp = fopen( "test.out", "w");
    if ( !fp ) 
        return -1;
    fseek(fp, (1L << 34), SEEK_SET);
    fprintf(fp, "\nhello world\n");
    fclose(fp);
    return 0;
}

Notice that 1L is just a long, this will produce a file that's 17GB and sticks a "\nhello world\n" to the end of it. Which you can verify is there by trivially using tail -n1 test.out or explicitly using:

dd if=test.out skip=$((1 << 25))

Note that dd typically uses block size of (1 << 9) so 34 - 9 = 25 will dump out '\nhello world\n'

Intension

Sorry but, Did you want to mean 64 instead 34? And as the type long is signed the file won't be 17GB, but 9GB instead right? – Cumings

It doesn't work this way for all x86_64 archs. Specifically, long on 64-bit windows is still 32-bit in size. – Lionhearted

Answer 2

At least on a 32bit OS ftell() it will overflow or error or simply run into Undefined Behaviour.

To get around this you might like to use off_t ftello(FILE *stream); and #define _FILE_OFFSET_BITS 64.

Verbatim from man ftello:

The fseeko() and ftello() functions are identical to fseek(3) and ftell(3) (see fseek(3)), respectively, except that the offset argument of fseeko() and the return value of ftello() is of type off_t instead of long.

On many architectures both off_t and long are 32-bit types, but compilation with

   #define _FILE_OFFSET_BITS 64

will turn off_t into a 64-bit type.

---

Update:

According to IEEE Std 1003.1, 2013 Edition ftell() shall return -1 and set errno to EOVERFLOW in such cases:

EOVERFLOW

For ftell(), the current file offset cannot be represented correctly in an object of type long.

Concur

I've had some persons say ftell always returns 0 in such cases, but an overflow seems to imply the possibility of a negative return. Are overflows guaranteed to "wrap around" in the C99 standard, or is this determined by the implementation? – Steamship

@VilhelmGray: I doubt it will return 0, I see no reson for this. Propably the whole thing will run into UB? Just avoid it! However UB could lead to anything ... it also might return 42 then! ;-) – Concur

I think it does boil down to UB. In section 6.5 paragraph 5 of the C99 standard (ISO/IEC 9899:TC3): If an exceptional condition occurs during the evaluation of an expression (that is, if the result is not mathematically defined or not in the range of representable values for its type), the behavior is undefined. – Steamship

@VilhelmGray: Thanks for looking up the Standard to proove this ... :-) – Concur

However, is there anything in the standard to imply that ftell itself causes the overflow -- or is the value it returns for these cases unspecified? – Steamship

Forgive me, I meant to ask for the C99 standard, not POSIX. – Steamship

Section 7.19.9.4 paragraph 3 of the C99 standard states: On failure, the ftell function returns −1L and stores an implementation-defined positive value in errno. Unfortunately, I believe it leaves the definition of "failure" up to the implementation; so perhaps the return value is UB for 2GB? – Steamship

Answer 3

There is no 64b aware method in C99 standard. What OS/environment are you using? On windows, there is _ftelli64.

On other platforms, look at http://forums.codeguru.com/showthread.php?277234-Cannot-use-fopen()-open-file-larger-than-4-GB

Radiobroadcast

ftell is C89 and C99 standard – Intension

@AhmedMasud: I feel there is a misunderstanding here. I suspect "There is no such method in C99 ..." refers to something like _ftelli64() and not to ftell(). – Concur

@Concur i don't see a reference to _ftelli64() in the OP question so I still don't understand how your answer contributes. – Intension

@Yossarian your answer is still incorrect .. on 64 bit systems long int is 64-bit ; C99 only requires long int to be AT LEAST 32-bit it puts no limits on its maximum size, on a 64-bit system a long and therefore ftell() offset would be 64-bit – Intension

@AhmedMasud: Please excuse, but requiring a long "to be at least 32 bits long" does mean it can be exactly 32 bits long. There is no need to have it being 64 bit. Also I do agree with you that it'll be quiet strange for a 64bit OS to have long being less then 64 bit long ... ;-) However, concluding from my argumentation the answer is correct, isn't it? – Concur

let us continue this discussion in chat – Intension

On windows with visual c targeting x64, sizeof(long) == 4 (I just checked). In practice ftell is not useful for large files, which is why _ftelli64 exists. Thanks @Yossarian. – Pantaloon

Answer 4

This worked for me on Windows32/MinGW to play with a 6GB file

#define _FILE_OFFSET_BITS 64
#include<stdio.h>

int main() {
    FILE *f = fopen("largefile.zip","rb");
    fseeko64(f, 0, SEEK_END);
    off64_t size = ftello64(f);

    printf("%llu\n", size);
}

gcc readlargefile.c -c -std=C99 -o readlargefile.exe

Every single detail, the macro, the compiler option, matters.

Warmongering