How to force linkage to older libc `fcntl` instead of `fcntl64`?

Question

It seems GLIBC 2.28 (released August 2018) made a fairly aggressive change to fcntl. The definition was changed in <fcntl.h> to no longer be an external function, but a #define to fcntl64.

The upshot is that if you compile your code on a system with this glibc--if it uses fcntl() at all--the resulting binary will not execute on a system from before August 2018. This affects quite a variety of applications...the manual page for fcntl() shows that it's the entry point for a small universe of sub-functions:

https://linux.die.net/man/2/fcntl

It would be nice if you could tell the linker what specific version of a GLIBC function you wanted. But the closest I found was this trick described in an answer to another post:

Answer to "Linking against older symbol version in a .so file"

This is a bit more complicated. fcntl is variadic without a vffcntl that takes a va_list. In such situations you cannot forward an invocation of a variadic function. :-(

When one has stable code with purposefully low dependencies, it's a let-down to build it on a current Ubuntu...then and have the executable refuse to run on another Ubuntu released only one year prior (nearly to the day). What recourse does one have for this?

Cleaves

Answer 1

What recourse does one have for this?

The fact that GLIBC didn't have a way to #define USE_FCNTL_NOT_FCNTL64 says a lot. Be it right or wrong, most OS+toolchain makers seem to have decided that targeting binaries for older versions of their systems from a newer one is not a high priority.

The path of least resistance is to keep a virtual machine around of the oldest OS+toolchain that builds your project. Use that to make binaries whenever you think that binary will be run on an old system.

But...

• If you believe your usages are in the subset of fcntl() calls that are not affected by the offset size change (which is to say you don't use byte range locks)
• OR are willing to vet your code for the offset cases to use a backwards-compatible structure definition
• AND are not scared of voodoo

...then keep reading.

The name is different, and fcntl is variadic without a vffcntl that takes a va_list. In such situations you cannot forward an invocation of a variadic function.

...then to apply the wrapping trick mentioned, you have to go line-by-line through fcntl()'s interface documentation, unpack the variadic as it would, and then call the wrapped version with a new variadic invocation.

Fortunately it's not that difficult a case (fcntl takes 0 or 1 arguments with documented types). To try saving anyone else some trouble, here's code for that. Be sure to pass --wrap=fcntl64 to the linker (-Wl,--wrap=fcntl64 if not calling ld directly):

asm (".symver fcntl64, fcntl@GLIBC_2.2.5");

extern "C" int __wrap_fcntl64(int fd, int cmd, ...)
{
    int result;
    va_list va;
    va_start(va, cmd);

    switch (cmd) {
      //
      // File descriptor flags
      //
      case F_GETFD: goto takes_void;
      case F_SETFD: goto takes_int;

      // File status flags
      //
      case F_GETFL: goto takes_void;
      case F_SETFL: goto takes_int;

      // File byte range locking, not held across fork() or clone()
      //
      case F_SETLK: goto takes_flock_ptr_INCOMPATIBLE;
      case F_SETLKW: goto takes_flock_ptr_INCOMPATIBLE;
      case F_GETLK: goto takes_flock_ptr_INCOMPATIBLE;

      // File byte range locking, held across fork()/clone() -- Not POSIX
      //
      case F_OFD_SETLK: goto takes_flock_ptr_INCOMPATIBLE;
      case F_OFD_SETLKW: goto takes_flock_ptr_INCOMPATIBLE;
      case F_OFD_GETLK: goto takes_flock_ptr_INCOMPATIBLE;

      // Managing I/O availability signals
      //
      case F_GETOWN: goto takes_void;
      case F_SETOWN: goto takes_int;
      case F_GETOWN_EX: goto takes_f_owner_ex_ptr;
      case F_SETOWN_EX: goto takes_f_owner_ex_ptr;
      case F_GETSIG: goto takes_void;
      case F_SETSIG: goto takes_int;

      // Notified when process tries to open or truncate file (Linux 2.4+)
      //
      case F_SETLEASE: goto takes_int;
      case F_GETLEASE: goto takes_void;

      // File and directory change notification
      //
      case F_NOTIFY: goto takes_int;

      // Changing pipe capacity (Linux 2.6.35+)
      //
      case F_SETPIPE_SZ: goto takes_int;
      case F_GETPIPE_SZ: goto takes_void;

      // File sealing (Linux 3.17+)
      //
      case F_ADD_SEALS: goto takes_int;
      case F_GET_SEALS: goto takes_void;

      // File read/write hints (Linux 4.13+)
      //
      case F_GET_RW_HINT: goto takes_uint64_t_ptr;
      case F_SET_RW_HINT: goto takes_uint64_t_ptr;
      case F_GET_FILE_RW_HINT: goto takes_uint64_t_ptr;
      case F_SET_FILE_RW_HINT: goto takes_uint64_t_ptr;

      default:
        fprintf(stderr, "fcntl64 workaround got unknown F_XXX constant")
    }

  takes_void:
    va_end(va);
    return fcntl64(fd, cmd);

  takes_int:
    result = fcntl64(fd, cmd, va_arg(va, int));
    va_end(va);
    return result;

  takes_flock_ptr_INCOMPATIBLE:
    //
    // !!! This is the breaking case: the size of the flock
    // structure changed to accommodate larger files.  If you
    // need this, you'll have to define a compatibility struct
    // with the older glibc and make your own entry point using it,
    // then call fcntl64() with it directly (bear in mind that has
    // been remapped to the old fcntl())
    // 
    fprintf(stderr, "fcntl64 hack can't use glibc flock directly");
    exit(1);

  takes_f_owner_ex_ptr:
    result = fcntl64(fd, cmd, va_arg(va, struct f_owner_ex*));
    va_end(va);
    return result;

  takes_uint64_t_ptr:
    result = fcntl64(fd, cmd, va_arg(va, uint64_t*));
    va_end(va);
    return result;
}

Note that depending on what version you're actually building on, you might have to #ifdef some of those flag sections out if they're unavailable.

This affects quite a variety of applications...the manual page for fcntl() shows that it's the entry point for a small universe of sub-functions

...and it should probably be a lesson to people: avoid creating such "kitchen sink" functions through variadic abuse.

Cleaves

For those who might want to discuss what motivated this post...here's a Discourse thread about the role of binary transferability in today's world – Cleaves

And what would this break? By dropping the use of fcntl64(), are there now going to be bugs introduced while accessing files larger than 2 GB? The only way to know is to do a full regression test of all fcntl() uses. keep a virtual machine around of the oldest OS+toolchain that builds your project THAT is the real answer and IMO should be up front. – Adinaadine

@AndrewHenle Reordered and edited to stop what I imagine the incompatibility is if someone actually does use locking... (I wasn't). – Cleaves

"targeting binaries for older versions of their systems from a newer one is not a high priority." -- it's not a "not high priority", it's an exlicit non goal. – Dodecagon

@HostileFork Expecting to something built in a newer version of an OS, library, or even CPU version to run on something older is fundamentally flawed. Nothing can ever be guaranteed to be forward-compatible. "Nothing will ever emerge in the future that won't run against this library/on this OS"? No one can make that promise, and you can't rely on that. – Adinaadine

@AndrewHenle "Nothing will ever emerge in the future that won't run against this library/on this OS"? No one can make that promise, and you can't rely on that." => This is patently false--in the sense that if one can use an older version of the OS/toolchain for a result, new versions of the toolchain could have a switch to get that result as well. Not long ago it would have been considered unacceptable to have a release of a compiler that could not build binaries that would run on a system considered "the latest" just the day before. I see that ship seems to have sailed for many here. – Cleaves

(cont) "[I]nstall[ing] an old OS" is a lot less "process" than the regression testing you have failed to do. For one simple function the regression testing necessary to ensure you didn't break anything is more work than just doing it right by building on an older OS. – Adinaadine

Instead of declaring the flock family incompatible, couldn't you allocate an old-style flock struct on the stack, convert it, pass in a pointer to it, then convert/copy it back to the passed-in pointer on the way out? – Hibbert

Answer 2

How to force linkage to older libc fcntl instead of fcntl64?

Compile against an older version of libc. Period.

Because glibc is not forward compatible, it is only backwards-compatible:

The GNU C Library is designed to be a backwards compatible, portable, and high performance ISO C library. It aims to follow all relevant standards including ISO C11, POSIX.1-2008, and IEEE 754-2008.

Without any guarantees of forward compatibility, you don't know what else won't work properly.

Adinaadine

To add on that, if possible just use normal package build process, normally the steps to build package in most distros include installing version of libraries used in that distro, that way if your build process runs tests you can catch any problems with it early – Hittel

This answer would be better if it included instructions on how to do that. Is it sufficient to just copy the older glibc.so to /some/random/path and tell the linker to look there first ? – Hypozeuxis

@Hypozeuxis True, but compiling and linking against a set of libraries other than the standard one(s) installed on any system would require pretty much a book of an answer to be both complete and correct. – Adinaadine