What is an application binary interface (ABI)?

Question

I never clearly understood what an ABI is. Please don't point me to a Wikipedia article. If I could understand it, I wouldn't be here posting such a lengthy post.

This is my mindset about different interfaces:

A TV remote is an interface between the user and the TV. It is an existing entity, but useless (doesn't provide any functionality) by itself. All the functionality for each of those buttons on the remote is implemented in the television set.

Interface: It is an "existing entity" layer between the functionality and consumer of that functionality. An interface by itself doesn't do anything. It just invokes the functionality lying behind.

Now depending on who the user is there are different type of interfaces.

Command Line Interface (CLI) commands are the existing entities, the consumer is the user and functionality lies behind.

functionality: my software functionality which solves some purpose to which we are describing this interface.

existing entities: commands

consumer: user

Graphical User Interface(GUI) window, buttons, etc. are the existing entities, and again the consumer is the user and functionality lies behind.

functionality: my software functionality which solves some problem to which we are describing this interface.

existing entities: window, buttons etc..

consumer: user

Application Programming Interface(API) functions (or to be more correct) interfaces (in interfaced based programming) are the existing entities, consumer here is another program not a user, and again functionality lies behind this layer.

functionality: my software functionality which solves some problem to which we are describing this interface.

existing entities: functions, Interfaces (array of functions).

consumer: another program/application.

Application Binary Interface (ABI) Here is where my problem starts.

functionality: ???

existing entities: ???

consumer: ???

• I've written software in different languages and provided different kinds of interfaces (CLI, GUI, and API), but I'm not sure if I have ever provided any ABI.

Wikipedia says:

ABIs cover details such as

• data type, size, and alignment;
• the calling convention, which controls how functions' arguments are passed and return values retrieved;
• the system call numbers and how an application should make system calls to the operating system;

Other ABIs standardize details such as

• the C++ name mangling,
• exception propagation, and
• calling convention between compilers on the same platform, but do not require cross-platform compatibility.

• Who needs these details? Please don't say the OS. I know assembly programming. I know how linking & loading works. I know exactly what happens inside.

• Why did C++ name mangling come in? I thought we are talking at the binary level. Why do languages come in?

Anyway, I've downloaded the [PDF] System V Application Binary Interface Edition 4.1 (1997-03-18) to see what exactly it contains. Well, most of it didn't make any sense.

• Why does it contain two chapters (4th & 5th) to describe the ELF file format? In fact, these are the only two significant chapters of that specification. The rest of the chapters are "processor specific". Anyway, I though that it is a completely different topic. Please don't say that ELF file format specifications are the ABI. It doesn't qualify to be an interface according to the definition.

• I know, since we are talking at such a low level it must be very specific. But I'm not sure how is it "instruction set architecture (ISA)" specific?

• Where can I find Microsoft Windows' ABI?

So, these are the major queries that are bugging me.

Federate

"Please don't say, OS" Compilers need to know the ABI. Linkers need to know the ABI. The kernel needs to know the ABI in order to set up the program in RAM for it to run properly. As for C++ see below, it intentionally turns labels into gibberish because of overloading and private methods, and the linker and any other compiler need to have compatible name mangling to work with it, in other words the same ABI. – Hove

"how is it "Instruction Set Architecture(ISA)" specific?" you cannot run an x86 executable on an ARM processor, period. So the ABI used on an x86 system cannot be the same one used on ARM. – Hove

I think the question is so clear; exactly describing what is the answer format expected and yet not a single satisfactory answer that which can be accepted. – Address

You are using a very strange definition of "interface". Consider merely an interface as the point at which, or mechanism through which, two or more entities interact. An ABI is merely a description of the low-level (i.e. bit twiddling) interaction of components in a software system---allowing, for instance, code written in C to call code written in Fortran by appropriate translation of data types and selection of calling conventions. No need to get philosphical. A CLI and an API are not meaningfully in the same class of entity, and attempting to find such a framework is fruitless. – Switchman

@Address My take on the issue is that OP does indeed know what an ABI is, but doesn't realize that. The vast majority of programmers will never ever design or provide an ABI, because that is the job of OS/platform designers. – Widen

@JesperE: I do agree to your point. But probably the OP wants to know it clearly, in the format s/he sees fit, even though s/he mightn't need to provide an ABI. – Address

I think the answer should be something like Consumer: Compiler (Writer), Provider: OS (Maker), Functionality: Defining protocol on how to pass data... (am just giving examples here). – Address

@JesperE: As a matter of fact, I didn't :) – Address

I'm not at all convinced that my template is faulty. Because every where this template for interface is holding true. So, yes I want I expect ABI also to fit into this template but thats not it. IMPORTANT thing is I still don't understand. I don't know if I'm so dumb or something else but it just not getting into my head. I'm unable to realize the answers and the wiki article. – Federate

I was ignorant. Recently while working with all these things. I realized what ABI is actually is. Yeah, I do agree that my template is faulty. Its not appropriate to fit ABI into my template. Thanks @ JasperE. It just took work experience to realize your answer. – Federate

@claws, This explanation may help: stackoverflow.com/questions/3784389/… – Votary

The easiest way to think of an interface, in general, is as a contract between a framework, library, platform, etc. If you want to use that system, you are expected to adhere to the rules laid out in the interface. It is a "Rules of engagement" notice. – Mentor

Answer 1

One easy way to understand "ABI" is to compare it to "API".

You are already familiar with the concept of an API. If you want to use the features of, say, some library or your OS, you will program against an API. The API consists of data types/structures, constants, functions, etc that you can use in your code to access the functionality of that external component.

An ABI is very similar. Think of it as the compiled version of an API (or as an API on the machine-language level). When you write source code, you access the library through an API. Once the code is compiled, your application accesses the binary data in the library through the ABI. The ABI defines the structures and methods that your compiled application will use to access the external library (just like the API did), only on a lower level. Your API defines the order in which you pass arguments to a function. Your ABI defines the mechanics of how these arguments are passed (registers, stack, etc.). Your API defines which functions are part of your library. Your ABI defines how your code is stored inside the library file, so that any program using your library can locate the desired function and execute it.

ABIs are important when it comes to applications that use external libraries. Libraries are full of code and other resources, but your program has to know how to locate what it needs inside the library file. Your ABI defines how the contents of a library are stored inside the file, and your program uses the ABI to search through the file and find what it needs. If everything in your system conforms to the same ABI, then any program is able to work with any library file, no matter who created them. Linux and Windows use different ABIs, so a Windows program won't know how to access a library compiled for Linux.

Sometimes, ABI changes are unavoidable. When this happens, any programs that use that library will not work unless they are re-compiled to use the new version of the library. If the ABI changes but the API does not, then the old and new library versions are sometimes called "source compatible". This implies that while a program compiled for one library version will not work with the other, source code written for one will work for the other if re-compiled.

For this reason, developers tend to try to keep their ABI stable (to minimize disruption). Keeping an ABI stable means not changing function interfaces (return type and number, types, and order of arguments), definitions of data types or data structures, defined constants, etc. New functions and data types can be added, but existing ones must stay the same. If, for instance, your library uses 32-bit integers to indicate the offset of a function and you switch to 64-bit integers, then already-compiled code that uses that library will not be accessing that field (or any following it) correctly. Accessing data structure members gets converted into memory addresses and offsets during compilation and if the data structure changes, then these offsets will not point to what the code is expecting them to point to and the results are unpredictable at best.

An ABI isn't necessarily something you will explicitly provide unless you are doing very low-level systems design work. It isn't language-specific either, since (for example) a C application and a Pascal application can use the same ABI after they are compiled.

Edit: Regarding your question about the chapters regarding the ELF file format in the SysV ABI docs: The reason this information is included is because the ELF format defines the interface between operating system and application. When you tell the OS to run a program, it expects the program to be formatted in a certain way and (for example) expects the first section of the binary to be an ELF header containing certain information at specific memory offsets. This is how the application communicates important information about itself to the operating system. If you build a program in a non-ELF binary format (such as a.out or PE), then an OS that expects ELF-formatted applications will not be able to interpret the binary file or run the application. This is one big reason why Windows apps cannot be run directly on a Linux machine (or vice versa) without being either re-compiled or run inside some type of emulation layer that can translate from one binary format to another.

IIRC, Windows currently uses the Portable Executable (or, PE) format. There are links in the "external links" section of that Wikipedia page with more information about the PE format.

Also, regarding your note about C++ name mangling: When locating a function in a library file, the function is typically looked up by name. C++ allows you to overload function names, so name alone is not sufficient to identify a function. C++ compilers have their own ways of dealing with this internally, called name mangling. An ABI can define a standard way of encoding the name of a function so that programs built with a different language or compiler can locate what they need. When you use extern "c" in a C++ program, you're instructing the compiler to use a standardized way of recording names that's understandable by other software.

Koffler

@bta, Thanks for the great answer. Does calling convention is a kind of ABI? Thanks – Agnostic

Nice answer. Except this isn't what an ABI is. An ABI is a set of rules that determines calling convention, and rules for laying out structures. Pascal passes arguments on the stack in the reverse order from C applications, so pascal and C compilers do NOT compile to the same ABI. The respective standards for C and Pascal compilers implicitly ensure that this will be the case. C++ compilers cannot define a "standard" way to mangle names, since there is no standard way. C++ name mangling conventions were not compatible between C++ compilers when there were competing C++ compilers on Windows. – Chaliapin

Definitely also see autotools.io/libtool/version.html and fedoramagazine.org/… – Votary

@RobinDavies: On platforms where Pascal compilers would have called functions pop arguments given by their callers, C compilers would generally define means by which a programmer could indicate that particular functions should use, or should be expected to use, the same calling conventions as the Pascal compilers even though C compilers would generally by default use a convention where called functions leave on the stack anything placed there by their callers. – Cajolery

@RobinDavies determines calling convention Dealing with stdcall, pascal, cdecl over the years, your comment made instant sense. – Keepsake

Answer 2

If you know assembly and how things work at the OS-level, you are conforming to a certain ABI. The ABI govern things like how parameters are passed, where return values are placed. For many platforms there is only one ABI to choose from, and in those cases the ABI is just "how things work".

However, the ABI also govern things like how classes/objects are laid out in C++. This is necessary if you want to be able to pass object references across module boundaries or if you want to mix code compiled with different compilers.

Also, if you have an 64-bit OS which can execute 32-bit binaries, you will have different ABIs for 32- and 64-bit code.

In general, any code you link into the same executable must conform to the same ABI. If you want to communicate between code using different ABIs, you must use some form of RPC or serialization protocols.

I think you are trying too hard to squeeze in different types of interfaces into a fixed set of characteristics. For example, an interface doesn't necessarily have to be split into consumers and producers. An interface is just a convention by which two entities interact.

ABIs can be (partially) ISA-agnostic. Some aspects (such as calling conventions) depend on the ISA, while other aspects (such as C++ class layout) do not.

A well defined ABI is very important for people writing compilers. Without a well defined ABI, it would be impossible to generate interoperable code.

EDIT: Some notes to clarify:

• "Binary" in ABI does not exclude the use of strings or text. If you want to link a DLL exporting a C++ class, somewhere in it the methods and type signatures must be encoded. That's where C++ name-mangling comes in.
• The reason why you never provided an ABI is that the vast majority of programmers will never do it. ABIs are provided by the same people designing the platform (i.e. operating system), and very few programmers will ever have the privilege to design a widely-used ABI.

Widen

I'm not at all convinced that my template is faulty. Because every where this template for interface is holding true. So, yes I want I expect ABI also to fit into this template but thats not it. IMPORTANT thing is I still don't understand. I don't know if I'm so dumb or something else but it just not getting into my head. I'm unable to realize the answers and the wiki article. – Federate

@jesperE, "The ABI govern things like how parameters are passed, where return values are placed. " is refer to "cdecl,stdcall,fastcall,pascal" right? – Agnostic

Yes. The proper name is "calling convention", which is a part of the ABI. en.wikipedia.org/wiki/X86_calling_conventions – Widen

this is the correct and precise answer without the verbosity (rather noise)! – Maxie

I recommend writing a bit of assembly. This will help people understand ABI in a more tangible way. – Shackleton

Answer 3

You actually don't need an ABI at all if--

• Your program doesn't have functions, and--
• Your program is a single executable that is running alone (i.e. an embedded system) where it's literally the only thing running and it doesn't need to talk to anything else.

An oversimplified summary:

API: "Here are all the functions you may call."

ABI: "This is how to call a function."

The ABI is set of rules that compilers and linkers adhere to in order to compile your program so that will work properly. ABIs cover multiple topics:

• Arguably the biggest and most important part of an ABI is the procedure call standard sometimes known as the "calling convention". Calling conventions standardize how "functions" are translated to assembly code.
• ABIs also dictate the how the names of exposed functions in libraries should be represented so that other code can call those libraries and know what arguments should be passed. This is called "name mangling".
• ABIs also dictate what type of data types can be used, how they must be aligned, and other low-level details.

Taking a deeper look at calling convention, which I consider to be the core of an ABI:

The machine itself has no concept of "functions". When you write a function in a high-level language like c, the compiler generates a line of assembly code like _MyFunction1:. This is a label, which will eventually get resolved into an address by the assembler. This label marks the "start" of your "function" in the assembly code. In high-level code, when you "call" that function, what you're really doing is causing the CPU to jump to the address of that label and continue executing there.

In preparation for the jump, the compiler must do a bunch of important stuff. The calling convention is like a checklist that the compiler follows to do all this stuff:

• First, the compiler inserts a little bit of assembly code to save the current address, so that when your "function" is done, the CPU can jump back to the right place and continue executing.
• Next, the compiler generates assembly code to pass the arguments.
  • Some calling conventions dictate that arguments should be put on the stack (in a particular order of course).
  • Other conventions dictate that the arguments should be put in particular registers (depending on their data types of course).
  • Still other conventions dictate that a specific combination of stack and registers should be used.
• Of course, if there was anything important in those registers before, those values are now overwritten and lost forever, so some calling conventions may dictate that the compiler should save some of those registers prior to putting the arguments in them.
• Now the compiler inserts a jump instruction telling the CPU to go to that label it made previously (_MyFunction1:). At this point, you can consider the CPU to be "in" your "function".
• At the end of the function, the compiler puts some assembly code that will make the CPU write the return value in the correct place. The calling convention will dictate whether the return value should be put into a particular register (depending on its type), or on the stack.
• Now it's time for clean-up. The calling convention will dictate where the compiler places the cleanup assembly code.
  • Some conventions say that the caller must clean up the stack. This means that after the "function" is done and the CPU jumps back to where it was before, the very next code to be executed should be some very specific cleanup code.
  • Other conventions say that the some particular parts of the cleanup code should be at the end of the "function" before the jump back.

There are many different ABIs / calling conventions. Some main ones are:

• For the x86 or x86-64 CPU (32-bit environment):
  • CDECL
  • STDCALL
  • FASTCALL
  • VECTORCALL
  • THISCALL
• For the x86-64 CPU (64-bit environment):
  • SYSTEMV
  • MSNATIVE
  • VECTORCALL
• For the ARM CPU (32-bit)
  • AAPCS
• For the ARM CPU (64-bit)
  • AAPCS64

Here is a great page that actually shows the differences in the assembly generated when compiling for different ABIs.

Another thing to mention is that an ABI isn't only relevant inside your program's executable module. It's also used by the linker to make sure your program calls library functions correctly. You have multiple shared libraries running on your computer, and as long as your compiler knows what ABI they each use, it can call functions from them properly without blowing up the stack.

Your compiler understanding how to call library functions is extremely important. On a hosted platform (that is, one where an OS loads programs), your program can't even blink without making a kernel call.

Explosive

Answer 4

Linux shared library minimal runnable ABI example

In the context of shared libraries, the most important implication of "having a stable ABI" is that you don't need to recompile your programs after the library changes.

So for example:

• if you are selling a shared library, you save your users the annoyance of recompiling everything that depends on your library for every new release

• if you are selling closed source program that depends on a shared library present in the user's distribution, you could release and test less prebuilts if you are certain that ABI is stable across certain versions of the target OS.

  This is specially important in the case of the C standard library, which many many programs in your system link to.

Now I want to provide a minimal concrete runnable example of this.

main.c

#include <assert.h>
#include <stdlib.h>

#include "mylib.h"

int main(void) {
    mylib_mystruct *myobject = mylib_init(1);
    assert(myobject->old_field == 1);
    free(myobject);
    return EXIT_SUCCESS;
}

mylib.c

#include <stdlib.h>

#include "mylib.h"

mylib_mystruct* mylib_init(int old_field) {
    mylib_mystruct *myobject;
    myobject = malloc(sizeof(mylib_mystruct));
    myobject->old_field = old_field;
    return myobject;
}

mylib.h

#ifndef MYLIB_H
#define MYLIB_H

typedef struct {
    int old_field;
} mylib_mystruct;

mylib_mystruct* mylib_init(int old_field);

#endif

Compiles and runs fine with:

cc='gcc -pedantic-errors -std=c89 -Wall -Wextra'
$cc -fPIC -c -o mylib.o mylib.c
$cc -L . -shared -o libmylib.so mylib.o
$cc -L . -o main.out main.c -lmylib
LD_LIBRARY_PATH=. ./main.out

Now, suppose that for v2 of the library, we want to add a new field to mylib_mystruct called new_field.

If we added the field before old_field as in:

typedef struct {
    int new_field;
    int old_field;
} mylib_mystruct;

and rebuilt the library but not main.out, then the assert fails!

This is because the line:

myobject->old_field == 1

had generated assembly that is trying to access the very first int of the struct, which is now new_field instead of the expected old_field.

Therefore this change broke the ABI.

If, however, we add new_field after old_field:

typedef struct {
    int old_field;
    int new_field;
} mylib_mystruct;

then the old generated assembly still accesses the first int of the struct, and the program still works, because we kept the ABI stable.

Here is a fully automated version of this example on GitHub.

Another way to keep this ABI stable would have been to treat mylib_mystruct as an opaque struct, and only access its fields through method helpers. This makes it easier to keep the ABI stable, but would incur a performance overhead as we'd do more function calls.

API vs ABI

In the previous example, it is interesting to note that adding the new_field before old_field, only broke the ABI, but not the API.

What this means, is that if we had recompiled our main.c program against the library, it would have worked regardless.

We would also have broken the API however if we had changed for example the function signature:

mylib_mystruct* mylib_init(int old_field, int new_field);

since in that case, main.c would stop compiling altogether.

Semantic API vs Programming API

We can also classify API changes in a third type: semantic changes.

The semantic API, is usually a natural language description of what the API is supposed to do, usually included in the API documentation.

It is therefore possible to break the semantic API without breaking the program build itself.

For example, if we had modified

myobject->old_field = old_field;

to:

myobject->old_field = old_field + 1;

then this would have broken neither programming API, nor ABI, but main.c the semantic API would break.

There are two ways to programmatically check the contract API:

• test a bunch of corner cases. Easy to do, but you might always miss one.

• formal verification. Harder to do, but produces mathematical proof of correctness, essentially unifying documentation and tests into a "human" / machine verifiable manner! As long as there isn't a bug in your formal description of course ;-)

  This concept is closely related to the formalization of Mathematics itself: https://math.stackexchange.com/questions/53969/what-does-formal-mean/3297537#3297537

List of everything that breaks C / C++ shared library ABIs

TODO: find / create the ultimate list:

• https://github.com/lvc/abi-compliance-checker automated tool to check it
• https://community.kde.org/Policies/Binary_Compatibility_Issues_With_C%2B%2B KDE C++ ABI guidelines
• https://plan99.net/~mike/writing-shared-libraries.html

Java minimal runnable example

What is binary compatibility in Java?

Tested in Ubuntu 18.10, GCC 8.2.0.

Koerlin

what does cc='gcc...' mean? cc should be the same as gcc in most Linux systems, but I don't get what ='gcc some_flags...' part means. – Rusk

@starriet it defines the bash shell variable cc to gcc .... Then I use gcc with those flags multiple times. It is just to not repeat the flags. – Koerlin

adding a new field will break ABI, you don't see it in your example because in your code the main is too small using only 1 library, had used a few others would have shown the violation – Ritz

@codeswimmer can you provide an example? – Koerlin

@CiroSantilliOurBigBook.com just reading a book books.google.com.au/… page 10 – Ritz

Answer 5

An application binary interface (ABI) is similar to an API, but the function is not accessible to the caller at source code level. Only a binary representation is accessible/available.

ABIs may be defined at the processor-architecture level or at the OS level. The ABIs are standards to be followed by the code-generator phase of the compiler. The standard is fixed either by the OS or by the processor.

Functionality: Define the mechanism/standard to make function calls independent of the implementation language or a specific compiler/linker/toolchain. Provide the mechanism which allows JNI, or a Python-C interface, etc.

Existing entities: Functions in machine code form.

Consumer: Another function (including one in another language, compiled by another compiler, or linked by another linker).

Rig

Why would the ABI be defined by the architecture? Why would different OSes on the same architecture not be able to define different ABIs? – Mechanical

Answer 6

Let me at least answer a part of your question. With an example of how the Linux ABI affects the systemcalls, and why that is usefull.

A systemcall is a way for a userspace program to ask the kernelspace for something. It works by putting the numeric code for the call and the argument in a certain register and triggering an interrupt. Than a switch occurs to kernelspace and the kernel looks up the numeric code and the argument, handles the request, puts the result back into a register and triggers a switch back to userspace. This is needed for example when the application wants to allocate memory or open a file (syscalls "brk" and "open").

Now the syscalls have short names "brk", etc. and corresponding opcodes, these are defined in a system specific header file. As long as these opcodes stay the same you can run the same compiled userland programs with different updated kernels without having to recompile. So you have an interface used by precompiled binarys, hence ABI.

Olnton

Answer 7

Functionality: A set of contracts which affect the compiler, assembly writers, the linker, and the operating system. The contracts specify how functions are laid out, where parameters are passed, how parameters are passed, how function returns work. These are generally specific to a (processor architecture, operating system) tuple.

Existing entities: parameter layout, function semantics, register allocation. For instance, the ARM architectures has numerous ABIs (APCS, EABI, GNU-EABI, never mind a bunch of historical cases) - using the a mixed ABI will result in your code simply not working when calling across boundaries.

Consumer: The compiler, assembly writers, operating system, CPU specific architecture.

Who needs these details? The compiler, assembly writers, linkers which do code generation (or alignment requirements), operating system (interrupt handling, syscall interface). If you did assembly programming, you were conforming to an ABI!

C++ name mangling is a special case - its a linker and dynamic linker centered issue - if name mangling is not standardized, then dynamic linking will not work. Henceforth, the C++ ABI is called just that, the C++ ABI. It is not a linker level issue, but instead a code generation issue. Once you have a C++ binary, it is not possible to make it compatible with another C++ ABI (name mangling, exception handling) without recompiling from source.

ELF is a file format for the use of a loader and dynamic linker. ELF is a container format for binary code and data, and as such specifies the ABI of a piece of code. I would not consider ELF to be an ABI in the strict sense, as PE executables are not an ABI.

All ABIs are instruction set specific. An ARM ABI will not make sense on an MSP430 or x86_64 processor.

Windows has several ABIs - for instance, fastcall and stdcall are two common use ABIs. The syscall ABI is different again.

Beaming

Answer 8

Summary

There are various interpretation and strong opinions of the exact layer that define an ABI (application binary interface).

In my view an ABI is a subjective convention of what is considered a given/platform for a specific API. The ABI is the "rest" of conventions that "will not change" for a specific API or that will be addressed by the runtime environment: executors, tools, linkers, compilers, jvm, and OS.

Defining an Interface: ABI, API

If you want to use a library like joda-time you must declare a dependency on joda-time-<major>.<minor>.<patch>.jar. The library follows best practices and use Semantic Versioning. This defines the API compatibility at three levels:

1. Patch - You don't need to change at all your code. The library just fixes some bugs.
2. Minor - You don't need to change your code since things where addded (open closed principle was respected)
3. Major - The interface (API) is changed and you might need to change your code.

In order for you to use a new major release of the same library a lot of other conventions are still to be respected:

• The binary language used for the libraries (in Java cases the JVM target version that defines the Java bytecode)
• Calling conventions
• JVM conventions
• Linking conventions
• Runtime conventions All these are defined and managed by the tools we use.

Examples

Java case study

For example, Java standardized all these conventions, not in a tool, but in a formal JVM specification. The specification allowed other vendors to provide a different set of tools that can output compatible libraries.

Java provides two other interesting case studies for ABI: Scala versions and Dalvik virtual machine.

Dalvik virtual machine broke the ABI

The Dalvik VM needs a different type of bytecode than the Java bytecode. The Dalvik libraries are obtained by converting the Java bytecode (with same API) for Dalvik. In this way you can get two versions of the same API: defined by the original joda-time-1.7.2.jar. We could call it joda-time-1.7.2.jar and joda-time-1.7.2-dalvik.jar. They use a different ABI one is for the stack-oriented standard Java vms: Oracle's one, IBM's one, open Java or any other; and the second ABI is the one around Dalvik.

Scala successive releases are incompatible

Scala doesn't have binary compatibility between minor Scala versions: 2.X . For this reason the same API "io.reactivex" %% "rxscala" % "0.26.5" has three versions (in the future more): for Scala 2.10, 2.11 and 2.12. What is changed? I don't know for now, but the binaries are not compatible. Probably the latest versions adds things that make the libraries unusable on the old virtual machines, probably things related to linking/naming/parameter conventions.

Java successive releases are incompatible

Java has problems with the major releases of the JVM too: 4,5,6,7,8,9. They offer only backward compatibility. Jvm9 knows how to run code compiled/targeted (javac's -target option) for all other versions, while JVM 4 doesn't know how to run code targeted for JVM 5. All these while you have one joda-library. This incompatibility flies bellow the radar thanks to different solutions:

1. Semantic versioning: when libraries target higher JVM they usually change the major version.
2. Use JVM 4 as the ABI, and you're safe.
3. Java 9 adds a specification on how you can include bytecode for specific targeted JVM in the same library.

Why did I start with the API definition?

API and ABI are just conventions on how you define compatibility. The lower layers are generic in respect of a plethora of high level semantics. That's why it's easy to make some conventions. The first kind of conventions are about memory alignment, byte encoding, calling conventions, big and little endian encodings, etc. On top of them you get the executable conventions like others described, linking conventions, intermediate byte code like the one used by Java or LLVM IR used by GCC. Third you get conventions on how to find libraries, how to load them (see Java classloaders). As you go higher and higher in concepts you have new conventions that you consider as a given. That's why they didn't made it to the semantic versioning. They are implicit or collapsed in the major version. We could amend semantic versioning with <major>-<minor>-<patch>-<platform/ABI>. This is what is actually happening already: platform is already a rpm, dll, jar (JVM bytecode), war(jvm+web server), apk, 2.11 (specific Scala version) and so on. When you say APK you already talk about a specific ABI part of your API.

API can be ported to different ABI

The top level of an abstraction (the sources written against the highest API can be recompiled/ported to any other lower level abstraction.

Let's say I have some sources for rxscala. If the Scala tools are changed I can recompile them to that. If the JVM changes I could have automatic conversions from the old machine to the new one without bothering with the high level concepts. While porting might be difficult will help any other client. If a new operating system is created using a totally different assembler code a translator can be created.

APIs ported across languages

There are APIs that are ported in multiple languages like reactive streams. In general they define mappings to specific languages/platforms. I would argue that the API is the master specification formally defined in human language or even a specific programming language. All the other "mappings" are ABI in a sense, else more API than the usual ABI. The same is happening with the REST interfaces.

Alluvion

Answer 9

In order to call code in shared libraries, or call code between compilation units, the object file needs to contain labels for the calls. C++ mangles the names of method labels in order to enforce data hiding and allow for overloaded methods. That is why you cannot mix files from different C++ compilers unless they explicitly support the same ABI.

Hove

Answer 10

The best way to differentiate between ABI and API is to know why and what is it used for:

For x86-64 there is generally one ABI (and for x86 32-bit there is another set):

http://www.x86-64.org/documentation/abi.pdf

https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/LowLevelABI/140-x86-64_Function_Calling_Conventions/x86_64.html

http://people.freebsd.org/~obrien/amd64-elf-abi.pdf

Linux + FreeBSD + MacOSX follow it with some slight variations. And Windows x64 have its own ABI:

http://eli.thegreenplace.net/2011/09/06/stack-frame-layout-on-x86-64/

Knowing the ABI and assuming other compiler follows it as well, then the binaries theoretically know how to call each other (libraries API in particular) and pass parameters over the stack or by registers etc. Or what registers will be changed upon calling the functions etc. Essentially these knowledge will help software to integrate with one another. Knowing the order of the registers / stack layout I can easily piece together different software written in assemblies together without much problem.

But API are different:

It is a high level functions names, with argument defined, such that if different software pieces build using these API, MAY be able to call into one another. But an additional requirement of SAME ABI must be adhered to.

For example, Windows used to be POSIX API compliant:

https://en.wikipedia.org/wiki/Windows_Services_for_UNIX

https://en.wikipedia.org/wiki/POSIX

And Linux is POSIX compliant as well. But the binaries cannot be just moved over and run immediately. But because they used the same NAMES in the POSIX compliant API, you can take the same software in C, recompile it in the different OS, and immediately get it running.

API are meant to ease integration of software - pre-compilation stage. So after compilation the software can look totally different - if the ABI are different.

ABI are meant to define exact integration of software at the binary / assembly level.

Frannie

The Windows x86-64 calling convention doesn't use the SysV calling convention that all other x86-64 OSes use. Linux/OS X/FreeBSD all share the same calling convention, but they don't share the full ABI. An OS's ABI includes system-call numbers. e.g. freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/… says that SYS_execve is 11 on 32bit linux, but 59 on FreeBSD. – Xerophagy

thanks for your comment, i have modified my comment to better answer the difference between ABI and API. – Frannie

You're still missing the difference between a calling convention and a full ABI (system calls and everything). You can run some FreeBSD binaries on Linux, because Linux (the kernel) provides a FreeBSD compatibility layer. Even then, this is limited to binaries that don't try to use any part of the FreeBSD ABI that Linux doesn't provide. (e.g. any FreeBSD-only system call). ABI-compatible means you can run the same binary on both systems, not just that they'd compile similarly. – Xerophagy

"FreeBSD compatibility layer", never heard of that. Can you point to the relevant linux kernel source code? But the reverse do exists: freebsd.org/doc/en_US.ISO8859-1/books/handbook/linuxemu.html. – Frannie

It's not something I use. I thought something like that existed, but maybe it doesn't anymore. tldp.org/HOWTO/Linux+FreeBSD-6.html says it's unmaintained, and that howto is from 2000. xD. unix.stackexchange.com/questions/172038/… confirms that it was abandoned and never re-done (since nobody wanted it badly enough to get it done). personality(2) can set PER_BSD. I think I remember seeing personality(PER_LINUX) in strace output all the time, but modern 64bit Linux binaries don't do that anymore. – Xerophagy

Also, I removed my downvote since sometimes ABI does just mean calling convention, name-mangling, and stuff like that, but not the whole system call interface to the OS. I still think this answer is bit messy and scattered, so I'm not upvoting it. IIRC, the other answers do a good job, and this one doesn't cover much if anything that isn't already covered by higher-voted answers. – Xerophagy

Answer 11

The term ABI is used to refer to two distinct but related concepts.

When talking about compilers it refers to the rules used to translate from source-level constructs to binary constructs. How big are the data types? how does the stack work? how do I pass parameters to functions? which registers should be saved by the caller vs the callee?

When talking about libraries it refers to the binary interface presented by a compiled library. This interface is the result of a number of factors including the source code of the library, the rules used by the compiler and in some cases definitions picked up from other libraries.

Changes to a library can break the ABI without breaking the API. Consider for example a library with an interface like.

void initfoo(FOO * foo)
int usefoo(FOO * foo, int bar)
void cleanupfoo(FOO * foo)

and the application programmer writes code like

int dostuffwithfoo(int bar) {
  FOO foo;
  initfoo(&foo);
  int result = usefoo(&foo,bar)
  cleanupfoo(&foo);
  return result;
}

The application programmer doesn't care about the size or layout of FOO, but the application binary ends up with a hardcoded size of foo. If the library programmer adds an extra field to foo and someone uses the new library binary with the old application binary then the library may make out of bounds memory accesses.

OTOH if the library author had designed their API like.

FOO * newfoo(void)
int usefoo(FOO * foo, int bar)
void deletefoo((FOO * foo, int bar))

and the application programmer writes code like

int dostuffwithfoo(int bar) {
  FOO * foo;
  foo = newfoo();
  int result = usefoo(foo,bar)
  deletefoo(foo);
  return result;
}

Then the application binary does not need to know anything about the structure of FOO, that can all be hidden inside the library. The price you pay for that though is that heap operations are involved.

Hesitate

Answer 12

The ABI needs to be consistent between caller and callee to be certain that the call succeeds. Stack use, register use, end-of-routine stack pop. All these are the most important parts of the ABI.

Mosenthal

Answer 13

Application Binary Interface(ABI)

ABI - Application Binary Interface is about a machine code communication in runtime between two binary parts like - application, library, OS... ABI describes how objects are saved in memory, how functions are called(calling convention), mangling...

A good example of API and ABI is iOS ecosystem with Swift language from v5.

• Application layer - When you create an application using different languages. For example you can create application using Swift and Objective-C^{[Mixing Swift and Objective-C]}

• Application - OS layer - runtime - Swift Standard Library and Swift Run Time Library^[About] are parts of OS and they should not be included into each bundle(e.g. app, framework). It is the same as like Objective-C uses. Available from iOS v12.2

• Library layer - Module Stability case - compile time - you will be able to import a framework which was built with another version of Swift's compiler. It means that it is safety to create a closed-source(pre-build) binary which will be consumed by a different version of compiler( .swiftinterface is used with .swiftmodule^[About]) and you will not get

  Module compiled with _ cannot be imported by the _ compiler
  //or
  Compiled module was created by a newer version of the compiler
  

• Library layer - Library Evolution case

  1. Compile time - if a dependency was changed, a client has not to be recompiled.
  2. Runtime - a system library or a dynamic framework can be hot-swapped by a new one.

[API vs ABI]
[Swift Module Stability and Library Stability]

Annotate

Answer 14

Application binary interface (ABI)

Functionality:

• Translation from the programmer's model to the underlying system's domain data type, size, alignment, the calling convention, which controls how functions' arguments are passed and return values retrieved; the system call numbers and how an application should make system calls to the operating system; the high-level language compilers' name mangling scheme, exception propagation, and calling convention between compilers on the same platform, but do not require cross-platform compatibility...

Existing entities:

• Logical blocks that directly participate in program's execution: ALU, general purpose registers, registers for memory/ I/O mapping of I/O, etc...

consumer:

• Language processors linker, assembler...

These are needed by whoever has to ensure that build tool-chains work as a whole. If you write one module in assembly language, another in Python, and instead of your own boot-loader want to use an operating system, then your "application" modules are working across "binary" boundaries and require agreement of such "interface".

C++ name mangling because object files from different high-level languages might be required to be linked in your application. Consider using GCC standard library making system calls to Windows built with Visual C++.

ELF is one possible expectation of the linker from an object file for interpretation, though JVM might have some other idea.

For a Windows RT Store app, try searching for ARM ABI if you really wish to make some build tool-chain work together.

Gypsie

Answer 15

I was also trying to understand ABI and JesperE’s answer was very helpful.

From a very simple perspective, we may try to understand ABI by considering binary compatibility.

KDE wiki defines a library as binary compatible “if a program linked dynamically to a former version of the library continues running with newer versions of the library without the need to recompile.” For more on dynamic linking, refer Static linking vs dynamic linking

Now, let’s try to look at just the most basic aspects needed for a library to be binary compatibility (assuming there are no source code changes to the library):

1. Same/backward compatible instruction set architecture (processor instructions, register file structure, stack organization, memory access types, along with sizes, layout, and alignment of basic data types the processor can directly access)
2. Same calling conventions
3. Same name mangling convention (this might be needed if say a Fortran program needs to call some C++ library function).

Sure, there are many other details but this is mostly what the ABI also covers.

More specifically to answer your question, from the above, we can deduce:

ABI functionality: binary compatibility

existing entities: existing program/libraries/OS

consumer: libraries, OS

Hope this helps!

Defense

Answer 16

A. Plainly speaking, one common thing an ABI has with an API is that it is an interface. A reusable program exposes a stable interface (API) that can be used to reuse the program in another program.

B. However, an ABI is an interface issued for some specific processor-platform for some specific language. All compiler-vendors desiring to target that platform for that same language will have to ensure that not only compiled code in form of relocatable object codes comply with the interface to be able to link and cross-link with each other but also executables comply with it to be able to run on the platform at all. So, ABI is much broader set of specifications/standard than a typical function API. It may include some API objects to be enforced upon the language-users by the compiler. The compiler-vendor will have to include support for the same in their distributions. Needless to say, the platform vendor is the rightful authority to issue ABIs for its platform. Both compiler vendors and ABIs need to comply with the corresponding language-standard (e.g. ISO standard for C++).

C. A definition of an ABI by a platform vendor is:

"1. The specifications to which an executable must conform in order to execute in a specific execution environment. For example, the Linux ABI for the Arm Architecture.

2. A particular aspect of the specifications to which independently produced relocatable files must conform in order to be statically linkable and executable. For example, the C++ ABI for the Arm Architecture, the Run-time ABI for the Arm Architecture, the C Library ABI for the Arm Architecture."

D. For example. A generic ABI for C++ based on Itanium architecture has also been issued by a consortium. The extent to which platform-vendors's own ABIs for C++ comply with it is entirely up to the platform vendors.

E. As another example. The C++ ABI for Arm Architecture is here.

F. Having said that, under the hood, it is the ABI of a processor-architecture that will ensure that the API between one reusable program and another program that reuses it works for that processor-architecture.

G. That brings us to service-oriented components (e.g. SOAP-based web services). They too require an API to exist between a SOAP-based web service and client program (could be an app, front-end or another web service) for the client program to reuse the web service.The API is described in terms of standardized protocols like WSDL (interface description) and SOAP(message format) and is language-neutral and platform-neutral. It is not targeted to any specific processor-platform and thus it is not "binary" like ABI. A client-program on any one platform type and written in any language can remotely reuse a web service written in any other language and hosted on an entirely different processor-platform. This is made possible by the fact that both WSDL and SOAP are text-based (XML) protocols. In case of RESTful web services, the transport protocol http--also a text-based protocol-- itself acts as the API (CRUD methods).

Dominique

Answer 17

In short and in philosophy, only things of a kind can get along well, and the ABI could be seen as the kind of which software stuff work together.

Gera