Why does gcc not implicitly supply the -fPIC flag when compiling static libraries on x86_64

Question

I've had numerous problems compiling shared objects that link statically against static libraries. This problem only shows up on x84_64 platforms. When doing the same compilation work on x86_32 I do not have any problems.

Perhaps this is a OS specific GCC configuration thing, but my research indicates that its how GCC works on x86_64 platforms. Anyhow, I am using gcc 4.4.3 on Ubuntu 10.04 x86_64.

How is the problem fixed ?... Making sure all the static library dependencies are compiled with -fPIC.

Question 1: What is the difference between -fpic and -fPIC (apparently -fPIC generates more instructions on x86) ? Why is the later type more relevant in the x86_64 context ?

Question 2: My assumption is that when you link against static code you are hard-wiring the functions into your binary at link time, why does it need the level of indirection the "position independant code" machinery provides ?

Question 3: Now if x86 doesn't need -fpic / -fPIC to link shared objects against static archives why is it needed in x86_64 ?

Question 4: even if it is needed why isn't it supplied implicitly ? I thought breaking changes was supposed to be a big no-no

Donell

As for Question 1. The difference between -fpic and -fPIC is described in "How to write shared libraries" akkadia.org/drepper/dsohowto.pdf. Look for the words "Which of the two options, -fpic or -fPIC have to be used". – Hockenberry

I have skimmed parts of that paper before, and I was hoping that I could skip having to extract the answer (also the question has two parts, does the section cover both parts ? ):D – Donell

I accepted your answer :D I haven't yet had the chance to review your information. But I will look into it and put a comment on your answer later. Thank you. – Donell

Answer 1

1. See question 3544035. Also discussed here and there.
2. It depends on what use you will have for your static library. If you only want to link it into programs, it doesn't need PIC code (libtool calls that a convenience library, because you could pretty much do without it, it simply helps get your compilation process to a reasonable size, for example). Otherwise, if you intend to link shared libraries against it, you need PIC code in your static library.
3. See question 3146744 and also here
4. It bloats your code, so it's not the default. One thing to see is that, when you compile a single object file, GCC doesn't know if you're going to create a shared library out of it or not. In most of my smaller projects, I simply link together a couple of object files, and do not need PIC code, for example.

Also, my advice would be: if you need to worry about that, you're doing it wrong (or you like to learn the hard way, which is nice because you'll get more out of the experience). Compilation systems (libtool, cmake, whatever you use) should do that for you.

Union

Point 2: When linking shared code against a static library, why does shared code require the static library to be compiled with -fPIC ? I don't get it, shouldn't it be able to add any missing addressing information during compile-time linking ? – Donell

When you link a static library into a shared one, it doesn't touch object code anymore, so it needs to have been compiled the right way (with PIC) in the first place. – Serapis

the problem I am facing is with dependancies, I use cmake myself. Boost / protobuf and openssl all do not supply fPIC for x86_64 static libraries – Donell

Regarding CMake, you could either ask on their user mailing-list or here. Googl'ing it, I've found discussion of related issues, so it's probably not an off-topic on their list. – Serapis

Answer 2

A typical .a static library is just a collection of regular .o objects

Therefore, conceptually, you can normally just replace the .a with the exact same list of .o files on the command line, which do not need to be relocatable.

Consider for example this minimal runnable example:

a.c

#include "a.h"

int a(void) { return 1; }

a.h

#ifndef A_H
#define A_H

int a(void);

#endif

b.c

#include "b.h"

int b(void) { return 2; }

b.h

#ifndef B_H
#define B_H

int b(void);

#endif

main.c

#include <assert.h>
#include <stdlib.h>

#include "a.h"
#include "b.h"

int main(void) {
    assert(a() == 1);
    assert(b() == 2);
    return EXIT_SUCCESS;
}

Compile and run:

gcc -ggdb3 -std=c89 -Wall -Wextra -pedantic-errors -fPIC -c 'main.c' -o 'main.o'
gcc -ggdb3 -std=c89 -Wall -Wextra -pedantic-errors -fPIC -c 'a.c' -o 'a.o'
gcc -ggdb3 -std=c89 -Wall -Wextra -pedantic-errors -fPIC -c 'b.c' -o 'b.o'
ar rcs ab.a a.o b.o
gcc -ggdb3 -std=c89 -Wall -Wextra -pedantic-errors main.o ab.a -o maina.out
./maina.out

From this we see clearly that ar simply packs a.o and b.o into ab.a.

As a result, the following command also works:

gcc -ggdb3 -std=c89 -Wall -Wextra -pedantic-errors main.o a.o b.o -o maina.out

From this it should hopefully be clear that there is no need to make the object files inside the .a archive be position independent in general.

You could make them position independent if you wanted e.g. to link them into a shared library for example. Everything applies as before: the .a just contains them without modifying them.

This answer may also be of interest: What is the -fPIE option for position-independent executables in gcc and ld?

Tested on Ubuntu 20.04.

Carnival