I'm having trouble with using a library that contains weak-symbols and the --as-needed linker flag.
No, you're not.
Find out where your libjack.so is, e.g.
$ locate libjack
/usr/lib/x86_64-linux-gnu/libjack.so
/usr/lib/x86_64-linux-gnu/libjack.so.0
/usr/lib/x86_64-linux-gnu/libjack.so.0.1.0
...
Then use nm to examine the symbol types of the JACK API in libjack.so:
$ nm -C -D /usr/lib/x86_64-linux-gnu/libjack.so | grep jack_
000000000000e7e0 T jack_acquire_real_time_scheduling
000000000000d530 T jack_activate
000000000002ccf0 T jack_client_close
000000000000e820 T jack_client_create_thread
....
....
000000000000f340 T jack_uuid_empty
000000000000f320 T jack_uuid_parse
000000000000f2e0 T jack_uuid_to_index
000000000000f330 T jack_uuid_unparse
You'll find they are all type T ( = ordinary global symbol in text section: man nm). There are
a few weak symbols in the library:
$ nm -C -D /usr/lib/x86_64-linux-gnu/libjack.so | egrep ' (w|W) '
w __cxa_finalize
w __gmon_start__
w _ITM_deregisterTMCloneTable
w _ITM_registerTMCloneTable
w _Jv_RegisterClasses
0000000000025410 W std::ctype<char>::do_widen(char) const
0000000000014c10 W void std::vector<unsigned short, std::allocator<unsigned short> >::_M_emplace_back_aux<unsigned short const&>(unsigned short const&)
0000000000014b10 W std::pair<std::_Rb_tree_iterator<unsigned short>, bool> std::_Rb_tree<unsigned short, unsigned short, std::_Identity<unsigned short>, std::less<unsigned short>, std::allocator<unsigned short> >::_M_insert_unique<unsigned short>(unsigned short&&)
0000000000014ad0 W std::_Rb_tree<unsigned short, unsigned short, std::_Identity<unsigned short>, std::less<unsigned short>, std::allocator<unsigned short> >::_M_erase(std::_Rb_tree_node<unsigned short>*)
But none of them are in the JACK API. Nothing you can do short of rebuilding your libjack.so is going to change that.
The right way to characterise your problem is:
I'm having trouble linking a library with the --as-needed linker flag to a program in which
I decided to weaken all my references to that library
The defining symbol references of the JACK API in libjack.so are all strong. You have
written a program directing the compiler to emit, in your object code, symbols that are
weak undefined references to the JACK API, and you are finding that, with as-needed linkage,
these weak references fail to compel linkage of libjack.so to provide their missing definitions.
it seems that the problem is:
jack declares all symbols as weak (if I include ).
when linking with --as-needed, the linker excludes any library, that does not reference at least one non-weak symbol.
some OSs (e.g. Ubuntu-16.04LTS) have --as-needed enabled by default.
The last two points are correct. The schism between distros that link shared libraries
as-needed by default and distros that don't goes back to Debian Wheezy, 2013,
which went over to as-needed.
Since then, the Debian-derived distro-clan has followed suit while the RedHat/Fedora
clan has stuck with the status quo ante.
The first point is confused. libjack.so, as we've noted, exports a strongly defined
JACK API that you cannot alter by writing and compiling new code.
If you include <jack/weakjack.h> in one of your source files, then you are
declaring all JACK API symbols weak, in your code, and the compiler will
give you an object file that contains only weak references to the JACK API. <jack/weakjack.h>
just defines macros that have that effect.
It would be surprising if an old and major linux library like libjack
has botched its adaptation to the as-needed schism. I suspect you've overlooked some of
the small print about jack/weakjack.h:
Detailed Description
One challenge faced by developers is that of taking advantage of new features
introduced in new versions of [ JACK ] while still supporting older versions of
the system. Normally, if an application uses a new feature in a library/API,
it is unable to run on earlier versions of the library/API that do not support
that feature. Such applications would either fail to launch or crash when an
attempt to use the feature was made. This problem cane be solved using
weakly-linked symbols.
...
A concrete example will help. Suppose that someone uses a version of a JACK
client we'll call "Jill". Jill was linked against a version of JACK that contains
a newer part of the API (say, jack_set_latency_callback()) and would like to use
it if it is available.
When Jill is run on a system that has a suitably "new" version of JACK, this
function will be available entirely normally. But if Jill is run on a system
with an old version of JACK, the function isn't available.
With normal symbol linkage, this would create a startup error whenever someone
tries to run Jill with the "old" version of JACK. However, functions added to
JACK after version 0.116.2 are all declared to have "weak" linkage which means
that their abscence doesn't cause an error during program startup. Instead, Jill
can test whether or not the symbol jack_set_latency_callback is null or not.
If its null, it means that the JACK installed on this machine is too old to
support this function. If its not null, then Jill can use it just like any other
function in the API. For example:
if (jack_set_latency_callback) {
jack_set_latency_callback (jill_client, jill_latency_callback, arg);
}
However, there are clients that may want to use this approach to parts of the
JACK API that predate 0.116.2. For example, they might want to see if even
really old basic parts of the API like jack_client_open() exist at runtime.
Such clients should include <jack/weakjack.h> before any other JACK header.
This will make the entire JACK API be subject to weak linkage, so that any and
all functions can be checked for existence at runtime. It is important to
understand that very few clients need to do this - if you use this feature you
should have a clear reason to do so.
[emphasis added]
This makes clear that a program, like yours, that takes the exceptional step of of including jack/weakjack.h for the purpose of weakening its references to the entire JACK API can be expected to run successfully only if it tests the definedness of every JACK API symbol before referencing it and handles the case in which it is not defined. Your program does not conform. This one does:
myjack1.c
#include <jack/weakjack.h>
#include <jack/jack.h>
#include <stdio.h>
int main() {
if (jack_client_open) {
jack_client_open("foobar", JackNoStartServer, 0, 0);
} else {
puts("`jack_client_open` is not available");
}
return 0;
}
and do does this one:
myjack2.c
#include <jack/weakjack.h>
#include <jack/jack.h>
#include <dlfcn.h>
#include <stdlib.h>
#include <stdio.h>
int main() {
jack_client_t * (*jack_client_open_fp)
(const char *, jack_options_t,jack_status_t *,...) = jack_client_open;
if (!jack_client_open_fp) {
void * dsoh = dlopen("libjack.so",RTLD_LAZY);
if (!dsoh) {
fputs("`libjack` is not available\n",stderr);
exit(EXIT_FAILURE);
}
*(void**)(&jack_client_open_fp) = dlsym(dsoh,"jack_client_open");
if (!jack_client_open_fp) {
fputs("`jack_client_open` is not available\n",stderr);
exit(EXIT_FAILURE);
}
}
jack_client_open_fp("foobar", JackNoStartServer, 0, 0);
exit(EXIT_SUCCESS);
}
which sketches the usual approach to a discoverable API - apt
for a program meant to install and run on a system that
might not provide libjack at all. So you'd build it without reference to libjack
like:
gcc -o myjack2 myjack2.c -ldl
and on Ubuntu 17.04 - which does provide libjack - it might run like:
$ ./myjack2
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for 4294967295, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for 4294967295, skipping unlock
So the library's T&Cs are in good order with respect to as-needed linkage. That
seems to leave you in a position of being independently dissatisfied that as-needed linkage works
the way it does, rather than in a different way that would allow you to weaken
all your references to the JACK API and still get libjack to be needed by your weak references to its
API symbols:-
I fail to see why a weak dependency is considered as no dependency at all. For me,
a weak dependency is to enable optional features. I do want these features to be
enabled if possible, and the decision whether this is possible should be a
runtime decision. With the current behavior, it becomes a compile-time decision.
Your view that a weak symbol reference gives rise to a weak linkage dependency on a library
that defines the symbol does not have a footing for the GNU linker. A program
depends on a library if its linkage needs a symbol definition that the libary provides; otherwise
it doesn't depend on that libary: there aren't weak and strong degrees of dependency. (The Darwin Mach-O linker does support a cognate distinction)
There are weak symbols, as opposed to the default and usual kind,
which is strong. {weak|strong} symbol is shorthand for {weakly|strongly} referenced
symbol, since the same symbol may be referenced in multiple linker input files,
sometimes or always weakly and sometimes or always strongly.
A strong symbol must have exactly one defining reference in the linkage.
A weak symbol is such that:
The linker is not obliged to find a definition for it: it may remain undefined in the output file
The linker is not obliged to fault multiple weak definitions of the same symbol
in different input files. If exactly one defining reference within the linkage is
strong, then that strong definition is picked and all weak ones ignored. If all
defining references in the linkage are weak then the linker will pick one at random.
From the first part of that it follows that an undefined weak reference to
a symbol does not give rise to a linkage dependency at all. A definition is
not needed and the fact that a definition is not needed is the result of a
decision by the programmer (e.g. #include <jack/weak_jack.h>) or perhaps by the
compiler. It is not reasonable to expect that the linker, if directed to link
only shared libraries that are needed, should then link libraries to furnish definitions
of symbols for which you or the compiler have told it that definitions are not needed.
If the linker were to behave like that in your case, that would constitute
a linktime decision to freeze and enable an API which, by including jack/weak_jack.h,
you have indicated you wish to reserve entirely for runtime discovery.
Linking your problem program with -no-as-needed is successful as a way of
smothering the bug in the program. The bug is that by including jack/weak_jack.h
you commit yourself to runtime discovery of the whole API, but don't fulfil that
committment and instead take the availability of the API for granted. Hence the
segfault with as-needed linkage. Linking with -no-as-needed just cancels the
the effect of including jack/weak_jack.h. Including it says your program doesn't
need any of the API definitions: -no-as-needed says, whatever they are, you're getting
them all anyway.
In the light of the fact that all JACK APIs post version 0.116.2 are weakly
defined without resort to jack/weak_jack.h, I think that you simply don't
have any use for this header unless you are indeed planning a program that
will do something worthwhile on a host from which libjack is missing. If you
are planning that, then you have no alternative to runtime discovery of all
the JACK APIs you use, regardless of linkage conventions, because you can't link
libjack anyway.
If not, then just link libjack and, if you merely call jack_client_open,
your program, on any host, will dynamically link all the API definitions, whatever
they are on that host, because your reference to jack_client_open (in the
absence of <jack/weak_jack.h>) will make libjack needed, whether that
matters to the linker that did the linking or not. If you want to be compatible
accross API versions, then you need to implement runtime detection
as documented
of any API that is documented with the attribute JACK_WEAK_EXPORT
- as opposed to JACK_OPTIONAL_WEAK_EXPORT, or JACK_OPTIONAL_WEAK_DEPRECATED_EXPORT: the latter denote fundamental APIs that
can only be forcibly weakened via <jack/weak_jack.h>.
NULL(for me, "the linker is not obliged to find a definition for it" is really a promise of best-effort, rather than "go with the simplest solution"). – Postrider