Detect and use optional external C library at runtime in Objective-C

Question

I am building an SDK that iPhone developers can include in their projects. It is delivered as a compiled ".a", without source code. Let's call my SDK "AAA".

The customer in his project (let's call it "BBB"), in addition to use AAA, may also use a 3rd party library called "CCC" - which also comes pre-compiled, closed-source. I do not sell CCC, it's a different company.

My SDK, AAA, can optionally use CCC to improve the product, using those 3rd party features. For example, let's say CCC is a security SDK to encrypt something. AAA does not require CCC, but will be more secured if the customer chooses to include CCC in their project as well.

Now here is an extra tricky part - the CCC library, is pure C code, made of C Structs and C functions - nothing object-oriented about it.

The issues are:

• How can I compile my AAA SDK to use functions/structs from CCC, without including CCC in my project (not legally allowed to, and don't want to keep up to date with version updates).
• How can I detect if the customer has CCC in his project, to use those extra features only if available?

Sturgeon

Usually library defines some flag notifying everyone about it's presents. Have a look, maybe CCC does? Then you can use #ifdef/#elif/#endif preprocessor commands – Footboard

this may be helpful. – Footboard

@mas'an I thought about this, and they do define a #def - but #ifdef are COMPILE-TIME features. I am delivering pre-compiled code to the customer... – Sturgeon

Could you create a third library only to wrap the CCC library? – Davao

Answer 1

Use dlsym to get the C function pointers by function name. If it can find them, they're there. Otherwise they're not. Just use RTLD_DEFAULT as the first parameter.

EDIT: having cast around for an iOS example, see Mike Ash's write up of PLWeakCompatibility, particularly the section on 'Falling Through'. You'll see he checks whether objc_loadWeakRetained (a runtime call related to weak references) is present. Under 5+ it is and his version calls the real one directly. Under 4 it isn't so his version does something else instead.

EDIT2: sample code:

Sample 1:

#import <Foundation/Foundation.h>
#include <dlfcn.h>

int main(int argc, char *argv[]) 
{
    @autoreleasepool
    {
        NSLog(@"%p", dlsym(RTLD_DEFAULT, "someFunc"));
    }
}

Outputs 0x0. Sample 2:

#import <Foundation/Foundation.h>
#include <dlfcn.h>

void someFunc()
{

}

int main(int argc, char *argv[])
{
    @autoreleasepool
    {
        NSLog(@"%p", dlsym(RTLD_DEFAULT, "someFunc"));
    }
}

Outputs an address other than 0x0.

Sample 3:

#import <Foundation/Foundation.h>
#include <dlfcn.h>

void someFunc()
{
    NSLog(@"Hi!");
}

int main(int argc, char *argv[])
{
    @autoreleasepool
    {
        void (* func)();
        func = dlsym(RTLD_DEFAULT, "someFunc");
        func();
    }
}

Outputs Hi!.

Structs have no presence in a .a or elsewhere at runtime, they're just instructions to the compiler on how to format data. So you'll need to include either the actual structs or a compatible restatement of them in your code.

Diarthrosis

Afaik dlsym doesn't work with functions included in a static library. I tried and the return value is always null. If the function is in a dynamic library or in the same project it works correctly – Aha

Did you ensure dead code stripping was disabled? I guess the quickest way to check: if you include a standard call to a library function, does dlsym then find it? See #16295342 for how to disable in general. A .a is just an archive of .os, there's no special linkage rules. – Diarthrosis

If I include a static library in a project, and then search for a function of the static library from the main project through dlsym, it works (turning off dead code stripping). But, if I call (from the main project) a function in the library A, that checks through dlsym the presence of a function in the library B (the situation of the OP), then it doesn't work. I'm trying to understand why... – Aha

My bad, it was a stupid problem...it works even in this case :-) good solution. – Aha

Is dlfcn.h an external library to download or is that bundled with the iOS SDK? – Sturgeon

It's part of the SDK; it's within the mechanism that iOS uses to load binaries. – Diarthrosis

Is dlsym expensive or trivial? OK to execute often? – Sturgeon

regarding the declaration void (* func)(); - Does it matter if my real function has parameters and return types? Does the placeholder need to mimic the real function? Or just void is enough? – Sturgeon

It all seems to work, except as mentioned, as soon as I remove all references to the real CCC library, XCode drops it. It means I need to tell customers to "force" load CCC into their project? – Sturgeon

No, they just need to disable dead code stripping — otherwise the linker will omit to include those functions it doesn't think are in use. So get them to add that linker flag when importing your library; see the link for flag specifics. Also, yes, the function pointer should nominate the correct arguments and return type; with the cast it's a good place for a typedef. – Diarthrosis

@Diarthrosis so no issues with disabling dead code stripping? See my follow up question here: #23405686 – Sturgeon

Nothing other than the obvious: usually the compiler is right when it thinks code isn't called; telling it not to act on that because you know it's going to be wrong in important cases can produce a larger binary overall. – Diarthrosis

Answer 2

You can do it using weak functions. In your static library, declare all function of ccc that you want to use like this:

int cccfunction(void) __attribute__((weak));

Don't include ccc in you lib. Since the functions are declared as weak, the compiler wouldn't complain about their absence, however you will be able to reference it in you code. Then, when you distribute the library to your users, give them a .c file with empty ccc functions inside, returning 0/null. This is necessary when the ccc lib is not available.
The user must delete this file if the CCC library is imported.

LOOK at this project

execute IOSLibraries and look at the log. At the first execution, you will see in the log

CCC not found   <--- this line is printed by libstatic (your library)

if you go in the optional.c file and comment the cccfunction(), you will see in the log

Executing a function of CCC  <--- this line is printed by libccc
CCC has been found and the function has been executed  <--- this line is printed by libstatic (your library)

If you remove both the ccc lib and the optional.c file you will see

Undefined symbols for architecture xxxxxx: "_cccfunction", referenced from: _wrapper_cccfunction in libstaticfirst_universal.a(wrapper_cccfunction.o)

This is the reason why you need to ship the optional.c file, so the user compiler won't complain about not found methods. When the user has the CCC lib, he can simply delete or comment the optional.c file. In your library you will be able to test for the presence of the CCC library looking at the return value of some control functions

EDIT - old answer: after realizing that you are on iOS, the below (and first) answer became not valid. Dynamic linking works only on OSX. However, I leave the old answer for persons using OSX

OLD ANSWER
I think that

I assume that CCC is a static library (if it's dynamic it's simpler). In this case, AFAIK, you can do nothing "automagically", but a good compromise can be something like this, using Dynamic libraries

user project --include--> your static library --include--> a dynamic library --can include–-> the CCC library

create two version of the dynamic library:

• one that implements, for example, empty functions of the CCC library -> when you call the function, they returns 0/null and you know that the library is not implemented. You can even use something smarter (a simple control function)

• give to the users the source code of a second dynamic library, that they can compile simply by doing drag-drop of the CCC library inside the project, and then moving the compiled library in the right place. This is not the source code of your library (your code is compiled in the static part), but only the code of the wrapper functions that you call from your static libraries.

• your static library don't call directly functions of the CCC library, but only wrapper functions that always exists (in both the "empty dynamic library" and in the "compiled-by-user dynamic library")

By doing this, the user can replace the "empty" dynamic library with the one that include CCC. If the dynamic library is the one with the CCC linked, the final project will use the function of CCC, otherwise it won't.

Look at the attached example:

• LibTests project implements the lib libstaticlib.a and call its function "usedynamic(int)"
• libstaticlib.a implements the dynamic library libdynamic1 and call its function "firstfunction(int)"
• libdynamic1 has two different copies: one has a firstfunction() that returns the number passed, the other returns the number*2

now, open LibTests (that should be project of your user), copy the first of the two compiled dynamic libraries in /usr/local/lib/ , then execute LibTests: you will see "10" in the console. Now, change the dynamic library with the second, and you will see "20".

This is what the user have to do: you sell the library with a dynamic "empty" component. If the user has bought CCC, you give instruction and code on how to compile the dynamic component with CCC bundled with it. After the dynamic library has been built, the user has simply to switch the .dylib file

Aha

I'd like to add that if your include order is correct, you won't need to delete the file, include guards should prevent it from being compiled and linked it. – Garlicky

What about STRUCTS, is there an equivalent? – Sturgeon

structs are "simply" instructions that you give to the compiler, you have simply to include all the CCC structs that you/the-user needs in a custom header files, and you'll be able to compile. – Aha

I would give you bonus points for providing a code sample. However Tommy's answer ended up being simpler. – Sturgeon

Answer 3

This is tricky, but manageable. If you only needed Objective-C classes from CCC, this would be easier, but you specifically said you need access to structs/functions.

1. Build a proxy class around all the CCC functionality. All CCC functionality must be encapsulated into instance methods of the proxy. All CCC types must be adapted into your own types. No part of CCC can be included in the anything outside the proxy class's implementation file. I will call this class MyCCCProxy.

2. Never directly reference the MyCCCProxy class object. More on this later.

3. Build your library without linking MyCCCProxy.m

4. Build a second static library with only MyCCCProxy.

5. Customers who have CCC will need to link AAA, CCC, and CCCProxy. Customers who don't have CCC will only link AAA.

The tricky step is number 2.

Most of the time when you create an instance of a class, you use:

MyCCCProxy *aCCCProxy = [[MyCCCProxy alloc] init];

This directly references the class object for MyCCCProxy and will cause user linking issues if MyCCCProxy is not included.

Instead, if you instead write:

MyCCCProxy *aCCCProxy = [[NSClassFromString(@"MyCCCProxy") alloc] init];

This does not directly reference the class object, it loads the class object dynamically. If MyCCCProxy does not exist as a class, then NSClassFromString returns Nil (the class version of nil). [Nil alloc] returns nil. [nil init] returns nil.

MyCCCProxy *aCCCProxy = [[NSClassFromString(@"MyCCCProxy") alloc] init];
if (aCCCProxy != nil) {
    // I have access to CCC through MyCCCProxy.
}

Minervamines

Step 4 - build MyCCCProxy. Since MyCCCProxy uses CCC, how can I compile it without including CCC?? – Sturgeon

If MyCCCProxy is a static lib, then you only need the headers of CCC for the declarations of the structs, you don't need to link CCC. – Minervamines

If you want to push this out even farther, then you can provide MyCCCProxy as source only which customers will need to include in their projects. – Minervamines

Answer 4

so Here is the gist of your problem...

a static library can't be swapped by your own process... that is at link time I linked against libfoo.1.a now at run time this process can't reliably swap in the symbols for libfoo.2.a

so you need to get around this limitation.

the simplest is to use a dynamic library and the dynamic linker... but you are using iOS so you don't have access to that.

if you could run a helper you could possibly change the actual objects in the first process, but you are on iOS and that won't work...

so that leaves trying to make an object modify its own contents... which code signing won't let you do...

so that leaves building an overflow into your program and trying to get it to execute :)

actually it is much simpler than that...

1. make a buffer
2. fill it with code fragment
3. set uo stack frame (requires a little asm)
4. set up arguments for the function you plan to call
5. run buffer + offset to your method
6. profit

as a side note I wrote a little thing that demonstrates dynamic binding at run time... but you need to have a compiler etc... this strategy won't work on iOS

https://github.com/gradyplayer/cfeedback

EDIT I actually re-read your problem, and it is a much easier one that I thought you were trying to solve...

you can use whatever is #def'ed by the other headers to do conditional compilation... and if there are places where you have to include one of these structs into an object you can just typedef the struct then only use pointers to it, as long as the library has construction and destruction functions.

Audacity

#def is compile-time. As I mentioned, the libraries are shipped pre-compiled. It needs to be runtime. – Sturgeon

it doesn't because you can control what symbols you reference at your customers compile time, and include references to your bridge code iff some symbol is defined. – Audacity

Answer 5

It is not exactly runtime, but may solve your problem depending on CCC license.

Option 1 (compile time)

Create a CCC_wrap library with #ifdef, and give the instructions to compile it with and without the CCC_library.

For each CCC_function, you must have a equivalent CCC_function_wrap

If HAVE_CCC == 1 the wrapper function should call CCC library, otherwise, do nothing or return error.

Create an extra function to discover how your library was compiled

int CCC_wrap_isfake(void) {
#if HAVE_CCC
    return 0;
#else
    return 1;
#endif
}

Option 2 (binary ready)

Create two new libraries, CCC_wrap and CCC_wrap_fake

Both libraries must contain all functions/classes needed to run the program, but the fake library all functions will do nothing, just return 0;

Than you create a extra function CCC_wrap_isfake

CCC_wrap_fake:

int CCC_wrap_isfake(void) { return 1;}

CCC_wrap:

int CCC_wrap_isfake(void) { return 0;}

Now you know if your code is running with the real wrap, or the fake one.

On compile time, you need to set a flag to determine how your library will be linked to your client software

CCC_wrap_fake:

LDFLGAS=-lCCC_wrap_fake

CCC_wrap:

LDFLGAS=-lCCC_wrap -lCCC

Both options should link correctly.

About license requirement

If you deliver the CCC_wrap library source, your client will be able to update the CCC library, without having any access to you main source.

In both cases, it will not be required to you to ship the CCC library together with your source code.

Davao

Answer 6

Your problem is more easly solved at compile time since your clients are already required to link everything by themselves.

Since your client is supposed to statically link all your "AAA" code together with "CCC" code, your problem can be solved by instructing your client to compile together "AAA.a" either with "AAA_with_CCC_glue.a" if they have "CCC.a" or "AAA_without_CCC_glue.a" if they don't. Both _glue.a would implement the set of functions that potentially uses CCC.a, difference is if they actually use it.

To solve this at runtime, you'd need at least to be able to call dlsym() (this post made me think that yes, you can, but it is old). Try that to look for all CCC.a functions that you care inside your app own memory.

Zeringue