Can a Mac OS X application prevent dlopen from loading a library?

Question

After some careful debugging, I have discovered a bug in my application that is rooted in the system frameworks loading a buggy Apple bundle:

/System/Library/CoreServices/MLTEFile.bundle

I am confident that I am not benefiting from the services of this bundle, and am doubtful that many apps are. In fact I learned by studying Xcode itself that the bundle CANNOT be loaded into Xcode's memory space because the bundle is not Garbage Collection compatible.

So ideally, I'd like to prevent this bundle from being loaded into my app as well. One way about doing that would be to require GC in my own app, but I'm not about to do that. Can anybody think of a way to circumvent attempts by system frameworks to call dlopen() on this library?

Haft

I imagine Apple engineers would like to know what issues you’re having with this. Sounds super-weird. Post a link to (Open)Radar when you get a chance to file a bug! – Laritalariviere

Answer 1

Thanks to a number of useful replies on Twitter, I am able to offer the "interpose" feature of dyld as a potential solution here. Search for "Interposing Functions in Dependent Libraries":

https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/UsingDynamicLibraries.html#//apple_ref/doc/uid/TP40002182-SW10

Short example here:

http://toves.freeshell.org/interpose/

And a simplified macro for declaring interposes here:

https://opensource.apple.com/source/dyld/dyld-97.1/include/mach-o/dyld-interposing.h

I am a bit confused by the various approaches demonstrated here. It seems that there is a mechanism for advertising the _interpose desire in the DYLD name table, and a completely (?) different mechanism that relies on using dlsym(RTLD_NEXT, …) to chain through to the original. In Apple's downloadable example (from the first documentation link), they use the dlsym technique, but it's not clear to me if this implicitly causes the interposition to occur.

Haft