Too many commands? Dyld Message: malformed mach-o: load commands size

Question

Some iOS 9 devices in the wild seem to crash with the error message that I receive from the very basic crash reporting in Xcode only

dyld: malformed mach-o: load commands size (16464) > 16384

Unfortunately that's all the info I get. I can't even debug or reproduce locally. Can anyone hint me into the right direction here?

It occurs after updating my Cocoapods, so I guess there's one of them (or their dependency) that misbehaves.

After some investigation of my mach-O binary, I saw that the sizeofcmds is really 16464. If I understand correctly, there seems to be a load command size limit of 16384, can anyone confirm this?

Does that mean I should remove dylibs and everything should be fine?

Greece

Can you list the dependencies here? – Bitthia

Running into this too this week. Nasty problem. What's strange is that other mentions I've found say 32768 for the limit (twice this). – Cosmonaut

let's solve this together. run otool -l on your exported executable and look at sizeofcmds for your architectures. Also please let me know which architectures you use, I use armv7 and arm64 (for army 64 the size of command exceeds the "limit" – Greece

Yeah, I had a look at the output from otool and it matched the errror message from dyld. I've just noticed that you're talking about devices and I'm dealing with it on the simulator (running on Sierra), so my arch is x86-64. I think it's the dyld on iOS 9, though, because a 10.x device and sim work fine. Still not sure what to do about it. A colleague has been looking into it, but I don't know yet if she's found anything. – Cosmonaut

Some load commands contain data for debuggers, you could play around with minimising emitted debug stuff in the linker. A convenient tool for verifying the binary is MachOView. You can also check my post on minimal set of LCs which are really required for the binary to work here: https://mcmap.net/q/819689/-what-is-required-for-a-mach-o-executable-to-load – Heirdom

Answer 1

At WWDC18 I went to an Apple engineer who is working on dyld. Here’s what he had to say:

• The Dyld code is downloadable from https://opensource.apple.com (the one specific to us can be found inside macOS 10.12)
• For iOS 9 the maximum size of load commands is indeed 16k aka 1 memory page (There’s no way around it! This is imposed by the OS itself. For customer service telling people to update to iOS 10 (all devices that run iOS 9 can except for iPhone 4S) would be viable.)
• Since iOS 10 the maximum size of commands is 32k
• Majority of the size of the load commands is determined by strings (paths) of the frameworks (use command otool -L to see them

Possible solutions:

• Use less libraries (that was our goto solution thus far, but we will change to umbrella libraries (see below))
• Shortening names (might screw up header lookup of cocoa pods, maybe use head maps to fix that inside the Xcode build process → maybe more (high-level) info in WWDC18 session “Behind the scenes of the Xcode Build Process”)
• Try to build static archives for libraries (should not have dynamic resources otherwise make copy phases and figure out where resources are)
• Build frameworks that re-export other frameworks (umbrella frameworks). Use -reexport-l as a linker flag (not done often) → gonna make some runtime overhead when starting the app, also uses a bit more memory (man ld → for info on re-exports)

The engineer recommended to file a bugreport via bugreport.apple.com, because in the future even hitting the 32k limit is possible.

Kilroy

Answer 2

I found a solution that will (at least temporarily) work for me - but I still encourage everyone to provide a real solution and more detailed insights.

Anyway:

Extract your binary

1. Xcode Archive
2. Export as IPA
3. Rename YourApp.ipa to YourApp.zip and extract
4. Navigate to the subfolder payload to find your YourApp.app
5. Right click & Show Package Contents of your YourApp.app file
6. Copy your binary YourApp (no file extension) to a different location

Investigate your mach-o binary

1. Run otool -f on your binary

Note the align for both architectures are listed which, for me, says 2^14 (16384). This seems to be the threshold for the size of load commands.

2. Run otool -l on your binary

You'll see that the different architectures and their load commands are listed - as well as their sizeofcmds (size of commands).

Now the funny thing: For arm64, the sizeofcmds (16464) was larger than the align (16384), while it wasn't for armv7.

Now I haven't found enough documentation on this, but I assume that align symbolizes a threshold that should not be reached by the load command size. And also that it adjusts automatically (since we are definitely not having that many frameworks in our app, there have to be apps that have more).

So I guess the error came from this unlikely case, that the sizeofcmds was different in between the architectures AND that one of them was actually valid (so that the align was not automatically adjusted).

Please correct me if I'm wrong, I am just assuming here and I really really want to understand why this happens.

Solve the issue

Remove frameworks until you are under the sizeofcmds for both architectures.

I know this is not scalable, we were lucky (and stupid) that we still had one barely used framework in there that we could easily remove.

Fortunately this only seems to be an issue on iOS9 and will therefore loose relevance over the next months, nevertheless, we should find out if I'm right

Investigation ideas

My assumption that the align is automatically adjusted could be investigated by just putting in more and more frameworks to see if it actually does.

If so, adding frameworks would also solve the original issue - not nice, but at least slightly more scalable.

Sidenote

I don't feel like I shed enough light on the origins of this issue and I had a lot of assumptions. While my solution works, I really hope you feel encouraged to investigate this as well and give a better answer.

Greece

Answer 3

So here's the problem:

The Mach-O header size is expected to be 16k (optimized for the platform's pagesize). In the reference by rachit it's basically the same thing but the limit is 32K. Both are correct in that this is hard limit of dyld, the loader.

The total size of load commands exceeds this max size. Removing frameworks and libraries works for you because that removes LC_LOAD_DYLIB commands (And, there is no reason why you'd need so many frameworks anyway). Instead of removing frameworks, build your app from the ground up starting with the core frameworks, and adding so long as you get linker errors.

btw, 'Align' has nothing to do with this - Alignment refers to the fat (universal) architecture slices, and doesn't have anything to do with the Mach-O.

Dilorenzo

Thanks for your explanation. My app has a quite significant size and complexity and every framework we use is relevant. It sums up to 89 imported dyllibs or frameworks. I can't really imagine that that's too much, or is it? – Greece

Even springboard, which is more complicated than your app, has 140-something frameworks, and that fits in the header. can you provide an output of "jtool -l" on your Mach-O? – Dilorenzo

Answer 4

I was able to resolve this for my team after reviewing the result of otool -l. Turns out we had the same directory included in our framework search paths 5x causing our dylibs to be added as rpaths 5x.

Lurlenelurline