Can't build multiple Android dex files with Ant from external jars

Question

I'm developing an Android app that requires multiple libraries (for Facebook, Google Maps v2 and Quickblox among others), resulting in a method amount overflow that goes over the 64K limit:

Unable to execute dex: method ID not in [0, 0xffff]: 65536
Conversion to Dalvik format failed: Unable to execute dex: method ID not in [0, 0xffff]: 65536

As I can't do without any of those libraries, I looked for a solution for the method limit bug. I found a popular blog entry from Android Developers, where a source code division is recommended. (The blog entry I'm talking about can be found here: http://android-developers.blogspot.com.es/2011/07/custom-class-loading-in-dalvik.html). I've been trying this solution with no success.

The problem I have now is that the biggest amount of code is not in my app itself, but in the required libraries, so I have to spread those libraries among the different dex files I must load in my app. My knowledge of Ant is very limited, and what I'd like to know is what I should write in my build.xml file to make dex copy each library where I want:

            <!-- Primary dex to include my source code and some libraries. -->
            <copy todir="${out.classes.absolute.dir}.1" >
                <fileset dir="${out.classes.absolute.dir}" >

                ...

                </fileset>
            </copy>


            <!-- Secondary dex to include some other libraries. -->
            <copy todir="${out.classes.absolute.dir}.2" >
                <fileset dir="${out.classes.absolute.dir}" >

                ...  

                </fileset>
            </copy>

Any help would be truly appreciated. Thanks in advance, kind regards!

Leenaleeper

did you find a solution for this? I have the exact same problem. – Claudette

Answer 1

The best answer I have heard to this problem so far is to use ProGuard in optimized mode (proguard-android-optimize.txt) with the -dontobfuscate flag to remove unused classes and methods from the final APK (without obfuscating them). Then you can use the ProGuard mapping.txt to help you strip the unused classes out of the library JARs that you use (I'm not aware of a good tool to do this). Unfortunately I don't think there is a feature in ProGuard to do this automatically.

ProGuard only runs when doing an Export in Eclipse, not when doing a Run As -> Android Application. This means it won't help avoid the limit on debug builds, unless you use a custom build process. The creator of ProGuard suggests using its commercial sibling DexGuard, which will run in Eclipse on both debug and release builds.

Using ProGuard is strongly encouraged for release builds as it will reduce your code size, improve performance (e.g. by inlining code), and will obfuscate your source too. Make sure you do enough testing on the final APK as it will be dramatically different to the debug one. Unfortunately, ProGuard on its own wasn't enough to solve my problem, so I removed one of my JAR dependencies as a workaround.

I checked the number of methods in my dependent JARs using this command:

dx --dex --output=temp.dex library.jar
cat temp.dex | head -c 92 | tail -c 4 | hexdump -e '1/4 "%d\n"'

Examples numbers of methods for our biggest libraries:

11222 guava-11.0.1.jar    
10452 aws-android-sdk-1.5.0-core.jar    
5761 org.restlet.jar    
5129 protobuf-java-2.4.1.jar    
2499 aws-android-sdk-1.5.0-s3.jar    
2024 ormlite-core-4.41.jar    
1145 gson-2.2.2.jar    
1716 google-http-client-1.11.0-beta.jar    

FYI, you can check the number of methods in your APK using dexdump (which you can find in your SDK build-tools folder, e.g. "Android Studio.app/sdk/build-tools/21.0.2/dexdump"):

dexdump -f MyApp.apk | grep method_ids_size
method_ids_size     : 64295

In our case we were only using Guava for doing YouTube searches, so it was easy to remove that dependency and give us more breathing room.

Update: I found the easiest way to strip out unused code from large JARs was to use dex2jar on my Proguard build, then use JD-GUI to compare that final JAR with the specific input JAR I wanted to trim. Then I remove top level packages that I know get removed by Proguard. This allowed me to remove >8000 methods from aws-android-sdk-1.5.0-core.jar.

Condensable

Thanks for sharing. May I know how do you remove unwanted packages from jar file? As jar file itself is in binary form. Do you de-compile them using JD-GUI in Java files form, and re-compile them back again to Jar? – Implausible

Use the "jar" tool to unjar and re-jar. – Condensable

Answer 2

ProGuard is a source code minification tool, it will remove any unused/duplicate classes when packaging your application. So even if you're using all these external JARs, you won't be using 100% all the classes inside those, and many of these JARs could have same classes inside them (e.g. Log4J, etc).

Proguard is built into dex. But it doesn't get activated by default, you have to run ant in release mode, and you may also have to comment out a line in project.properties.

Most likely, ProGuard may end up removing some classes used in your application. If when compiling when ProGuard, you get errors like class X not found, then configure ProGuard in proguard.cfg that you need that class:

-keep public class <full path to the class that you need>

Check out the Android ProGuard documentation and see if enabling proguard reduces the size (of classes and methods) in your APK.

Martini