Custom Class Loading in Dalvik with Gradle (Android New Build System)
Asked Answered
N

3

18

As per the introduction of Custom Class Loading in Dalvik by Fred Chung on the Android Developers Blog:

The Dalvik VM provides facilities for developers to perform custom class loading. Instead of loading Dalvik executable (“dex”) files from the default location, an application can load them from alternative locations such as internal storage or over the network.

However, not many developers have the need to do custom class loading. But those who do and follow the instructions on that blog post, might have some problems mimicking the same behavior with Gradle, the new build system for Android introduced in Google I/O 2013.

How exactly one can adapt the new build system to perform the same intermediary steps as in the old (Ant based) build system?

Naturism answered 11/8, 2013 at 16:15 Comment(0)
N
27

My team and I recently reached the 64K method references in our app, which is the maximum number of supported in a dex file. To get around this limitation, we need to partition part of the program into multiple secondary dex files, and load them at runtime.

We followed the blog post mentioned in the question for the old, Ant based, build system and everything was working just fine. But we recently felt the need to move to the new build system, based on Gradle.

This answer does not intend to replace the full blog post with a complete example. Instead, it will simply explain how to use Gradle to tweak the build process and achieve the same thing. Please note that this is probably just one way of doing it and how we are currently doing it in our team. It doesn't necessarily mean it's the only way.

Our project is structured a little different and this example works as an individual Java project that will compile all the source code into .class files, assemble them into a single .dex file and to finish, package that single .dex file into a .jar file.

Let's start...

In the root build.gradle we have the following piece of code to define some defaults:

ext.androidSdkDir = System.env.ANDROID_HOME

if(androidSdkDir == null) {
    Properties localProps = new Properties()
    localProps.load(new FileInputStream(file('local.properties')))

    ext.androidSdkDir = localProps['sdk.dir']
}

ext.buildToolsVersion = '18.0.1'
ext.compileSdkVersion = 18

We need the code above because although the example is an individual Java project, we still need to use components from the Android SDK. And we will also be needing some of the other properties later on... So, on the build.gradle of the main project, we have this dependency:

dependencies {
    compile files("${androidSdkDir}/platforms/android-${compileSdkVersion}/android.jar")
}

We are also simplifying the source sets of this project, which might not be necessary for your project:

sourceSets {
    main {
        java.srcDirs = ['src']
    }
}

Next, we change the default configuration of the build-in jar task to simply include the classes.dex file instead of all .class files:

configure(jar) {
    include 'classes.dex'
}

Now we need to have new task that will actually assemble all .class files into a single .dex file. In our case, we also need to include the Protobuf library JAR into the .dex file. So I'm including that in the example here:

task dexClasses << {
    String protobufJarPath = ''

    String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''

    configurations.compile.files.find {
        if(it.name.startsWith('protobuf-java')) {
            protobufJarPath = it.path
        }
    }

    exec {
        commandLine "${androidSdkDir}/build-tools/${buildToolsVersion}/dx${cmdExt}", '--dex',
                    "--output=${buildDir}/classes/main/classes.dex",
                    "${buildDir}/classes/main", "${protobufJarPath}"
    }
}

Also, make sure you have the following import somewhere (usually at the top, of course) on your build.gradle file:

import org.apache.tools.ant.taskdefs.condition.Os

Now we must make the jar task depend on our dexClasses task, to make sure that our task is executed before the final .jar file is assembled. We do that with a simple line of code:

jar.dependsOn(dexClasses)

And we're done... Simply invoke Gradle with the usual assemble task and your final .jar file, ${buildDir}/libs/${archivesBaseName}.jar will contain a single classes.dex file (besides the MANIFEST.MF file). Just copy that into your app assets folder (you can always automate that with Gradle as we've done but that is out of scope of this question) and follow the rest of the blog post.

If you have any questions, just shout in the comments. I'll try to help to the best of my abilities.

Naturism answered 11/8, 2013 at 16:15 Comment(13)
Do you mind to have some input in #18629521 ? Thanks.Punjab
Comprehensive and very useful. Obviously, you're busy, but this would be a significant contribution as a public github repository :) Thanks for the detailed right up.Bataan
configure(jar) { include 'classes.dex' } doesn't work for me - I get an error: "A problem occurred evaluating project> Could not find property 'jar' on project."Tricostate
@Tricostate This assumes the plugin you are applying is the java one, not the android or android-plugin. IF you use one of those, you'll need to adapt the script for that.Naturism
@Ricardo, thank you. I think it would be great if you mention that you apply plugin 'java', because originally the question was about Android build system. And if I understood correctly, using java plugin in main project leads to building all dependent Android library projects manually without the help of android-library plugins, isn't it? But also it looks like Google made impossible to customize its android plugin for using dynamic dex-classes loading...Tricostate
@Tricostate I've stated in my answer that this is a Java project and it needs tweaking to fit other needs. You can probably do the same fir an android-library project, you just need to figure it out how it works and hook to the proper tasks to do what I did above.Naturism
Looks like android plugin doesn't have jar task, so it can't be customized not to include lib jar filesTricostate
@Ricardo Hi, unfortunately I've faced the same problem with dex limit. I have a little knowledge about gradle so my question is: is there posibility to use your solution to build aplication with many dependencies from maven repository (which are build as apklib, aar or jar) in an automatic way. I'd like to specify which dependencies should go to separate dex (their resources and assets should be processed normally).Sixteenth
I'm new to gradle and do not quite understand the pieces, would you please post the entire build.gradle in gist? thanks!Dulcy
@Dulcy I don't have one to share anymore. Sorry.Naturism
This doesn't explain how to split the classes into two separate classes.dex files, so it's not really an answer to the original question.Peipeiffer
@RicardoAmaral There is any performance difference in this two methods.Falconer
@RicardoAmaral I am getting Class not found expcetion in my android application after I enable multi dex, do you have any idea on it? I am facing it since very long and I don't have any clue to resolve it? I posted question on SO here - #32004370Armyworm
P
2

The Android Studio Gradle plugin now provides native multidex support, which effectively solves the Android 65k method limit without having to manually load classes from a jar file, and thus makes Fred Chung's blog obsolete for that purpose. However, loading custom classes from a jar file at runtime in Android is still useful for the purpose of extensibility (e.g. making a plugin framework for your app), so I'll address that usage scenario below:

I have created a port of the original example app on Fred Chung's blog to Android Studio on my github page over here using the Android library plugin rather than the Java plugin. Instead of trying to modify the existing dex process to split up into two modules like in the blog, I've put the code which we want to go into the jar file into its own module, and added a custom task assembleExternalJar which dexes the necessary class files after the main assemble task has finished.

Here is relevant part of the build.gradle file for the library. If your library module has any dependencies which are not in the main project then you will probably need to modify this script to add them.

apply plugin: 'com.android.library'
// ... see github project for the full build.gradle file

// Define some tasks which are used in the build process
task copyClasses(type: Copy) { // Copy the assembled *.class files for only the current namespace into a new directory
    // get directory for current namespace (PLUGIN_NAMESPACE = 'com.example.toastlib')
    def namespacePath = PLUGIN_NAMESPACE.replaceAll("\\.","/")
    // set source and destination directories
    from "build/intermediates/classes/release/${namespacePath}/"
    into "build/intermediates/dex/${namespacePath}/"

    // exclude classes which don't have a corresponding .java entry in the source directory
    def remExt = { name -> name.lastIndexOf('.').with {it != -1 ? name[0..<it] : name} }
    eachFile {details ->
        def thisFile = new File("${projectDir}/src/main/java/${namespacePath}/", remExt(details.name)+".java")
        if (!(thisFile.exists())) {
            details.exclude()
        }
    }
}

task assembleExternalJar << {
    // Get the location of the Android SDK
    ext.androidSdkDir = System.env.ANDROID_HOME
    if(androidSdkDir == null) {
        Properties localProps = new Properties()
        localProps.load(new FileInputStream(file('local.properties')))
        ext.androidSdkDir = localProps['sdk.dir']
    }
    // Make sure no existing jar file exists as this will cause dx to fail
    new File("${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar").delete();
    // Use command line dx utility to convert *.class files into classes.dex inside jar archive
    String cmdExt = Os.isFamily(Os.FAMILY_WINDOWS) ? '.bat' : ''
    exec {
        commandLine "${androidSdkDir}/build-tools/${BUILD_TOOLS_VERSION}/dx${cmdExt}", '--dex',
                    "--output=${buildDir}/intermediates/dex/${PLUGIN_NAMESPACE}.jar",
                    "${buildDir}/intermediates/dex/"
    }
    copyJarToOutputs.execute()
}

task copyJarToOutputs(type: Copy) {
    // Copy the built jar archive to the outputs folder
    from 'build/intermediates/dex/'
    into 'build/outputs/'
    include '*.jar'
}


// Set the dependencies of the build tasks so that assembleExternalJar does a complete build
copyClasses.dependsOn(assemble)
assembleExternalJar.dependsOn(copyClasses)

For more detailed information see the full source code for the sample app on my github.

Peipeiffer answered 2/12, 2014 at 2:49 Comment(2)
I've added a feature request for the Gradle Plugin to support using Ant style <Exclude ....> syntax to modify the Dex task, which would make this a lot easier and less fragile.Peipeiffer
I am having few queries regarding similar question. Can you please kindly check my question here - #32004370Armyworm
P
0

See my answer over here. The key points are:

  • Use the additionalParameters property on the dynamically created dexCamelCase tasks to pass --multi-dex to dx and create multiple dex files.
  • Use the multidex class loader to use the multiple dex files.
Proportionable answered 22/9, 2014 at 14:25 Comment(0)

© 2022 - 2024 — McMap. All rights reserved.