How to share split APKs created while using instant-run, within Android itself?
Asked Answered
N

4

42

Background

I have an app (here) that, among other features, allows to share APK files.

In order to do so, it reaches the file by accessing the path of packageInfo.applicationInfo.sourceDir (docs link here), and just shares the file (using ContentProvider when needed, as I've used here).

The problem

This works fine in most cases, especially when installing APK files from either the Play Store or from a standalone APK file, but when I install an app using Android-Studio itself, I see multiple APK files on this path, and none of them are valid ones that can be installed and run without any issues.

Here's a screenshot of the content of this folder, after trying out a sample from "Alerter" github repo :

enter image description here

I'm not sure when this issue has started, but it does occur at least on my Nexus 5x with Android 7.1.2. Maybe even before.

What I've found

This seems to be caused only from the fact that instant run is enabled on the IDE, so that it could help updating the app without the need to re-build it all together :

enter image description here

After disabling it, I can see that there is a single APK, just as it used to be in the past:

enter image description here

You can see the difference in file size between the correct APK and the split one.

Also, it seems that there is an API to get the paths to all of the splited APKs:

https://developer.android.com/reference/android/content/pm/ApplicationInfo.html#splitPublicSourceDirs

The question

What should be the easiest way to share an APK that got to be split into multiple ones ?

Is it really needed to somehow merge them?

It seems it is possible according to the docs :

Full paths to zero or more split APKs that, when combined with the base APK defined in sourceDir, form a complete application.

But what's the correct way to do it, and is there a fast and efficient way to do it? Maybe without really creating a file?

Is there maybe an API to get a merged APK out of all the split ones? Or maybe such an APK already exist anyway in some other path, and there is no need for merging?

EDIT: just noticed that all third party apps that I've tried are supposed to share an installed app's APK fail to do so in this case.

Necessarian answered 27/2, 2017 at 19:41 Comment(13)
Why the downvote? This is a new issue. It's not written about here. Sharing an APK file that's created this way doesn't work as with normal ones.Necessarian
Why you need to share installed (via Android studio) app in the first place?Perilymph
@Perilymph I want to be able to share any installed app. It doesn't matter the source of where you've installed the app from. In my case, for example, I sometimes try out apps from Github, and then later, outside of the office/home, I want to share the APK of what I've tried (share with my colleagues, for example).Necessarian
Then forget about instant run. It isn't meant for sharing app. it is meant for checking changes fast. Even if you share outputs/apk/debug-ss.apk it will crash on other phone (personal experience). And it's just how it works, i doubt that this issue can be fixed without major platform changes. for more info read this: medium.com/google-developers/…Perilymph
@Perilymph I want to use instant-run because it makes (at least sometimes) it a bit faster to build&run apps. Also, I'm not the only user of the app. I never had an issue before (unless the Android version doesn't match the range the app supports, of course)Necessarian
Use Android studio 2.3 It builds significantly faster and now on stable channel. You want to buy alive chicken at KFC here.Perilymph
@Perilymph large projects still take at least 1-2 minutes to build, and people can use this feature, so again, not interested in avoiding the problem.Necessarian
Use Instant Run for yourself. Make non-Instant Run builds for sharing.Cavallaro
@Cavallaro Developers are users too, and they can install apps using instant-run, and so they should be able to share them.Necessarian
when you use instant run and do an incremental update, are the updates saved to the apk anyway? I just tried to install my app with install run enabled, did a hot swap and then killed the app. when I restarted the app the updates were goneDeeply
@androiddeveloper, just to be clear on the question: you want to be able to combine the instant run splits, oats, libs and base.apk into an APK that can be installed on its own on a different device?Balfore
@HypotheticalintheClavicle Yes, so that the person who got the APK can install and run it.Necessarian
@Deeply I've tested, and saw that there are multiple APK files, none can really be installed alone. What can happen later doesn't matter to me, because this state stays for a very long time (didn't notice them get merged, so it's long enough).Necessarian
I
18

I am the tech lead @Google for the Android Gradle Plugin, let me try to answer your question assuming I understand your use case.

First, some users mentioned you should not share your InstantRun enabled build and they are correct. The Instant Run builds on an application is highly customized for the current device/emulator image you are deploying to. So basically, say you generate an IR enabled build of your app for a particular device running 21, it will fail miserably if you try to use those exact same APKs on say a device running 23. I can go into much deeper explanation if necessary but suffice to say that we generate byte codes customized on the APIs found in android.jar (which is of course version specific).

So I do not think that sharing those APKs make sense, you should either use a IR disabled build or a release build.

Now for some details, each slice APK contains 1+ dex file(s), so in theory, nothing prevents you from unziping all those slice APKs, take all the dex files and stuff them back into the base.apk/rezip/resign and it should just work. However, it will still be an IR enabled application so it will start the small server to listen to IDE requests, etc, etc... I cannot imagine a good reason for doing this.

Hope this helps.

Inkle answered 13/4, 2017 at 4:2 Comment(5)
About the merging, this is an issue, because the various split APKs have same-name files. About the way that instant-run works, this is disappointing, as this ruins a very common use for developers, who want to share the APK on the go. Why can't it just work using delta-changes ? Meaning that there will always be a single APK (that should work for all supported devices and Android versions), and each time there is a change, patch the APK with the new change? Why is it so specific to the device&Android version? Anyway, how can I detect split APKs, to warn the user it's not possible to share it?Necessarian
I think that in order to check if the installed app has split APK, is to check if packageInfo.applicationInfo.publicSourceDir is not empty (and not null, of course). Is it ok? Do apps installed from the Play Store guaranteed to stay intact, without any split APKs ?Necessarian
Up to 2.4.0, we did not have incremental dexing, which mean that each time you would change .class file we would redex the entire main apk contents. With slicing, we only re-dex one split apk (about 10% of the overall cost). Starting in 2.4.0, we could potentially implement your suggestion but we would still need to merge all dexes and repackage a much bigger APK (zip) file, this would be big regression.Inkle
How come handling with multiple files be more efficient than a single file? Or take less? Can't the files be cached on the PC, and all the device has to do is to reach the needed positions and patch the file? Also, why would it be a bigger APK file? It should contain the whole code as installed on any other device. That's the whole point of a single cross-versions APK ... For me it's a regression of how APKs work. Now all of the split APKs are invalid, cannot be used anywhere. Can't even restore them on the same device (useful when going to Android O and back to N).Necessarian
How can I merge the split APK files?Necessarian
E
1

To merge multiple split apks to an single apk might be a little complicated.

Here is a suggestion to share the split apks directly and let the system to handle the merge and installation.

This might not be an answer to the question, since it's a little long, I post here as an 'answer'.

Framework new API PackageInstaller can handle monolithic apk or split apk.

In development environment

  • for monolithic apk, using adb install single_apk

  • for split apk, using adb install-multiple a_list_of_apks

You can see these two modes above from android studio Run output depends on your project has Instant run enable or disable.

For the command adb install-multiple, we can see the source code here, it will call the function install_multiple_app.

And then perform the following procedures

pm install-create # create a install session
pm install-write  # write a list of apk to session
pm install-commit # perform the merge and install

What the pm actually do is call the framework api PackageInstaller, we can see the source code here

runInstallCreate
runInstallWrite
runInstallCommit

It's not mysterious at all, I just copied some methods or function here.

The following script can be invoked from adb shell environment to install all split apks to device, like adb install-multiple. I think it might work programmatically with Runtime.exec if your device is rooted.

#!/system/bin/sh

# get the total size in byte
total=0
for apk in *.apk
do
    o=( $(ls -l $apk) )
    let total=$total+${o[3]}
done

echo "pm install-create total size $total"

create=$(pm install-create -S $total)
sid=$(echo $create |grep -E -o '[0-9]+')

echo "pm install-create session id $sid"

for apk in *.apk
do
    _ls_out=( $(ls -l $apk) )
    echo "write $apk to $sid"
    cat $apk | pm install-write -S ${_ls_out[3]} $sid $apk -
done

pm install-commit $sid

I my example, the split apks include (I got the list from android studio Run output)

app/build/output/app-debug.apk
app/build/intermediates/split-apk/debug/dependencies.apk
and all apks under app/build/intermediates/split-apk/debug/slices/slice[0-9].apk

Using adb push all the apks and the script above to a public writable directory, e.g /data/local/tmp/slices, and run the install script, it will install to your device just like adb install-multiple.

The code below is just another variant of the script above, if your app has platform signature or device is rooted, I think it will be ok. I didn't have the environment to test.

private static void installMultipleCmd() {
    File[] apks = new File("/data/local/tmp/slices/slices").listFiles(new FileFilter() {
        @Override
        public boolean accept(File pathname) {
            return pathname.getAbsolutePath().endsWith(".apk");
        }
    });
    long total = 0;
    for (File apk : apks) {
        total += apk.length();
    }

    Log.d(TAG, "installMultipleCmd: total apk size " + total);
    long sessionID = 0;
    try {
        Process pmInstallCreateProcess = Runtime.getRuntime().exec("/system/bin/sh\n");
        BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(pmInstallCreateProcess.getOutputStream()));
        writer.write("pm install-create\n");
        writer.flush();
        writer.close();

        int ret = pmInstallCreateProcess.waitFor();
        Log.d(TAG, "installMultipleCmd: pm install-create return " + ret);

        BufferedReader pmCreateReader = new BufferedReader(new InputStreamReader(pmInstallCreateProcess.getInputStream()));
        String l;
        Pattern sessionIDPattern = Pattern.compile(".*(\\[\\d+\\])");
        while ((l = pmCreateReader.readLine()) != null) {
            Matcher matcher = sessionIDPattern.matcher(l);
            if (matcher.matches()) {
                sessionID = Long.parseLong(matcher.group(1));
            }
        }
        Log.d(TAG, "installMultipleCmd: pm install-create sessionID " + sessionID);
    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    }

    StringBuilder pmInstallWriteBuilder = new StringBuilder();
    for (File apk : apks) {
        pmInstallWriteBuilder.append("cat " + apk.getAbsolutePath() + " | " +
                "pm install-write -S " + apk.length() + " " + sessionID + " " + apk.getName() + " -");
        pmInstallWriteBuilder.append("\n");
    }

    Log.d(TAG, "installMultipleCmd: will perform pm install write \n" + pmInstallWriteBuilder.toString());

    try {
        Process pmInstallWriteProcess = Runtime.getRuntime().exec("/system/bin/sh\n");
        BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(pmInstallWriteProcess.getOutputStream()));
        // writer.write("pm\n");
        writer.write(pmInstallWriteBuilder.toString());
        writer.flush();
        writer.close();

        int ret = pmInstallWriteProcess.waitFor();
        Log.d(TAG, "installMultipleCmd: pm install-write return " + ret);
        checkShouldShowError(ret, pmInstallWriteProcess);
    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    }

    try {
        Process pmInstallCommitProcess = Runtime.getRuntime().exec("/system/bin/sh\n");
        BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(pmInstallCommitProcess.getOutputStream()));
        writer.write("pm install-commit " + sessionID);
        writer.flush();
        writer.close();

        int ret = pmInstallCommitProcess.waitFor();
        Log.d(TAG, "installMultipleCmd: pm install-commit return " + ret);
        checkShouldShowError(ret, pmInstallCommitProcess);
    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    }
}

private static void checkShouldShowError(int ret, Process process) {
    if (process != null && ret != 0) {
        BufferedReader reader = null;
        try {
            reader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
            String l;
            while ((l = reader.readLine()) != null) {
                Log.d(TAG, "checkShouldShowError: " + l);
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            if (reader != null) {
                try {
                    reader.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

Meanwhile, the simple way, you can try the framework api. Like the sample code above, it might work if the device is rooted or your app has platform signature, but I didn't get a workable environment to test it.

private static void installMultiple(Context context) {
    if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.LOLLIPOP) {
        PackageInstaller packageInstaller = context.getPackageManager().getPackageInstaller();

        PackageInstaller.SessionParams sessionParams = new PackageInstaller.SessionParams(PackageInstaller.SessionParams.MODE_FULL_INSTALL);

        try {
            final int sessionId = packageInstaller.createSession(sessionParams);
            Log.d(TAG, "installMultiple: sessionId " + sessionId);

            PackageInstaller.Session session = packageInstaller.openSession(sessionId);

            File[] apks = new File("/data/local/tmp/slices/slices").listFiles(new FileFilter() {
                @Override
                public boolean accept(File pathname) {
                    return pathname.getAbsolutePath().endsWith(".apk");
                }
            });

            for (File apk : apks) {
                InputStream inputStream = new FileInputStream(apk);

                OutputStream outputStream = session.openWrite(apk.getName(), 0, apk.length());
                byte[] buffer = new byte[65536];
                int count;
                while ((count = inputStream.read(buffer)) != -1) {
                    outputStream.write(buffer, 0, count);
                }

                session.fsync(outputStream);
                outputStream.close();
                inputStream.close();
                Log.d(TAG, "installMultiple: write file to session " + sessionId + " " + apk.length());
            }

            try {
                IIntentSender target = new IIntentSender.Stub() {

                    @Override
                    public int send(int i, Intent intent, String s, IIntentReceiver iIntentReceiver, String s1) throws RemoteException {
                        int status = intent.getIntExtra(PackageInstaller.EXTRA_STATUS, PackageInstaller.STATUS_FAILURE);
                        Log.d(TAG, "send: status " + status);
                        return 0;
                    }
                };
                session.commit(IntentSender.class.getConstructor(IIntentSender.class).newInstance(target));
            } catch (InstantiationException | IllegalAccessException | NoSuchMethodException | InvocationTargetException e) {
                e.printStackTrace();
            }
            session.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}

In order to use the hidden api IIntentSender, I add the jar library android-hidden-api as the provided dependency.

Ectoplasm answered 18/3, 2017 at 11:32 Comment(5)
No, I didn't merge it.Ectoplasm
The question was how to merge it to be able to share the APK, so that the person who got the APK could install it using the normal installer.Necessarian
@androiddeveloper you can share the all the split apks, and use this method to install it.Ectoplasm
I can't tell users who get the APKs, to install an app to install them... I think it even requires root... Have you tried using this function? Did it work for you?Necessarian
Is it possible to merge the split APK files into one? Is it possible, without root, to install split APK files within the device?Necessarian
K
1

Those are called split apks. Which is mainly used in the PlayStore. As you may know, PlayStore only shows apps to the user if it's compatible with the device. Same in this case. The split has files varies from devices. Like if you used Different resources for Different devices which makes app really heavy. By making splits, it saves space for downloading and running for the user by only downloading the usable split apks.

Is it possible to merge them into single apk? Yes. I used an app called Anti Split which allows that. Plus Apk Editor Ultra has same.

Can we save it into a single file? Yes you can. As like for Anti Split, you have to first backup the app. Like in this case you have to back it up as apks file or xapk which is called bundled app in Android Studio. I have created a library for doing this. It's working perfectly for me. Am using it to backup apps into xapk which can later be installed using SIA app or XAPK Installer or we can use xapk file to merge it and make apk

Kantianism answered 25/11, 2021 at 16:33 Comment(0)
X
-4

For me instant run was a nightmare, 2-5 minute build times, and maddeningly often, recent changes were not included in builds. I highly recommend disabling instant run and adding this line to gradle.properties:

android.enableBuildCache=true

First build often takes some time for large projects (1-2mins), but after it's cached subsequent builds are usually lightnight fast (<10secs).

Got this tip from reddit user /u/QuestionsEverythang which has saved me SO much hassling around with instant run!

Xylina answered 15/3, 2017 at 22:48 Comment(4)
although I share your frustrations it is better now, Android Studio 2.3, com.android.tools.build:gradle:2.3.0, distributionUrl=https\://services.gradle.org/distributions/gradle-3.4.1-all.zip, and building to devices running Android 7.0. Pretty much any other combination will be a nightmare.Snuggle
It's not a solution at all to the problem, but it is interesting: where do you put it exactly inside the gradle file ?Necessarian
Downvoted because this does not answer the question and appears to be an attempt to snag a hefty bounty for nothingBalfore
whatever, just tryin to helpXylina

© 2022 - 2024 — McMap. All rights reserved.