Proguard and XStream with omitField() on Android

Question

I was using XStream for deserialization of xml in my Android app, and now I'm struggling to add Proguard (obfuscator) to the mix.

Here's the runtime exception I run into (full: pastebin):

WARN/System.err(6209): net.lp.collectionista.util.a.g: XStream could not parse the response
WARN/System.err(6209):     at net.lp.collectionista.a.s.a(Collectionista:215)
    ...
WARN/System.err(6209): Caused by: com.thoughtworks.xstream.converters.ConversionException: id : id in loader dalvik.system.PathClassLoader[/data/app/net.lp.collectionista-2.apk] : id : id in loader dalvik.system.PathClassLoader[/data/app/net.lp.collectionista-2.apk]
WARN/System.err(6209): ---- Debugging information ----
WARN/System.err(6209): message             : id : id in loader dalvik.system.PathClassLoader[/data/app/net.lp.collectionista-2.apk]
WARN/System.err(6209): cause-exception     : com.thoughtworks.xstream.mapper.CannotResolveClassException
WARN/System.err(6209): cause-message       : id : id in loader dalvik.system.PathClassLoader[/data/app/net.lp.collectionista-2.apk]
WARN/System.err(6209): class               : net.lp.collectionista.jaxb.googlebooks.search.Feed
WARN/System.err(6209): required-type       : java.lang.Object
WARN/System.err(6209): path                : /feed/entry/id
WARN/System.err(6209): line number         : 1
WARN/System.err(6209): -------------------------------
WARN/System.err(6209):     at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(Collectionista:89)
    ...
WARN/System.err(6209):     at com.thoughtworks.xstream.XStream.fromXML(Collectionista:861)
    ...
WARN/System.err(6209): Caused by: com.thoughtworks.xstream.mapper.CannotResolveClassException: id : id in loader dalvik.system.PathClassLoader[/data/app/net.lp.collectionista-2.apk]
WARN/System.err(6209):     at com.thoughtworks.xstream.mapper.DefaultMapper.realClass(Collectionista:68)
    ...

Needless to say this works fine without Proguard. I'm using shrinking, optimizing and obfuscating here, though I disabled it all on any XStream class, as well as any class that stands model for the xml fields:

-keep class net.lp.collectionista.jaxb.** { *; }
-keep class com.thoughtworks.xstream.** { *; }

I can confirm, from the obfuscated jar, as well as from the mapping.txt (for methods), that any classes mentioned exist and are not obfuscated, so untouched AFAICT. I also am retaining annotations.

The exception is pretty clear to me. I have:

        xstream.omitField(Feed.class, "id");

among others. It seems the omitField() call does not work anymore and it starts looking for an "id" model class, because of Proguard. This is where I am stuck, even after diving into the XStream code. The whole omitField call in the obfuscated end result seems to be intact, so what could be additionally broken here? It should also not be "Feed.class" as that one is also still there. What am I missing? What is a good next step for debugging?

EDIT: I did notice the class files of xstream classes in my obfuscated jar are slightly smaller than the original ones, even with -dontoptimize. What is still being dropped?

EDIT2: I'm starting to think it has to do with the absence of dex warnings similar to the following:

[apply] warning: Ignoring InnerClasses attribute for an anonymous inner class
[apply] (com.thoughtworks.xstream.converters.reflection.AbstractReflectionConverter$1) that doesn't come with an
[apply] associated EnclosingMethod attribute. This class was probably produced by a
[apply] compiler that did not target the modern .class file format. The recommended
[apply] solution is to recompile the class from source, using an up-to-date compiler
[apply] and without specifying any "-target" type options. The consequence of ignoring
[apply] this warning is that reflective operations on this class will incorrectly
[apply] indicate that it is *not* an inner class.

... or maybe not ...

EDIT3: Finally, in spite of dealing with many other bugs and problems such as the SimException bug, I've been able to get it working in some limited cases. That way I could pinpoint it to the obfuscation step. That is, at least, if I add "-dontobfuscate", the problem goes away. it isn't the first time I'm playing with that, so it must be the workarounds for the other problems, or the narrower configuration, that alleviates this problem as well. So here's me asking again: When I've already safeguarded the main parts of xstream and my model classes from obfuscation using "-keep", then what else could be creating this mess?

If you need more info, let me know.

Heilungkiang

Answer 1

As I said, the problem disappears if you -dontobfuscate but let's suppose you don't want that.

The solution is to keep more attributes:

-keepattributes EnclosingMethod, InnerClasses
-keepattributes *Annotation*
-keepattributes Signature

Once you get it working you can narrow down which parts of the XStream code to keep as well. I have:

-keep class com.thoughtworks.xstream.converters.extended.SubjectConverter { *; }
-keep class com.thoughtworks.xstream.converters.extended.ThrowableConverter { *; }
-keep class com.thoughtworks.xstream.converters.extended.StackTraceElementConverter { *; }
-keep class com.thoughtworks.xstream.converters.extended.CurrencyConverter { *; }
-keep class com.thoughtworks.xstream.converters.extended.RegexPatternConverter { *; }
-keep class com.thoughtworks.xstream.converters.extended.CharsetConverter { *; }

-keep class com.thoughtworks.xstream.annotations.** { *; }

You can also disable a lot of warnings related to XStream.

For more details you can find my version controlled project files here:

• proguard.cfg

• build.xml

Heilungkiang

Can disabling warnings for XStream be dangerous (provided that i'll never replace my XStream jars)? I mean, can it lead to any difficult-to-find errors in the future? – Negro

Yeah, I believe, when the warnings are not false-positives, you'll mis them. – Heilungkiang

Answer 2

I seem to be one of many to have problems with ProGuard and XStream for Android. After some trial and research, the following is what works for me - a complete config file together with some explanatory comments why I did what I did. Note that my priority was to obfuscate, but I did not care much about optimizing or shrinking.

And you need to remember that with this config (which keeps public members for libraries - see below) you will need to use "public" members for the class that is used by XStream to create your XML, because XStream will use member names for XML labels - and you do not want your XML labels to be changed to "a", "b" or "c" :) Good luck!

###########################################################
#
# FLAGS
#
###########################################################


# Not sure if I need this one, but seems to do no harm
-keepdirectories

# I needed NOT to optimize for SWT or XStream would not work, but for Android I do not seem to need to do that.
# However, if I try to shrink, XStream fails for Android. This different behaviour is a bit odd and black magic.
# However, I do not much care about optimization or size, and stability is more important for me, so let's
# neither optimize nor shrink (and shrinking saved us only about 1.5% of the size anyway).
#
# Note: this was not all that was necessary to make XStream run for Android - see other comments
# (search for XStream)
-dontshrink
-dontoptimize
# The following was configured for Android by default but now it does not make sense because I do not optmize, so disable just in case.
# -optimizationpasses 5

# Not sure if I need this one, but seems to do no harm.
-keeppackagenames

# This was configured for Android by default and it can only help.
-dontusemixedcaseclassnames

# This was configured for Android by default, and it is the default option as of ProGuard 4.5 anyway.
-dontskipnonpubliclibraryclasses

# ProGuard documentation says:
# For Java 6, preverification is optional, but as of Java 7, it is required.
# Only when eventually targeting Android, it is not necessary, so you can then
# switch it off to reduce the processing time a bit.
-dontpreverify

# Specifies to write out some more information during processing. If the
# program terminates with an exception, this option will print out the
# entire stack trace, instead of just the exception message.
-verbose

# Since I have turned off optmization, it makes no sense to have the following
# option enabled.
#-optimizations !code/simplification/arithmetic,!field/*,!class/merging/*

# The followig was necessary or it would not build, as it otherwise wants a totally clean build first.
-ignorewarnings

###########################################################
#
# -keep SPECIFICATIONS
#
###########################################################

# I tried adding those to fix the XStream problem, but in the end fixed it differently (see other comments).
#-keepattributes EnclosingMethod, InnerClasses
#-keepattributes *Annotation*
#-keepattributes Signature

# The following was configured for Android by default.
-keep public class * extends android.app.Activity
-keep public class * extends android.app.Application
-keep public class * extends android.app.Service
-keep public class * extends android.content.BroadcastReceiver
-keep public class * extends android.content.ContentProvider
-keep public class * extends android.app.backup.BackupAgentHelper
-keep public class * extends android.preference.Preference
-keep public class com.android.vending.licensing.ILicensingService

# I tried adding those to fix the XStream problem, but in the end fixed it differently (see other comments).
# However, it might still be a good idea to stay away from thoughtworks for stability's sake.
# (Not sure if the second keep does not include the first one.)
-keep class com.thoughtworks.xstream.*
-keep class com.thoughtworks.xstream.* {
    public protected <methods>;
    public protected <fields>;
}

# The following plus not-shrinking seems necessary to make XStream run for Android.
# But again, as for SWT, I did not need to exclude all, public and protected methods and fields:
# just doing the public fields was enough.
#    public protected <methods>;
#    public protected <fields>;
-keep public class * {
    public <fields>;
}

# This was configured for Android by default - and very necessary it is too.
-keepclassmembers enum * {
    public static **[] values();
    public static ** valueOf(java.lang.String);
}

# I put it in because we might need this one in the future.
# This was default for the Windows installation of ProGuard, which said:
# Also keep - Database drivers. Keep all implementations of java.sql.Driver.
-keep class * extends java.sql.Driver

# This was configured for Android by default.
-keepclasseswithmembernames class * {
    native <methods>;
}

# This was configured for Android by default.
-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet);
}

# This was configured for Android by default.
-keepclasseswithmembers class * {
    public <init>(android.content.Context, android.util.AttributeSet, int);
}

# This was configured for Android by default.
-keepclassmembers class * extends android.app.Activity {
   public void *(android.view.View);
}

# This was configured for Android by default.
-keep class * implements android.os.Parcelable {
  public static final android.os.Parcelable$Creator *;
}

Callow

Thanks for that. I'm not sure if I'll be able to try it out any time soon though. Can you highlight the different approach versus the accepted answer and/or why the accepted answer is incorrect/incomplete? – Heilungkiang

Answer 3

XStream seems to introspect on the EnclosingMethod attribute, so keeping it in your ProGuard configuration may help:

-keepattributes EnclosingMethod

The ProGuard manual provides a list of attributes that you may want to keep. The error message suggests that you're already keeping the InnerClasses attribute, which is probably required indeed.

Miniskirt

Hi Eric, thanks for your answer. I'm already keeping the EnclosingMethod as well, and also all Annotations attributes. Can it be anything else? – Heilungkiang