Introduction
If a program depends on a library it usually means that it uses methods of the library. Removing a dependency is therefore not a simple task. You effectively want to take away code that is - at least formally - required by the program.
There are three ways of removing dependencies:
- Adapt the source code to not depend on the library and compile it from scratch.
- Modify the bytecode to remove references to the library the project depends on.
- Manipulate the runtime to not require the dependency. The easiest way is to recreate the required classes and to put them into the jar file.
None of these ways are really pretty. All of them can require a lot of work. None are guaranteed to work without side effects.
Solution
I will describe my solution by presenting the files and steps I used to solve the problem. To reproduce, you will need the following files (in a single directory):
lib/xxx-v.v.v.jar: The library jars (httpclient and dependencies, excluding commons-logging-1.1.3.jar)
jarjar-1.4.jar: Used for repackaging the jars
rules.txt: The jarjar rules
rule org.apache.http.** my.http.@1
rule org.apache.commons.logging.** my.logging.@1
build.xml: Ant build configuration
<project name="MyProject" basedir=".">
<target name="logimpl">
<javac srcdir="java/src" destdir="java/bin" target="1.5" />
<jar jarfile="out/logimpl.jar" basedir="java/bin" />
</target>
<target name="merge">
<zip destfile="httpclient-4.3.1.jar">
<zipgroupfileset dir="out" includes="*.jar"/>
</zip>
</target>
</project>
java/src/Log.java
package my.logging;
public interface Log {
public boolean isDebugEnabled();
public void debug(Object message);
public void debug(Object message, Throwable t);
public boolean isInfoEnabled();
public void info(Object message);
public void info(Object message, Throwable t);
public boolean isWarnEnabled();
public void warn(Object message);
public void warn(Object message, Throwable t);
public boolean isErrorEnabled();
public void error(Object message);
public void error(Object message, Throwable t);
public boolean isFatalEnabled();
public void fatal(Object message);
public void fatal(Object message, Throwable t);
}
java/src/LogFactory.java
package my.logging;
public class LogFactory {
private static Log log;
public static Log getLog(Class<?> clazz) {
return getLog(clazz.getName());
}
public static Log getLog(String name) {
if(log == null) {
log = new Log() {
public boolean isWarnEnabled() { return false; }
public boolean isInfoEnabled() { return false; }
public boolean isFatalEnabled() { return false; }
public boolean isErrorEnabled() {return false; }
public boolean isDebugEnabled() { return false; }
public void warn(Object message, Throwable t) {}
public void warn(Object message) {}
public void info(Object message, Throwable t) {}
public void info(Object message) {}
public void fatal(Object message, Throwable t) {}
public void fatal(Object message) {}
public void error(Object message, Throwable t) {}
public void error(Object message) {}
public void debug(Object message, Throwable t) {}
public void debug(Object message) {}
};
}
return log;
}
}
do_everything.sh
#!/bin/sh
# Repackage library
mkdir -p out
for jf in lib/*.jar; do
java -jar jarjar-1.4.jar process rules.txt $jf `echo $jf | sed 's/lib\//out\//'`
done
# Compile logging implementation
mkdir -p java/bin
ant logimpl
# Merge jar files
ant merge
That's it. Open up a console and execute
cd my_directory && ./do_everything.sh
This will create a folder "out" containing single jar files and "httpclient-4.3.1.jar" which is the final, independent and working jar file. So, what did we just do?
- Repackaged httpclient (now in
my.http)
- Modified the library to use
my.logging instead of org.apache.commons.logging
- Compiled required classes to be used by the library (
my.logging.Log and my.logging.LogFactory).
- Merged the repackaged libraries and the compiled classes into a single jar file, httpclient-4.3.1.jar.
Pretty simple, isn't it? Just read the shell script line by line to discover the single steps. To check whether all dependencies were removed you can run
java -jar jarjar-1.4.jar find class httpclient-4.3.1.jar commons-logging-1.1.3.jar
I tried the generated jar file with SE7 and Android 4.4, it worked in both cases (see below for remarks).
Class file version
Every class file has a major version and a minor version (both depend on the compiler). The Android SDK requires class files to have a major version less than 0x33 (so everything pre 1.7 / JDK 7). I added the target="1.5" attribute to the ant javac task so the generated class files have a major version of 0x31 and can therefore be included in your Android app.
Alternative (bytecode manipulation)
You're lucky. Logging is (almost always) a one-way operation. It barely causes side effects affecting the main program. That means that removing commons-logging should be possible as it won't affect the functionality of the program.
I chose the second way, bytecode manipulation, which you suggested in your question. The concept is basically just this (A is httpclient, B is commons-logging):
- If the return type of a method of A is part of B, the return type will be changed to
java.lang.Object.
- If any argument of a method of A has a type that is part of B, the argument type will be changed to
java.lang.Object.
- Invocations of methods belonging to B are removed entirely.
pop and constant instructions are inserted to repair the VM stack.
- Types belonging to B are removed from descriptors of methods called from A. This requires the target class (the class containing the called method) to be processed. All object types belonging to B will be replaced with
java.lang.Object.
- Instructions that attempt to access fields of classes belonging to B are removed.
pop and constant instructions are inserted to repair the VM stack.
- If a method tries to access a field of a type that belongs to B, the field signature referenced by the instruction is changed to
java.lang.Object. This requires the target class (the class containing the accessed field) to be processed.
- Fields of a type contained in B but belonging to classes of A are modified so that their type is
java.lang.Object.
As you can see, the idea behind this is to replace all referenced classes with java.lang.Object and to remove all accesses to class members belonging to commons-logging.
I don't know whether this is reliable and I did not test the library after applying the manipulator. But from what I saw (the disassembled class files and no VM errors while loading the class files) I am fairly sure the code works.
I tried to document almost everything the program does. It uses the ASM Tree API which provides rather simple access to the class file structure. And - to avoid unnecessary negative reviews - this is "quick 'n' dirty" code. I did not really test it a lot and I bet there are faster ways of bytecode manipulation. But this program seems to fulfill the OP's needs and that's all I wrote it for.
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Enumeration;
import java.util.List;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.jar.JarOutputStream;
import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassWriter;
import org.objectweb.asm.Opcodes;
import org.objectweb.asm.Type;
import org.objectweb.asm.tree.AbstractInsnNode;
import org.objectweb.asm.tree.ClassNode;
import org.objectweb.asm.tree.FieldInsnNode;
import org.objectweb.asm.tree.FieldNode;
import org.objectweb.asm.tree.InsnList;
import org.objectweb.asm.tree.InsnNode;
import org.objectweb.asm.tree.MethodInsnNode;
import org.objectweb.asm.tree.MethodNode;
public class DependencyFinder {
public static void main(String[] args) throws IOException {
if(args.length < 2) return;
DependencyFinder df = new DependencyFinder();
df.analyze(new File(args[0]), new File(args[1]), "org.apache.http/.*", "org.apache.commons.logging..*");
}
@SuppressWarnings("unchecked")
public void analyze(File inputFile, File outputFile, String sClassRegex, String dpClassRegex) throws IOException {
JarFile inJar = new JarFile(inputFile);
JarOutputStream outJar = new JarOutputStream(new FileOutputStream(outputFile));
for(Enumeration<JarEntry> entries = inJar.entries(); entries.hasMoreElements();) {
JarEntry inEntry = entries.nextElement();
InputStream inStream = inJar.getInputStream(inEntry);
JarEntry outEntry = new JarEntry(inEntry.getName());
outEntry.setTime(inEntry.getTime());
outJar.putNextEntry(outEntry);
OutputStream outStream = outJar;
// Only process class files, copy all other resources
if(inEntry.getName().endsWith(".class")) {
// Initialize class reader and writer
ClassReader classReader = new ClassReader(inStream);
ClassWriter classWriter = new ClassWriter(0);
String className = classReader.getClassName();
// Check whether to process this class
if(className.matches(sClassRegex)) {
System.out.println("Processing " + className);
// Parse entire class
ClassNode classNode = new ClassNode(Opcodes.ASM4);
classReader.accept(classNode, 0);
// Check super class and interfaces
String superClassName = classNode.superName;
if(superClassName.matches(dpClassRegex)) {
throw new RuntimeException(className + " extends " + superClassName);
}
for(String iface : (List<String>) classNode.interfaces) {
if(iface.matches(dpClassRegex)) {
throw new RuntimeException(className + " implements " + superClassName);
}
}
// Process methods
for(MethodNode method : (List<MethodNode>) classNode.methods) {
Type methodDesc = Type.getMethodType(method.desc);
boolean changed = false;
// Change return type if necessary
Type retType = methodDesc.getReturnType();
if(retType.getClassName().matches(dpClassRegex)) {
retType = Type.getObjectType("java/lang/Object");
changed = true;
}
// Change argument types if necessary
Type[] argTypes = methodDesc.getArgumentTypes();
for(int i = 0; i < argTypes.length; i++) {
if(argTypes[i].getClassName().matches(dpClassRegex)) {
argTypes[i] = Type.getObjectType("java/lang/Object");
changed = true;
}
}
if(changed) {
// Update method descriptor
System.out.print("Changing " + method.name + methodDesc);
methodDesc = Type.getMethodType(retType, argTypes);
method.desc = methodDesc.getDescriptor();
System.out.println(" to " + methodDesc);
}
// Remove method invocations
InsnList insns = method.instructions;
for(int i = 0; i < insns.size(); i++) {
AbstractInsnNode insn = insns.get(i);
// Ignore all other nodes
if(insn instanceof MethodInsnNode) {
MethodInsnNode mnode = (MethodInsnNode) insn;
Type[] cArgTypes = Type.getArgumentTypes(mnode.desc);
Type cRetType = Type.getReturnType(mnode.desc);
if(mnode.owner.matches(dpClassRegex)) {
// The method belongs to one of the classes we want to get rid of
System.out.println("Removing method call " + mnode.owner + "." +
mnode.name + " in " + method.name);
boolean isStatic = (mnode.getOpcode() == Opcodes.INVOKESTATIC);
if(!isStatic) {
// pop instance
insns.insertBefore(insn, new InsnNode(Opcodes.POP));
}
for(int j = 0; j < cArgTypes.length; j++) {
// pop argument on stack
insns.insertBefore(insn, new InsnNode(Opcodes.POP));
}
// Insert a constant value to repair the stack
if(cRetType.getSort() != Type.VOID) {
InsnNode valueInsn = getValueInstruction(cRetType);
insns.insertBefore(insn, valueInsn);
}
// Remove the actual method call
insns.remove(insn);
// Go back one instruction to not skip the next one
i--;
} else {
changed = false;
if(cRetType.getClassName().matches(dpClassRegex)) {
// Change return type
cRetType = Type.getObjectType("java/lang/Object");
changed = true;
}
for(int j = 0; j < cArgTypes.length; j++) {
if(cArgTypes[j].getClassName().matches(dpClassRegex)) {
// Change argument type
cArgTypes[j] = Type.getObjectType("java/lang/Object");
changed = true;
}
}
if(changed) {
// Update method invocation
System.out.println("Patching method call " + mnode.owner + "." +
mnode.name + " in " + method.name);
mnode.desc = Type.getMethodDescriptor(cRetType, cArgTypes);
}
}
} else if(insn instanceof FieldInsnNode) {
// Yeah I lied... we must not ignore all other instructions
FieldInsnNode fnode = (FieldInsnNode) insn;
Type fieldType = Type.getType(fnode.desc);
if(fnode.owner.matches(dpClassRegex)) {
System.out.println("Removing field access to " + fnode.owner + "." +
fnode.name + " in " + method.name);
// Patch code
switch(fnode.getOpcode()) {
case Opcodes.PUTFIELD:
case Opcodes.GETFIELD:
// Pop instance
insns.insertBefore(insn, new InsnNode(Opcodes.POP));
if(fnode.getOpcode() == Opcodes.PUTFIELD) break;
case Opcodes.GETSTATIC:
// Repair stack
insns.insertBefore(insn, getValueInstruction(fieldType));
break;
default:
throw new RuntimeException("Invalid opcode");
}
// Remove instruction
insns.remove(fnode);
i--;
} else {
if(fieldType.getClassName().matches(dpClassRegex)) {
// Change field type
System.out.println("Patching field access to " + fnode.owner +
"." + fnode.name + " in " + method.name);
fieldType = Type.getObjectType("java/lang/Object");
}
// Update field type
fnode.desc = fieldType.getDescriptor();
}
}
}
}
// Process fields
for(FieldNode field : (List<FieldNode>) classNode.fields) {
Type fieldType = Type.getType(field.desc);
if(fieldType.getClassName().matches(dpClassRegex)) {
System.out.print("Changing " + fieldType.getClassName() + " " + field.name);
fieldType = Type.getObjectType("java/lang/Object");
field.desc = fieldType.getDescriptor();
System.out.println(" to " + fieldType.getClassName());
}
}
// Class processed
classNode.accept(classWriter);
} else {
// Nothing changed
classReader.accept(classWriter, 0);
}
// Write class to JAR entry
byte[] bClass = classWriter.toByteArray();
outStream.write(bClass);
} else {
// Copy file
byte[] buffer = new byte[1024 * 64];
int read;
while((read = inStream.read(buffer)) != -1) {
outStream.write(buffer, 0, read);
}
}
outJar.closeEntry();
}
outJar.flush();
outJar.close();
inJar.close();
}
InsnNode getValueInstruction(Type type) {
switch(type.getSort()) {
case Type.INT:
case Type.BOOLEAN:
return new InsnNode(Opcodes.ICONST_0);
case Type.LONG:
return new InsnNode(Opcodes.LCONST_0);
case Type.OBJECT:
case Type.ARRAY:
return new InsnNode(Opcodes.ACONST_NULL);
default:
// I am lazy, I did not implement all types
throw new RuntimeException("Type not implemented: " + type);
}
}
}
