BlackBerry: create COD from JAR source file in Ant script

Question

How do I use Ant (bb-ant-tools) to compile a JAR file into a COD? i.e. with no other source files

---

Reason

Ultimately, I need a script that is going to run Jar Jar Links (jarjar) during the build, to get around the namespace issues of using our sdk in multiple apps. Jarjar takes a JAR file as input, and I presume it outputs a JAR file.

So to run jarjar in a bb-ant-tools build, I need to know how to compile that output JAR into a COD.

I am focusing on just this part of the issue to minimise other possible sources of error. I will attempt the jarjar step once I can master building a JAR into a COD.

Progress

1) I can build my project into a working COD, sign and run on device.

2) To include the sdk code, I currently add the source folders to the main project build. (In the future I would like to know how to include the library into the build as a JAR - BlackBerry - Ant script to include JAR in project without external dependancies)

3) Output of this step includes the usual files:

• MyApp.cod (if I sign this, it runs perfectly on device)
• MyApp.csl
• MyApp.cso
• MyApp.debug
• MyApp.jad
• MyApp.jar (I want to run jarjar on this)
• MyApp.rapc

4) I tried to run a 2nd build, taking the above JAR file and using it as the only source file in the rapc call. I do this by pointing the src tag of rapc at a folder containing only my JAR file.

In the final part of the build, when rapc is adding the files, I get the error:

 java.util.zip.ZipException: duplicate entry: MyApp-1.cod

(full details of error below in edited build output)

I can see that this is the 2nd time that rapc tries to add this file. I do not understand why, since there is only one copy of that file in the JAR.

---

Build Output (edited with ...etc to make it readable )

build:
    [mkdir] Skipping C:\development\ant\new_test\MyApp\build because it already exists.
     [copy] Copying 1 file to C:\development\ant\new_test\MyApp\build
     [copy] Copying C:\development\ant\new_test\MyApp\icon.png to C:\development\ant\new_test\MyApp\build\icon.png
     [rapc] Compiling 1 source files to MyApp.cod
     [rapc] Executing 'C:\Java\jdk1.6.0_24\jre\bin\java.exe' with arguments:
     [rapc] '-classpath'
     [rapc] 'C:\Java\jdk1.6.0_24\lib\tools.jar;C:\development\tools\bb-jde\jde5.0\components\bin\rapc.jar'
     [rapc] 'net.rim.tools.compiler.Compiler'
     [rapc] '-verbose'
     [rapc] 'import=C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar'
     [rapc] 'codename=MyApp'
     [rapc] 'MyApp.rapc'
     [rapc] '@sources.txt'
     [rapc]
     [rapc] The ' characters around the executable and arguments are
     [rapc] not part of the command.
     [rapc] Setting environment variable: PATH=........etc
     [rapc] Reading resource: MyApp.cod
...etc
     [rapc] Parsing classfile: com/MyApp/ui/views/WelcomeBar.class
...etc
     [rapc] Parsing import: C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar(net_rim_amms.cod)
...etc
     [rapc] Resolving
...etc
     [rapc] Optimizing
     [rapc] Utilities.java:449: Warning!: local variable(s) { finished } initialized but not used in: com.cobi.library.Utilities.split(String,String)
...etc
     [rapc] Populating
     [rapc] Invoking: jar -cfmv C:\development\ant\new_test\MyApp\build\MyApp.jar C:\Users\Richard\AppData\Local\Temp\rapc_598c0c5a.dir\META-INF\MANIFEST.MF MyApp.cod MyApp-1.cod MyApp-2.cod MyApp.csl MyApp.cso -C C:\Users\Richard\AppData\Local\Temp\rapc_598c2ad7.dir .
     [rapc] added manifest
     [rapc] adding: MyApp.cod(in = 63208) (out= 41042)(deflated 35%)
     [rapc] adding: MyApp-1.cod(in = 75448) (out= 42559)(deflated 43%)
     [rapc] adding: MyApp.csl(in = 91) (out= 69)(deflated 24%)
     [rapc] adding: MyApp.cso(in = 157) (out= 93)(deflated 40%)
...etc - adding all files I can see in the JAR...

     [rapc] adding: MyApp-1.cod

     java.util.zip.ZipException: duplicate entry: MyApp-1.cod
     [rapc]     at java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:175)
     [rapc]     at java.util.jar.JarOutputStream.putNextEntry(JarOutputStream.java:92)
     [rapc]     at sun.tools.jar.Main.addFile(Main.java:713)
     [rapc]     at sun.tools.jar.Main.create(Main.java:466)
     [rapc]     at sun.tools.jar.Main.run(Main.java:180)
     [rapc]     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     [rapc]     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
     [rapc]     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
     [rapc]     at java.lang.reflect.Method.invoke(Method.java:597)
     [rapc]     at net.rim.tools.compiler.c.e.if(Unknown Source)
     [rapc]     at net.rim.tools.compiler.c.e.a(Unknown Source)
     [rapc]     at net.rim.tools.compiler.Compiler.a(Unknown Source)
     [rapc]     at net.rim.tools.compiler.Compiler.a(Unknown Source)
     [rapc]     at net.rim.tools.compiler.Compiler.compile(Unknown Source)
     [rapc]     at net.rim.tools.compiler.Compiler.main(Unknown Source)

     [rapc] java.io.IOException: jar command failed: jar -cfmv C:\development\ant\new_test\MyApp\build\MyApp.jar C:\Users\Richard\AppData\Local\Temp\rapc_598c0c5a.dir\META-INF\MANIFEST.MF MyApp.cod MyApp-1.cod MyApp-2.cod MyApp.csl MyApp.cso -C C:\Users\Richard\AppData\Local\Temp\rapc_598c2ad7.dir .
     [rapc]     at net.rim.tools.compiler.Compiler.a(Unknown Source)
     [rapc]     at net.rim.tools.compiler.Compiler.a(Unknown Source)
     [rapc]     at net.rim.tools.I/O Error: jar command failed: jar -cfmv C:\development\ant\new_test\MyApp\build\MyApp.jar C:\Users\Richard\AppData\Local\Temp\rapc_598c0c5a.dir\META-INF\MANIFEST.MF MyApp.cod MyApp-1.cod MyApp-2.cocompiler.Compiler.compile(Unknown Source)
     [rapc]     at net.rim.tools.compiler.Compiler.main(Unknown Source)
     [rapc] d MyApp.csl MyApp.cso -C C:\Users\Richard\AppData\Local\Temp\rapc_598c2ad7.dir .

BUILD FAILED
C:\development\ant\new_test\MyApp\build.xml:65: Java returned: -1
        at org.apache.tools.ant.taskdefs.Java.execute(Java.java:111)
        at ca.slashdev.bb.tasks.RapcTask.executeRapc(RapcTask.java:583)
        at ca.slashdev.bb.tasks.RapcTask.execute(RapcTask.java:401)
        at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
        at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
        at org.apache.tools.ant.Task.perform(Task.java:348)
        at org.apache.tools.ant.Target.execute(Target.java:390)
        at org.apache.tools.ant.Target.performTasks(Target.java:411)
        at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1399)
        at org.apache.tools.ant.Project.executeTarget(Project.java:1368)
        at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
        at org.apache.tools.ant.Project.executeTargets(Project.java:1251)
        at org.apache.tools.ant.Main.runBuild(Main.java:809)
        at org.apache.tools.ant.Main.startAnt(Main.java:217)
        at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
        at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

Total time: 9 seconds

C:\development\ant\new_test\MyApp>

---

Workaround

Esaj has a good answer below that runs the Ant replace task on the source code before doing the compile. This means that there is no need to run jarjar on any resulting JAR file to solve the namespace problems.

This will work for me, as I have the SDK source code. It will not work for my clients, because I distribute my SDK in JAR format. So I'm still hoping for an answer to this question as it stands.

---

Related

This is a duplicate of [ How to convert JAR FILE to COD file using Ant Build ] but that question has not been answered, and I have added more detail.

This is not a duplicate of:

• [ Convert a .jar file into a .cod file using bb-ant ] - I have used the accepted answer, and my error is different.
• [ BlackBerry - Ant script to include JAR in project without external dependancies ] - this question is about adding the library in as a JAR instead of using the source code as I do above. In a way, it is a precursor step, as its output would still need to be run through jarjar.

Torment

Are you using srcdir to indicate folder containing jar? – Susysuter

If so, try using src and fileset instead. – Susysuter

I have been using src and fileset already, as suggested. – Torment

Answer 1

To answer my own question with some details...

One must not call rapc multiple times - it will create too many COD files. That is why I was getting that error.

Following up from Michael's answer, the correct way to proceed is to build the final JAR file using the normal java tools (javac & jar) as well as RIM's preverify command.

Only use rapc for the last step - converting that JAR file into a COD.

A full ANT build framework to deal with this problem is too big to place here, but the steps needed to create it are listed below. Each of the steps can be easily researched on this site (or with some google). Each step is very simple, and can be debugged individually.

Steps:

1. javac the SDK to create CLASS files
2. preverify the CLASS files
3. jar the SDK
4. Copy the SDK JAR file into the project
5. javac the project - use the SDK JAR as the classpath
6. preverify the project CLASS files (again, use the SDK JAR in the classpath)
7. jar the project - add the SDK JAR as a zipfileset
8. jarjar this project JAR to refactor package names as required
9. Finally, run rapc on this JAR - it will find no duplicate COD files & should run fine.

Note: Steps 1-3 can be combined by just using rapc on the SDK (which is needed if you need to run preprocessor tags on the SDK code).

By breaking it down into simple steps like this, I learnt how the normal java tools link into RIM's toolchain (normally this is all hidden when you simply invoke rapc on a source folder).

Of course, you still need to sign the COD with the sigtool.

I do all of this in ANT. I use a different folder to store the output from each step as I go. This way I end up with 5 temp folders at the end, but it made it easy to debug the steps as I go.

---

I finally understand now why so few people were able to offer conclusive answers to my various BB ANT build script questions. The process is laborious and very long, and hard to explain.

A full ANT build framework to accomplish this can stretch over many different files (in my case I think I'm now using 8 including property files). And it requires a good working knowledge of ANT, normal java build tools, and RIM's rapc command.

I think I have documented each step of the process quite well in my questions about this, and picked up some great answers along the line. For more details, have a look at those other questions & answers. Each one contains useful links, and some good insight from the other developers in this community

Torment

Answer 2

Copying this answer from BlackBerry - Ant script to include JAR in project without external dependancies as per @RichardLeMesurier's request:

I had a similar problem last year, I had created a "framework" that was used as a base for multiple BB-applications, but run into problems with multiple CODs (I don't remember exactly what, something like the device refused to install multiple applications that had same external cods, if the external CODs weren't installed separately first, and then the applications). As the applications could be installed separately (one person might install only app A, another might install only app B, and yet another might install both A and B), the whole framework needed to be included in all the apps using it. I cooked up this Ant-script using bb-ant-tools (hopefully I didn't break anything removing some stuff specific to our apps and obfuscating package names etc):

<?xml version="1.0" encoding="UTF-8"?>

<project name="${description}" default="build" basedir=".">

    <taskdef resource="bb-ant-defs.xml" classpath="lib/bb-ant-tools.jar" />

    <!-- rapc and sigtool require the jde.home property to be set -->
    <!-- NOTE: You may need to copy the signature files from Eclipse\plugins\net.rim.ejde\vmTools to the components\bin -dir 
        if the keys were installed using the Eclipse-plugin     -->
    <property name="jdehome" value="C:\BB\Eclipse\plugins\net.rim.ejde.componentpack5.0.0_5.0.0.25\components" />

    <!-- Framework source locations, these must be set correctly -->
    <property name="frameworkRes.dir" value="C:\BB\workspace\BB_Framework\res" />
    <property name="frameworkSrc.dir" value="C:\BB\workspace\BB_Framework\src\com\whatever\frame" />

    <!-- Locations for simulator, binaries, jde home, don't touch these -->
    <property name="simulator" value="${jdehome}\simulator" />
    <property name="bin" value="${jdehome}\bin" />
    <property name="jde.home" location="${jdehome}" />

    <!-- directory of simulator to copy files to -->
    <property name="simulator.home" location="${simulator}" />

    <property name="src.dir" location="src" />
    <property name="build.dir" location="build" />
    <property name="temp.dir" location="C:\tempsrc" />

    <!-- Project specific -->
    <!-- Application title -->
    <property name="app.title" value="Application Name" />
    <property name="app.version" value="1.0.0" />
    <!-- Value to prepend before frame-class packages -->
    <property name="frame.prefix" value="appname" />
    <!-- Name of the COD to produce --> 
    <property name="cod.name" value="Appname" />

    <target name="build">
        <mkdir dir="${build.dir}" />
        <delete dir="${temp.dir}" />
        <mkdir dir="${temp.dir}" />
        <mkdir dir="${temp.dir}\${frame.prefix}" />

        <copy toDir="${temp.dir}">
            <fileset dir="${src.dir}">
                <include name="**/*.java" />
            </fileset>
        </copy>

        <copy toDir="${temp.dir}\${frame.prefix}">
            <fileset dir="${frameworkSrc.dir}">
                <include name="**/*.java" />
            </fileset>
        </copy>

        <copy toDir="${temp.dir}\res">
            <fileset dir="${frameworkRes.dir}">
                <include name="**/*" />
            </fileset>
        </copy>

        <copy toDir="${temp.dir}\res">
            <fileset dir="res">
                <include name="**/*" />
            </fileset>
        </copy>

        <!-- This replaces the package names for classes copied from under 
            framework-directory to ${frame.prefix} -directory as well as changing any 
            imports using the classes in framework-package -->  
        <replace dir="${temp.dir}" value="${frame.prefix}">
            <include name="**/*.java"/>
            <replacetoken>com.whatever.frame</replacetoken>
        </replace>

        <rapc output="${cod.name}" srcdir="${temp.dir}" destdir="${build.dir}">
            <jdp title="${app.title}" 
                version="${app.version}" 
                vendor="Your Company"
                icon="../res/img/icon.png"
            />
        </rapc>
    </target>

    <target name="sign">
        <sigtool codfile="${build.dir}/${cod.name}.cod" />
    </target>

    <target name="clean">
        <delete dir="${build.dir}" />
    </target>

    <target name="load-simulator" depends="build">
        <copy todir="${simulator.home}">
            <fileset dir="${build.dir}" includes="*.cod,*.cso,*.debug,*.jad,*.jar" />
        </copy>
    </target>

</project>

What this does, is copy all the java-files and resources from your current project and then from the framework-project to a temporary directory, replacing package names on the way for the frame-work files (as they're put into a separately named directory), this has to do with the fact that the devices also refused to install multiple applications that had same classes under same packages (namely, the framework classes, for your case this might not be necessary). After the copying and replacing is done, the application is built to target build-directory using rapc. There are separate tasks for signing, cleaning and loading the app to a simulator. Hope this helps.

Viva

This is a very useful answer, as it provides a very good workaround - by running replace, there is no need to run jarjar. Caveat - this only works if you have access to the library source code (like I do). – Torment

Answer 3

From your console output, the command that is failing:

jar -cfmv C:\development\ant\new_test\MyApp\build\MyApp.jar C:\Users\Richard\AppData\Local\Temp\rapc_598c0c5a.dir\META-INF\MANIFEST.MF MyApp.cod MyApp-1.cod MyApp-2.cod MyApp.csl MyApp.cso -C C:\Users\Richard\AppData\Local\Temp\rapc_598c2ad7.dir .

Looking up those options in the jar tool documentation:

-C dir
Temporarily changes directories (cd dir) during execution of the jar command while processing the following inputfiles argument. Its operation is intended to be similar to the -C option of the UNIX tar utility.

Based on that, I think rapc is putting the unpacked cod file in C:\Users\Richard\AppData\Local\Temp\rapc_598c2ad7.dir and that is causing conflicts with the cod files specified on the command line. Is that directory still around? Look inside to see what's there.

Taro

1) the dir contains all the image & text resources for the app, PLUS the COD files as you expected - so that is WHY this is happening. 2) this command is called automatically by bb-ant-tools' rapc command - maybe my INPUT JAR should not include the COD files. Would you create that JAR file using normal jar command, and then run rapc as the final step only? – Torment

@michael thx for the hint - I've broken the build into individual steps and got it working. Detailed in my self-answer. – Torment