How come maven dependencies in POM file can't be used?

Asked 7/4, 2021 at 10:0 Answered 13/4, 2021 at 8:36

Solved android maven pom.xml dependency-management jitpack

Background

I'm tasked to provide a way for developers to add a single dependency into build.gradle file on Android project, to use some (obfuscated) SDK I've prepared (AAR file - Android library).

The SDK itself uses various dependencies, so it's important that they will be available in the final code, meaning the whoever uses the SDK won't see crashes due to class-not-found.

The SDK is stored on Github as a private repository, and available for developers via Jitpack, which scans POM file and some other files on the repository.

Using the AAR file alone, you have to set the same (or newer) dependencies as on the SDK.

However, if you use POM file while publishing, the consumer (whoever uses the SDK) shouldn't need to write each dependency that's on the SDK, as it gets added from the POM file.

The problem

For some reason, this behavior doesn't exist for me. Meaning that it's as if the SDK doesn't use any depdendency, so if I try to use any of the depdendencies I can't reach the classes, and if I try to use the SDK when it uses some depdendency, it causes an exception of ClassNotFoundException.

At first I thought it's because of some Proguard rules, but then I noticed that it happens on debug-variant too (of the app that uses the SDK).

So for example, as the library uses this in the dependencies:

implementation 'androidx.security:security-crypto:1.1.0-alpha03'

And if I don't use the same, I can't reach any class of this dependency, and if the SDK tries to reach any class of it, the app will crash.

So instead of a single dependency of using the SDK and that's it, I have to add all of the dependencies that the SDK already uses.

What I've tried

To generate the AAR file, I use "assembleRelease" gradle task, and to generate the POM file, I use the "generatePomFileForReleasePublication" (or "publishReleasePublicationToMavenLocal") gradle task.

The library has quite some dependencies. This is the build.gradle file of it:

plugins {
    id 'com.android.library'
    id 'kotlin-android'
    id 'kotlin-kapt'
    id 'kotlin-parcelize'
    id 'io.michaelrocks.paranoid'
    id 'maven-publish'
    id 'com.github.dcendents.android-maven'
}

android {
    compileSdkVersion 30
    defaultConfig {
        minSdkVersion 21
        targetSdkVersion 30
        versionCode 1
        versionName "1.0"
    }

    buildTypes {
        release {
            minifyEnabled true
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
            consumerProguardFiles 'consumer-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
    kotlinOptions {
        jvmTarget = '1.8'
    }
    configurations {
        all {
            exclude module: 'httpclient'
        }
    }
    packagingOptions {
        //for google login token
        exclude 'META-INF/DEPENDENCIES'
    }
}

dependencies {
    api "org.jetbrains.kotlin:kotlin-stdlib:$kotlin_version"
    api 'androidx.core:core-ktx:1.6.0-alpha01'
    api 'androidx.collection:collection-ktx:1.2.0-alpha01'
    api 'androidx.security:security-crypto:1.1.0-alpha03'
    def room_version = "2.2.6"
    api "androidx.room:room-runtime:$room_version"
    kapt "androidx.room:room-compiler:$room_version"
    api "androidx.room:room-ktx:$room_version"
    api 'com.squareup.retrofit2:retrofit:2.9.0'
    api 'com.squareup.retrofit2:converter-gson:2.9.0'
    api 'com.squareup.okhttp3:okhttp:5.0.0-alpha.2'
    api 'com.squareup.okhttp3:logging-interceptor:5.0.0-alpha.2'
    api('com.google.android.gms:play-services-auth:19.0.0')
    api('com.google.auth:google-auth-library-oauth2-http:0.25.0')
    api('com.google.apis:google-api-services-people:v1-rev99-1.22.0') {
        exclude group: 'com.google.guava'
    }
    api 'io.michaelrocks:libphonenumber-android:8.12.20'
}

afterEvaluate {
    publishing {
        publications {
            release(MavenPublication) {
                from components.release
            }
        }
    }
}

Running the task of "generatePomFileForReleasePublication" I got the POM file "pom-default.xml" :

<?xml version="1.0" encoding="UTF-8"?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <!-- This module was also published with a richer model, Gradle metadata,  -->
  <!-- which should be used instead. Do not delete the following line which  -->
  <!-- is to indicate to Gradle or any Gradle module metadata file consumer  -->
  <!-- that they should prefer consuming it instead. -->
  <!-- do_not_remove: published-with-gradle-metadata -->
  <modelVersion>4.0.0</modelVersion>
  <groupId>MySdk</groupId>
  <artifactId>library</artifactId>
  <version>unspecified</version>
  <packaging>aar</packaging>
  <dependencies>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-stdlib</artifactId>
      <version>1.4.32</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.core</groupId>
      <artifactId>core-ktx</artifactId>
      <version>1.6.0-alpha01</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.collection</groupId>
      <artifactId>collection-ktx</artifactId>
      <version>1.2.0-alpha01</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.security</groupId>
      <artifactId>security-crypto</artifactId>
      <version>1.1.0-alpha03</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.room</groupId>
      <artifactId>room-runtime</artifactId>
      <version>2.2.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>androidx.room</groupId>
      <artifactId>room-ktx</artifactId>
      <version>2.2.6</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.retrofit2</groupId>
      <artifactId>retrofit</artifactId>
      <version>2.9.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.retrofit2</groupId>
      <artifactId>converter-gson</artifactId>
      <version>2.9.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>okhttp</artifactId>
      <version>5.0.0-alpha.2</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.squareup.okhttp3</groupId>
      <artifactId>logging-interceptor</artifactId>
      <version>5.0.0-alpha.2</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.google.android.gms</groupId>
      <artifactId>play-services-auth</artifactId>
      <version>19.0.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.google.auth</groupId>
      <artifactId>google-auth-library-oauth2-http</artifactId>
      <version>0.25.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>com.google.apis</groupId>
      <artifactId>google-api-services-people</artifactId>
      <version>v1-rev99-1.22.0</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
        <exclusion>
          <artifactId>*</artifactId>
          <groupId>com.google.guava</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>io.michaelrocks</groupId>
      <artifactId>libphonenumber-android</artifactId>
      <version>8.12.20</version>
      <scope>compile</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-parcelize-runtime</artifactId>
      <version>1.4.32</version>
      <scope>runtime</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>io.michaelrocks</groupId>
      <artifactId>paranoid-core</artifactId>
      <version>0.3.3</version>
      <scope>runtime</scope>
      <exclusions>
        <exclusion>
          <artifactId>httpclient</artifactId>
          <groupId>*</groupId>
        </exclusion>
      </exclusions>
    </dependency>
  </dependencies>
</project>

According to various websites, this seems valid and good, but sadly as I wrote, when I use it, it's as if none of these dependencies exist, and I need to declare them all on the consumer side.

In Jitpack's file ("jitpack.yml" ) , I have something like this:

install: 
  - FILE="-Dfile=library-release.aar" 
  - mvn install:install-file $FILE -DgroupId=my-sdk -DartifactId=MySdk -Dversion=1.0 -Dpackaging=aar -DgeneratePom=true

What I tried:

I tried using either "implementation" and "api". Didn't help.
I tried to remove the scope from all of the dependency tags. Didn't help.
I tried adding transitive = true on consumer side, but this didn't do anything, probably because it's as such by default anyway.
Removing the POM file showed no warning on Jitpack, showing that it might not even check it out.
I've noticed the POM file mentions "This module was also published with a richer model, Gradle metadata, which should be used instead", so I tried finding this file, but couldn't. Running publishToMavenLocal task, I got another file ("module.json"), and even putting it instead of just pom-default.xml file alone (and also as a complete replacement), it didn't help.
I thought that there is some inconsistency between the file of "jitpack.yml" and the gradle file, so I've set on both of them the same values for "groupId","artifactId", and "version".

The questions

How come the dependencies are ignored on consumer side? How can I fix this? It's probably something very simple that I've missed...
Which files exactly does Jitpack need to have? Which are mandatory? Which are optional?
Would it also be possible to avoid the part of packagingOptions {exclude 'META-INF/DEPENDENCIES' } on the consumer's side ?
Bonus: I've noticed that JavaDocs (or Kdocs in my case, as I wrote everythign in Kotlin) are also completely ignored. How can I make these reachable and readable too?

Jerriejerrilee answered 7/4, 2021 at 10:0 Comment(6)

Hey, sorry for the delayed reply but I just read through your question and I'd like to know one thing from your side. When you said SDK is published on Jitpack, is that hosted somewhere secure or it's publicly available? Meaning if it's hosted in proxied/private env. like internal servers it's possible that gradle is not able to resolve transitive dependencies. I've seen same issues in my organization where our nexus repository had two urls, one started with maven-public packages and one with maven-release but release one somehow ended up not resolving all deps. – Carlos 8/4, 2021 at 20:13

Second thing I see is you're using gradle task to generate POM files which are enough in my opinion to resolve transitive deps on consumer side, meaning issue relies somewhere with your Jitpack hosting. Also, I'm not sure about packaging options but for docs you can try with JavaDocs type. here's the reference: gist.github.com/Robyer/a6578e60127418b380ca133a1291f017 – Carlos 8/4, 2021 at 20:22

I wrote "The SDK is stored on Github as a private repository" . Github stores the SDK files. Jitpack has access to there (had to authorize access). Jitpack has full access to all the repositories and files there now. Jitpack is like a mediator. It looks at the files of the repository, and generates the dependency for you. In the case of open sourced Android libraries, this is very easy: you put the URL and that's it. Example of one of my own: github.com/AndroidDeveloperLB/AutoFitTextView . If you put the URL into Jitpack's website, you will get the dependency you need to use. That's it – Jerriejerrilee 8/4, 2021 at 21:3

So, for open sourced cases, it works perfectly. On this case of the issue, though, it's not open sourced. I have generated an obfuscated AAR file, and tried to generate POM file too (and docs, but let's leave it as a bonus for now). Jitpack couldn't make the dependencies optional. Now to use the SDK, the person who uses it has to put all the dependencies that the SDK uses (and you can see it's quite a lot). – Jerriejerrilee 8/4, 2021 at 21:5

Yep, I know how Jitpack works but in your case of SDK like you shared your AutoFitTextView, can I access SDK just by putting artifact name in my own app or I need to have credentials for accessing it. If it needs authorization, that's where something is wrong because Gradle is not able to breach through transitive deps. – Carlos 9/4, 2021 at 6:54

@JeelVankhede As I wrote, Jitpack has full access to the repository on Github (whether it's public like the AutoFitTextView case, or private like the SDK case). Access to repositories is not the issue here. Whoever wants to use the AutoFitTextView library just has to put what Jitpack says on the website (which is just one line in gradle file, if you ignore the part of classpath). For private repositories it's a bit different, but still this is not the issue. – Jerriejerrilee 9/4, 2021 at 8:21

OK I've found the way to do this. The answer came from Jitpack support itself. First, the quote from them:

This is caused by the install command and ' -DgeneratePom=true' parameter. It will generate a new pom file but it doesn't know anything about dependencies.

If you already have a pom file with all the dependencies then you could add it to the GitHub repository. Instead of ' -DgeneratePom=true' you could use '-DpomFile='

We recommend trying that command locally before running on JitPack.

The other option is to put all the source code of the library on JitPack and build from source.

And indeed, if I use this (and have "api" in the library's dependencies) I got most of my questions answered:

It fixed the dependencies issue (and yes, you don't have to customize the -DgroupId=my-sdk -DartifactId=MySdk -Dversion=1.0 ) :

install: 
  - FILE="-Dfile=library-release.aar" 
  - mvn install:install-file $FILE -Dpackaging=aar -DpomFile=pom-default.xml

The files that are currently there seem to be just those: jitpack.yml, library-release.aar, pom-default.xml .
Apparently it's not needed now. Probably fixed by the same thing that fixed the dependencies issue.

That being said, now I have these questions about it:

What's with the recommendation inside "pom-default.xml" to use a different file ("This module was also published with a richer model, Gradle metadata, which should be used instead.") ? Where is it? Is it perhaps the "module.json" file? I tried to use it instead, but then Jitpack got an error with it.
Where is the documentation of Jitpack of this use case? They have something about private repositories, but not about AAR files. In fact there is nothing I could see about the "jitpack.yml" file.
What's the purpose of customizing the -DgroupId=my-sdk -DartifactId=MySdk -Dversion=1.0 part? Either in the "jitpack.yml" file or in the gradle file of the library? I don't think they do anything at all... It seems the Github repository itself is the one that determines to the consumer how to use it.
How can I also add JavaDocs and Kdocs (of Kotlin) to be used here?

Even though this is the main answer for my questions, I will grant the bounty for whoever answers me well for the rest :)

Jerriejerrilee answered 13/4, 2021 at 8:36 Comment(4)

Thanks @androiddeveloper, I have been looking around for this info for months. Jitpack has poor documentation on private repo and publishing aar/jar. – Schoonmaker 18/5, 2021 at 16:25

@AkashRaj You're welcomed. Later I tried to figure out how to add Kdocs (like Java-Docs, but for Kotlin), and failed, but someone answered when I couldn't check it out anymore: https://mcmap.net/q/139019/-how-do-you-publish-kdoc-for-a-kotlin-library-using-maven-on-jitpack/878126 Please have a look and if you do this, let me know if indeed it works. – Jerriejerrilee 19/5, 2021 at 9:14

Im gonna bookmark this. Currently, the project is in Java. If we migrate to Kotlin, I will give it a try. Thanks for suggesting. @androiddeveloper – Schoonmaker 20/5, 2021 at 10:52

@AkashRaj Wait, this issue doesn't exist for Java? How do you do this for Java? It's also interesting... – Jerriejerrilee 21/5, 2021 at 19:17

I believe the problem is with your jitpack.yml. Jitpack's default install command relies on maven-publish and thus uses the generated pom.xml. You override it with the custom one that doesn't know anything about your dependencies and cannot generate a correct pom.xml because the command is Maven-based and your project is Gradle-based.

Remove jitpack.yml or its install section and everything should work fine.

UPDATE:

You have to configure Maven publication in the Gradle configuration in order to make it all work. To do that you have to apply maven-publish plugin

apply plugin: 'maven-publish'

and configure a publication

afterEvaluate {
    publishing {
        publications {
            release(MavenPublication) {
                from components.release
                groupId = 'my-sdk'
                artifactId = 'MySdk'
                version = '1.0'
            }
        }
    }
}

Please take a look at the official Jitpack's Android sample. They configure Maven publication in build.gradle and use neither jitpack.yml, nor maven utility.

Pedaiah answered 11/4, 2021 at 19:41 Comment(7)

Deletion of jitpack.yml didn't work. It took 40 minutes and only then Jitpack wrote

No build file found. Looking for Gradle/Maven/Sbt/Lein build file in the root of the project See the documentation at https://jitpack.io/docs/

(and of course nothing on this link talks about this file). What exactly do you say I could change in the file, instead of deleting it? Also, maybe the files names are incorrect and can't be handled? Currently the files that I get are "library-release.aar", "module.json", "pom-default.xml" . Which do I need, and in which names? – Jerriejerrilee 12/4, 2021 at 7:26

"Looking for Gradle/Maven/Sbt/Lein build file in the root of the project" means you should have build.gradle in the root and you probably do. Doesn't Jitpack publish your artifact when you remove jitpack.yml? – Pedaiah 12/4, 2021 at 15:53

When I remove the "jitpack.yml" (and then commit&push the changes) and tell Jitpack to check the repository, it freezes for 40 minutes and then gives me the error I've written. – Jerriejerrilee 12/4, 2021 at 17:24

I don't have any experience with Jitpack but their sample doesn't have jitpack.yml, which I believe is correct. – Pedaiah 12/4, 2021 at 20:4

Take a look at their sample. It uses maven-publish plugin and this plugin is necessary for successful publication with Jitpack. – Pedaiah 12/4, 2021 at 20:9

But the sample is open sourced (has Java/Kotlin inside). What I work with is an (obfuscated) AAR file . I already know how to handle simple, open sourced Android-libraries on Github . I also wrote about this on the comments: #66984261 . As for the plugin, look at the gradle file I wrote. It already uses it (id 'maven-publish'). – Jerriejerrilee 13/4, 2021 at 6:35

I got it right and added my answer. If you somehow know the answers to what I wrote there, I can still grant the bounty (but won't mark as accepted answer as what I wrote is the answer to the main issues I've presented). – Jerriejerrilee 13/4, 2021 at 8:49

Hot tags

Godot Unity Godot Help Programming Godot 4.X GUI GDScript 3D 2D Physics CSharp Godot 3.X VR XR Projects C++

Background

The problem

What I've tried

The questions

Recommended topics

Hot tags