Gradle multiproject optional subproject's transitive dependency should be resolved to an existing subproject

Question

suppose the following project. the master project is a multi-project, however every part of the larger project can be developed individually or mixed in:

/master/build.gradle
/m1/build.gradle
/m2/build.gradle
/m3/build.gradle

suppose m3 uses m2 and m2 uses m1 ( m1 <- m2 <- m3 )

the presence of m2 is optional a multi-project with the following layout also reasonable

/master/build.gradle
/m1/build.gradle
/m3/build.gradle

but in this case m2 would be pulled in from the artifact repository which is fine...however m1 was a transitive dependency of m2 which is good, but how can i tell gradle to use the local version of m1 instead of the baked artifact?

I'm stuck with this, every place i have access to override these thing gradle gives me "just" ModuleVersionSelector level access, how can i add a DefaultProjectDependency according to the downloaded artifact transitive dependencies?

i may have an alternative if i can access the full dependency graph of the archived artifacts, and put in some overrides/excludes.

EDIT:

the best i've come up with is using a filter using resolutionStrategy, i've created an example by further developing the 'elastic-deps' project

https://github.com/kgyrtkirk/elastic-deps

Chaddy

Answer 1

Starting with elastic-deps and with the help of this answer (also from Peter) I came up with the trick below.

In the top-level build.gradle():

// make sure we've parsed the subproject dependencies
evaluationDependsOnChildren()

def subprojectsByName = subprojects.collectEntries { it -> [it.name, it] }

subprojects.each { p ->
  def hacks = [] // list of changes we're going to make
  p.configurations.each { c ->
    c.dependencies.each { d ->
      if (d.group.startsWith("my.group.prefix")) {
        def sub = subprojectsByName[d.name]
        if (sub != null) {
          hacks.add({
            // can't do this immediately or we'll get ConcurrentModificationExceptions
            c.dependencies.remove(d)
            p.dependencies.add(c.name, sub)
          })
        }
      }
    }
  }
  // Now we can safely apply the changes
  for (hack in hacks) {
    hack()
  }
}

The nice thing about this is that unlike elastic-deps you don't have to modify the subprojects.

This still has the problem that once you hit a binary dependency, any purely transitive dependencies are resolved as binary. E.g., say I have a project cyan which depends directly on green and blue and transitively, through green, on yellow:

compile - Compile classpath for source set 'main'.
+--- my.shared:blue:+ -> 2.0-SNAPSHOT
+--- my.shared:green:+ -> 2.0-SNAPSHOT
|    +--- my.shared:yellow:+ -> 2.0-SNAPSHOT
|    \--- my.shared:blue:+ -> 2.0-SNAPSHOT

Now if I add blue and yellow to my multi-module project, but not green, I get:

compile - Compile classpath for source set 'main'.
+--- com.iii.shared:green:+ -> 2.0-SNAPSHOT
|    +--- com.iii.shared:yellow:+ -> 2.0-SNAPSHOT
|    \--- com.iii.shared:blue:+ -> project :blue
\--- project :blue

Note that blue is resolved correctly to the project even when it's transitive, but yellow isn't.

Personally I think this is a feature, not a bug -- it reflects what would actually happen at distribution time. I can make all the changes to yellow I want, but if I don't put a new yellow artifact in my repository and also an updated green with the updated dependency, then the actual release of cyan isn't going to get those changes.

Premonition

Answer 2

Working with a dynamic subset of a Gradle build is a planned feature. In the meantime, the best solution I've come up with is to introduce a new dependency notation that gets dynamically mapped to either a project dependency or an external dependency. You can find a proof-of-concept here: https://github.com/pniederw/elastic-deps

PS: Before embarking on implementing this feature on your own, reconsider if you truly need it at this point. You might save yourself some headaches by waiting until it is officially supported.

Catchpenny

we already implemented it...we're exploring the possibilities of having sub2 from binary artifact and in that case sub3's transitivity is resolved to binary...which isn't desired – Millikan

OK, you didn't mention that. I'm not sure if and how well this can be achieved with current Gradle. I'm afraid you'll have to dig deeper yourself. – Catchpenny