Duplicate class error when using Firestore and Google Speech to Text

Question

When attempting to build project with both Firestore and the Google Speech to Text libraries, I get a "Duplicate class" error. From what I understand, this is due to both libraries introducing proto libraries. Exclusions produce runtime errors. How do I resolve the duplication issue?

Here is the error (concatenated, there are hundreds more lines):

Duplicate class com.google.api.Advice found in modules jetified-proto-google-common-protos-1.17.0.jar (com.google.api.grpc:proto-google-common-protos:1.17.0) and jetified-protolite-well-known-types-17.0.0-runtime.jar (com.google.firebase:protolite-well-known-types:17.0.0)
Duplicate class com.google.api.Advice$1 found in modules jetified-proto-google-common-protos-1.17.0.jar (com.google.api.grpc:proto-google-common-protos:1.17.0) and jetified-protolite-well-known-types-17.0.0-runtime.jar (com.google.firebase:protolite-well-known-types:17.0.0)
Duplicate class com.google.api.Advice$Builder found in modules jetified-proto-google-common-protos-1.17.0.jar (com.google.api.grpc:proto-google-common-protos:1.17.0) and jetified-protolite-well-known-types-17.0.0-runtime.jar (com.google.firebase:protolite-well-known-types:17.0.0)

Here are my dependencies:

dependencies {
    implementation fileTree(dir: 'libs', include: ['*.jar'])
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation 'androidx.appcompat:appcompat:1.1.0'
    implementation 'androidx.core:core-ktx:1.2.0'
    implementation 'androidx.constraintlayout:constraintlayout:1.1.3'
    implementation 'androidx.recyclerview:recyclerview:1.1.0'
    implementation 'com.google.code.gson:gson:2.8.6'
    implementation 'com.google.firebase:firebase-analytics:17.4.1'
    implementation 'com.firebaseui:firebase-ui-auth:6.2.0'
    implementation 'com.google.firebase:firebase-firestore:21.4.3'

    // add these dependencies for the speech client
    implementation 'io.grpc:grpc-okhttp:1.29.0'
    implementation 'com.google.cloud:google-cloud-speech:1.23.0'

    testImplementation 'junit:junit:4.12'
    androidTestImplementation 'androidx.test.ext:junit:1.1.1'
    androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
}

If I add the following exclusions, compile works, but then I get a runtime error for Firestore:

configurations {
    implementation.exclude module:'protolite-well-known-types'
    implementation.exclude module:'protobuf-javalite'
}

Here is the runtime Firestore error:

java.lang.RuntimeException: Internal error in Cloud Firestore (21.4.3).
        at com.google.firebase.firestore.util.AsyncQueue.lambda$panic$3(com.google.firebase:firebase-firestore@@21.4.3:534)
        at com.google.firebase.firestore.util.AsyncQueue$$Lambda$3.run(Unknown Source:2)
        at android.os.Handler.handleCallback(Handler.java:873)
        at android.os.Handler.dispatchMessage(Handler.java:99)
        at android.os.Looper.loop(Looper.java:193)
        at android.app.ActivityThread.main(ActivityThread.java:6669)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:493)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:858)
     Caused by: java.lang.ExceptionInInitializerError
        at com.google.firestore.v1.ListenRequest.getDefaultInstance(com.google.firebase:firebase-firestore@@21.4.3:915)
        at com.google.firestore.v1.FirestoreGrpc.getListenMethod(com.google.firebase:firebase-firestore@@21.4.3:408)
        at com.google.firebase.firestore.remote.WatchStream.<init>(com.google.firebase:firebase-firestore@@21.4.3:61)
        at com.google.firebase.firestore.remote.Datastore.createWatchStream(com.google.firebase:firebase-firestore@@21.4.3:115)
        at com.google.firebase.firestore.remote.RemoteStore.<init>(com.google.firebase:firebase-firestore@@21.4.3:167)
        at com.google.firebase.firestore.core.FirestoreClient.initialize(com.google.firebase:firebase-firestore@@21.4.3:281)
        at com.google.firebase.firestore.core.FirestoreClient.lambda$new$0(com.google.firebase:firebase-firestore@@21.4.3:109)
        at com.google.firebase.firestore.core.FirestoreClient$$Lambda$1.run(Unknown Source:8)
        at com.google.firebase.firestore.util.AsyncQueue.lambda$enqueue$2(com.google.firebase:firebase-firestore@@21.4.3:436)
        at com.google.firebase.firestore.util.AsyncQueue$$Lambda$2.call(Unknown Source:2)
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor.lambda$executeAndReportResult$1(com.google.firebase:firebase-firestore@@21.4.3:322)
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$$Lambda$2.run(Unknown Source:4)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$DelayedStartFactory.run(com.google.firebase:firebase-firestore@@21.4.3:229)
        at java.lang.Thread.run(Thread.java:764)
     Caused by: java.lang.RuntimeException: Unable to get message info for com.google.firestore.v1.ListenRequest
        at com.google.protobuf.GeneratedMessageInfoFactory.messageInfoFor(GeneratedMessageInfoFactory.java:62)
        at com.google.protobuf.ManifestSchemaFactory$CompositeMessageInfoFactory.messageInfoFor(ManifestSchemaFactory.java:143)
        at com.google.protobuf.ManifestSchemaFactory.createSchema(ManifestSchemaFactory.java:55)
        at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:93)
        at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:107)
        at com.google.protobuf.GeneratedMessageLite.makeImmutable(GeneratedMessageLite.java:171)
        at com.google.firestore.v1.ListenRequest.<clinit>(com.google.firebase:firebase-firestore@@21.4.3:911)
        at com.google.firestore.v1.ListenRequest.getDefaultInstance(com.google.firebase:firebase-firestore@@21.4.3:915) 
        at com.google.firestore.v1.FirestoreGrpc.getListenMethod(com.google.firebase:firebase-firestore@@21.4.3:408) 
        at com.google.firebase.firestore.remote.WatchStream.<init>(com.google.firebase:firebase-firestore@@21.4.3:61) 
        at com.google.firebase.firestore.remote.Datastore.createWatchStream(com.google.firebase:firebase-firestore@@21.4.3:115) 
        at com.google.firebase.firestore.remote.RemoteStore.<init>(com.google.firebase:firebase-firestore@@21.4.3:167) 
        at com.google.firebase.firestore.core.FirestoreClient.initialize(com.google.firebase:firebase-firestore@@21.4.3:281) 
        at com.google.firebase.firestore.core.FirestoreClient.lambda$new$0(com.google.firebase:firebase-firestore@@21.4.3:109) 
        at com.google.firebase.firestore.core.FirestoreClient$$Lambda$1.run(Unknown Source:8) 
        at com.google.firebase.firestore.util.AsyncQueue.lambda$enqueue$2(com.google.firebase:firebase-firestore@@21.4.3:436) 
        at com.google.firebase.firestore.util.AsyncQueue$$Lambda$2.call(Unknown Source:2) 
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor.lambda$executeAndReportResult$1(com.google.firebase:firebase-firestore@@21.4.3:322) 
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$$Lambda$2.run(Unknown Source:4) 
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458) 
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$DelayedStartFactory.run(com.google.firebase:firebase-firestore@@21.4.3:229) 
        at java.lang.Thread.run(Thread.java:764) 
     Caused by: java.lang.UnsupportedOperationException
        at com.google.firestore.v1.ListenRequest.dynamicMethod(com.google.firebase:firebase-firestore@@21.4.3:903)
        at com.google.protobuf.GeneratedMessageLite.dynamicMethod(GeneratedMessageLite.java:252)
        at com.google.protobuf.GeneratedMessageLite.buildMessageInfo(GeneratedMessageLite.java:280)
        at com.google.protobuf.GeneratedMessageInfoFactory.messageInfoFor(GeneratedMessageInfoFactory.java:60)
        at com.google.protobuf.ManifestSchemaFactory$CompositeMessageInfoFactory.messageInfoFor(ManifestSchemaFactory.java:143) 
        at com.google.protobuf.ManifestSchemaFactory.createSchema(ManifestSchemaFactory.java:55) 
        at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:93) 
        at com.google.protobuf.Protobuf.schemaFor(Protobuf.java:107) 
        at com.google.protobuf.GeneratedMessageLite.makeImmutable(GeneratedMessageLite.java:171) 
        at com.google.firestore.v1.ListenRequest.<clinit>(com.google.firebase:firebase-firestore@@21.4.3:911) 
        at com.google.firestore.v1.ListenRequest.getDefaultInstance(com.google.firebase:firebase-firestore@@21.4.3:915) 
        at com.google.firestore.v1.FirestoreGrpc.getListenMethod(com.google.firebase:firebase-firestore@@21.4.3:408) 
        at com.google.firebase.firestore.remote.WatchStream.<init>(com.google.firebase:firebase-firestore@@21.4.3:61) 
        at com.google.firebase.firestore.remote.Datastore.createWatchStream(com.google.firebase:firebase-firestore@@21.4.3:115) 
        at com.google.firebase.firestore.remote.RemoteStore.<init>(com.google.firebase:firebase-firestore@@21.4.3:167) 
        at com.google.firebase.firestore.core.FirestoreClient.initialize(com.google.firebase:firebase-firestore@@21.4.3:281) 
        at com.google.firebase.firestore.core.FirestoreClient.lambda$new$0(com.google.firebase:firebase-firestore@@21.4.3:109) 
        at com.google.firebase.firestore.core.FirestoreClient$$Lambda$1.run(Unknown Source:8) 
        at com.google.firebase.firestore.util.AsyncQueue.lambda$enqueue$2(com.google.firebase:firebase-firestore@@21.4.3:436) 
        at com.google.firebase.firestore.util.AsyncQueue$$Lambda$2.call(Unknown Source:2) 
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor.lambda$executeAndReportResult$1(com.google.firebase:firebase-firestore@@21.4.3:322) 
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$$Lambda$2.run(Unknown Source:4) 
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:458) 
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301) 
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167) 
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641) 
        at com.google.firebase.firestore.util.AsyncQueue$SynchronizedShutdownAwareExecutor$DelayedStartFactory.run(com.google.firebase:firebase-firestore@@21.4.3:229) 
        at java.lang.Thread.run(Thread.java:764) 

Lockup

Answer 1

In my case, I was using DialogFlow with Firestore.

Using below dependency was enough for me

   implementation('com.google.firebase:firebase-firestore:21.4.3') {
    exclude module: 'protolite-well-known-types'
    exclude module: 'protobuf-javalite'
    exclude module: 'protobuf-java'
    exclude module: 'protobuf-java-util'
}

Killifish

Although it removes the error at build time, but if you are using some of the dependents that you just excluded here you will start seeing the error of missing classes at build time. The only solution is to resolve the duplicates manually and import the jar – Pitta

Answer 2

I've figured it out. The Google Speech-to-Text Java library does not currently support Android, hence the error. From what I understand, protobuf-java (used by Google Speech-to-Text) is used for desktop solutions and protobuf-lite (used by Firestore) is used for mobile solutions. Excluding either module will break their respective dependents.

My workaround was to create my own .jar that included classes from both, including that .jar in my project, then excluding the modules from their dependents in my build.gradle. This will ensure that I have the right modules in my project, but also that I will only have one copy of the classes.

Steps I took:

1. Copy each .jar with conflicts to a new folder (doesn't matter where)
2. Change extension to .zip
3. Extract the contents in each zip to a single folder. When asked to overwrite, do so.
4. IMPORTANT: make sure the last zips you extract are protolite-well-known-types and protobuf-javalite, this will ensure that Firestore has the right libraries. Make sure they overwrite any existing files.
5. Zip the contents of the folder.
6. Change the file extension to .jar
7. Add the new .jar to your /lib folder
8. Exclude the modules from their dependents in your build.gradle

   implementation('com.google.firebase:firebase-firestore:21.4.3') {
        exclude module: 'protolite-well-known-types'
        exclude module: 'protobuf-javalite'
        exclude module: 'protobuf-java'
        exclude module: 'protobuf-java-util'
    }

    implementation('com.google.cloud:google-cloud-speech:1.23.0') {
        exclude module: 'protolite-well-known-types'
        exclude module: 'protobuf-javalite'
        exclude module: 'proto-google-common-protos'
        exclude module: 'protobuf-java'
        exclude module: 'protobuf-java-util'
    }

EDIT:

This does not work fully. It allows the project to compile and calls to Firestore to run, but will produce a runtime error when attempting to make any method calls utilizing the Google Speech-to-Text library.

As of now, I do not think there is any way to use Firebase with the Google Speech-to-Text library, as Google Speech-to-Text is not yet supported by Android.

Lockup

You can implement Google Speech-to-text without a library using this sample (github.com/GoogleCloudPlatform/android-docs-samples/blob/master/…). But thanks for this explanation Josell! – Ultrasound