Firebase Device Test lab: Failed to create a test matrix

Question

I want tot try out firebase device testing - but i get a really generic error message doing so:

Failed to create a test matrix.

Anyone knows how to get around this? Quota should not be the problem - the button I press says "Start 2 tests" I already removed some devices in case it is quota. Unfortunately googling for the error does not really lead to a result..

Prunelle

Answer 1

Ahmed here from the Firebase Test Lab team.

The issue happens when you exceed the quota for the day. There have been two issues on the Test Lab side contributing to the confusion here:

1. The message shown is confusing and it's hard to understand what's going on
2. To enforce this daily quota on usage, we require to have the API enabled on your project. We needed to have the API enabled on all Firebase projects. When we launched, we had the following in place (i) All new projects have the API enabled by default and (ii) All existing Firebase Test Lab users had the API backfilled. The issue happens when you have created a Firebase project in the past (before October 1st 2016) and never used Test Lab then went back to use it afterwards.

Issue (1) is still ongoing and we're currently working on making our error messages more informative. Stay tuned!

Issue (2) is now resolved completely. We've backfilled all the Firebase projects that were ever created with the Test Lab APIs.

You should now expect seeing this issue only in cases where you exceed the daily quota till we provide a better error message.

Thanks so much and apologies for the confusion!

Porphyrin

thanks for the follow-up - moved the accepted answer to your's as it is correct for more users – Prunelle

Staying tuned here as we approach 2020 ;-) I am still getting this message and uninformative error. – Hiram

Its already 2023 and its still there. – Bra

Answer 2

Sorry for the confusion. We are improving this error message moving forward.

The problem is that you don't have the Google Cloud Testing API enabled for your project. Please can you do that?

Or just create a new project for which it will be automatically enabled.

Reasonless

I have the same error, where do I enable the Testing API? – Selfexamination

I have enabled the Google Cloud Testing API, still I get the same error. – Hirsute

Answer 3

Go here https://console.developers.google.com/apis/api/testing.googleapis.com/overview

select the project that you are using to run tests with Test Lab, and click "Enable".

Subadar

I am a regular user of the Firebase (first added Analytics, than learned about Test Lab free tests) so I think it will help in most of the cases. Go through that link and add your app (your app name will be available for selecting in dialog box) after you add and refresh the page, make sure that you're in ENABLED state (on top there will be a button with text "Disable', which means you're enabled now) – Kamerad

Answer 4

I had exactly the same problem and I check all the suggested solutions, like give admin rights, enable the testing API and it didn't work.

Lastly, I tried creating the tests from Android Studio. From Android Studio, go to Tools menu, then Firebase. I just followed the assistant and voilà, tests are running now!.

Emit

Answer 5

If I upload an AAB file I will get this message but it works when using an APK.

Westsouthwest

Same here, make sure you have V1 and V2 signatures. – Hiram

Answer 6

I went to https://console.firebase.google.com/iam-admin/iam/iam-zero Selected my project and edited myself to be admin of everything (app engine,big query, cloud debugger)

Xanthus

Answer 7

It depends of the payment plan that you have chosen. I have chosen the payment plan as Spark and I am able to test for 1 TEST at a time with the Test matrix getting generated successfully.

Demilune

Answer 8

I had the same problem and the reason for the error message was in my case that I took an apk from Firebase App Distribution.

When I created the apk in Intellij / Android Studio and uploaded this one it worked.

Strickland

Answer 9

In my case the problem was that I tried to upload obfuscated build

Elaboration