Firebase A/B testing is still running after rollout the leader 100%

Question

I run a Firebase A/B testing in my recent app version, It's an experiment with one remote config parameter.

After a few days, there is a leader generated, so I rollout the leader and applied changes. As I expected, the experiment should become a completed experiment now because I already got the leader and rollout it, but the real is the experiment is still in running experiments list, not in completed experiments list.

I'm confused. Do anyone encountered the same problem and know how to fix the problem?

Is it a bug of firebase console or something is wrong with my actions?

Homophone

I am also confused like you, so we need to stop the experiment after rolling out? – Cote

Answer 1

Firebaser here!

There is nothing wrong with your actions. Rolling out a Remote Config experiment changes the default value of the parameter being experimented on in your Remote Config. It does not stop your experiment.

To stop your experiment, you can click on the overflow menu and hit "Stop experiment".

Alight

I'm still confused, After I rollout the leader, the experiment is still running, and the user number in the loser group is still increasing. Why not Firebase stop the experiment automatically and let all of my users receive the values of the win group? – Homophone

@robert, I believe that as Sinan explained, you need to hit "Stop experiment". When you declared a winner, Firebase set the default value in the "Remote config" tab to your winner. But the experiment is still overriding this value, so you need to stop the experiment yourself. – Zone

The option is mislabelled. It shouldn't read 'Rollout variant/leader' but rather 'Change remote config default while continuing experiment' – Tratner

Please add this note to the Firebase A/B Test documentation! I was confused for a while about why I was still seeing inconsistent values after rolling out the successful value of an experiment. – Irresoluble