Describe the bug
When we reset an experiment from dashboard, starting numbered versions of experiments in data, it makes confusion in User.max_experiments_reached? and considers the current experiment as already taken.
The symptom is that subsequent calls to ab_test will always return control, once the user starts to be excluded from alternative choice by max_experiments_reached.
To Reproduce
Steps to reproduce the behavior:
I wrote an example app that reproduces the error. It's in rails because it was easier to me.
here is the app.
- Go to home page. You'll see the alternatives returned by multiple calls to
ab_test. This time it works as usual;
- Go to split dashboard and click in
reset data, so to start a new version in the experiment
- Now refreshes the app until the first alternative is choosed as
other.
- In this case, the result you see is
oher, control, control showing that, after that first call subsequent ones will always return control
Additional context
I've already debugged it. The problem is in User#max_experiments_reached?. The count in L31 doesn't clean up experiment key, that now has :<version> at the end.
I can submit the PR if you like so.
Describe the bug
When we reset an experiment from dashboard, starting numbered versions of experiments in data, it makes confusion in
User.max_experiments_reached?and considers the current experiment as already taken.The symptom is that subsequent calls to
ab_testwill always returncontrol, once the user starts to be excluded from alternative choice bymax_experiments_reached.To Reproduce
Steps to reproduce the behavior:
I wrote an example app that reproduces the error. It's in rails because it was easier to me.
here is the app.
ab_test. This time it works as usual;reset data, so to start a new version in the experimentother.oher, control, controlshowing that, after that first call subsequent ones will always returncontrolAdditional context
I've already debugged it. The problem is in
User#max_experiments_reached?. The count in L31 doesn't clean up experiment key, that now has:<version>at the end.I can submit the PR if you like so.