1 visitor - 1 test : make the QA easier
complete
Léo Wiel
Merged in a post:
Override "1 user, 1 test" with QA Assistant
J
Jacqui Dent
I am part of a UX team running multiple tests every fortnight. As our capacity to launch tests increases, so too does the risk of tests overlapping. We were excited to discover the "1 user, 1 test" feature which would mitigate the risk of overlapping tests contaminating data. However, we've discovered this feature makes it difficult for us to QA our tests.
As we are on the website all the time QA'ing and checking in on tests, of course myself and my colleagues are all bucketed into past or current tests, which means we are automatically rejected from test targeting for any test we are trying to QA with this feature turned on.
As a workaround, we can QA our tests and then afterwards turn on the "1 user, 1 test" feature. However, that prevents us from QAing and launching any future tests while the test requiring this feature is switched on. As we would only want to use "1 user, 1 test" when we are running many tests, this seems to make the feature a bit redundant. We can do manual QA but that is cumbersome.
It would be great if the QA assistant could override "1 user, 1 test" settings so that we can have this feature running, have interacted with other current or recent tests, and still be able to use the QA assistant for new tests.
Léo Wiel
complete
You can now force the display of a campaign, even when 1v/1t is active.
(soon to be released)
Léo Wiel
Merged in a post:
Improve 1 Visitor 1 test & QA rejection
N
Nick Stratton
Hello,
There is no good way currently to QA when you have 1 Visitor 1 Test enabled.
Problem:
I'm running into situations where clients use 1 visitor 1 test and express frustrations with not being able to QA. I'm trying to find a path forward to help these clients have a better QA experience while having this feature enabled.
As you know, QA with 1V1T enabled can be a pain.
- If you're already accepted to a test and you're trying to QA a new test with the same targeting, you'll be rejected with a status of 1 test 1 visitor
- Even if you fully flush the ABTasty cookie and try to QA that new test directly, you'll still get the same 1 test 1 visitor rejection as the test acceptance is conditioned based on numerical order of the test IDs
Clients get frustrated at this and it often ends up with them launching the test without QA. This is dangerous and can lead to them missing certain behaviors or errors in the test setup, leading to more frustrations.
Léo Wiel
Merged in a post:
Exclude 1 visitor 1 test from QA
Pierre Pellegrin
Hi team,
Showroomprivé uses 1 visitor 1 test option.
When they want to QA a test they have to remove cookies or use a private session in order to be targeted by a new test.
They find very painful.
They would like this option not to be activated when in QA.
Thanks :)
N
Nick Stratton
Hello, Five Below ($200k ARR) also report this pain
Léo Wiel
Hello Jacqui Dent,
Thank you for your feedback!
We are currently designing a major update of our QA Assistant and the behavior you are describing is on our top priority list of things we want to improve.
We are aware that it can be a pain to have to enter a test to be able to QA it. It is not only 1 visitor / 1 test, but also the whole targeting that needs to be validated. Our plan is to give our users an option to force the display of a campaign, even if the conditions are not met at all, whatever they are.
I can't give a proper ETA for this project, but we will be designing the solution probably before the end of 2024.
J
Jacqui Dent
Léo Wiel Wonderful, looking forward to it!
Blandine Paulet
under review
l
laurent.babicz@showroomprive.net
It will be very useful to continue to launch several tests at the same time
Arthur Charbit
laurent.babicz@showroomprive.net: Hi Laurent, thanks for your comment!
By this you mean launching several test and making them mutually exclusive?
l
laurent.babicz@showroomprive.net
Arthur Charbit We run several test in prod on same page with the 1 visitor = 1 test rule. But when we want to QA new ones we are assigned to existing test and sometimes in private mode too
I would like under QA to be able to switch from tests such as variations