Consider adding a failsafe for publishing/unpublishing a test in AQA mode for lower-level User Assess types
planned
B
Byron Antonas
Good day Product team,
I experienced a scenario recently, where I gave my QA tester "Creator" level access, ignorant of the nuance of how the play/pause restrictions were setup on your platform. I noticed that the QA user turned on the IP restriction in my test's QA module - and then the QA user enabled the QA assistant for desktop, which essentially published the test, as part of its native system function.
However, when the user was done QAing, they ignorantly only turned off the IP restriction, not realizing the test should have been paused first. Which, because they could not un-publish the test, opened the experiment up to actual user traffic. I caught this relatively quickly and paused the test, but when I investigated what happened with the QA user. I obviously discovered she was not able to pause the test, given her user role (which I corrected). But again for her - the button, simply wasn't interactable. But given the fact that she is still able to publish the test by engaging with the QA assistant, there needs to be something that also un-publishes the test whe nthey are done.
I wonder if maybe, if you add an alert or something that detects that a user is trying to disable the IP restriction, while the test is still live, so it warns or even automatically pauses the test as a failsafe, to ensure that no external traffic begins accessing the test.
I would appreciate this feature. Thank you.
Léo Wiel
planned
Hello,
Thank you for your feedback.
We are currently designing a major rework of QA Assistant and of the overall QA process and campaign lifecycle. The issue you are talking about is, indeed, a natural flaw of how it has been originally designed. We want to better encapsulate the QA status of a campaign to make sure it is safely deployed to the website only for internal users and that it doesn't go public without a real launch of the campaign.
I am adding your case to the list of other feedbacks about QA Assistant! We will be working on this rework in 2024.
Léo