Description: The A/B Testing test evaluates a candidate’s ability to design and analyze A/B tests in order to identify customer preferences and increase customer engagement and conversions.
It’s an ideal test for pre-employment screening of candidates applying for a variety of job roles, including digital marketers, marketing analysts, and user researchers. A good candidate for an A/B testing job will be competent in both test design and results analysis, and will understand how statistical significance impacts confidence levels.
This test requires candidates to demonstrate the ability to identify appropriate metrics for A/B testing, set up rigorous tests, and produce actionable insights from analysis of test data.
A/B testing is an essential user experience research methodology that compares two or more versions of a variable (e.g. a web page) to identify which is most successful with customers.
The ability to determine necessary sample sizes and an understanding of how sample sizes influence test duration are essential knowledges for anyone designing A/B tests or analyzing testing data.
Public questions (free account) are common interview questions. They are great for practicing, or if you want to filter candidates using the classic problems.
This is a new addition to our question library.
You have a new client, who operates an eCommerce business receiving traffic of 500 visitors per week. The client insists on presenting you with a list of ideas for improving their website and expects you to approve or reject each idea after careful consideration.
In the list below, enter “Y” if you would approve the idea and “N” if you would reject the idea.
_____ 1. Testing small changes to the landing page to increase conversions by 2%.
_____ 2. Testing radically different redesigns of the landing page to increase conversions by 50%.
_____ 3. Test variations of the product page to increase Add to Cart clicks by 20%.
_____ 4. Test variations of the checkout process to increase completed purchases by 1%.
_____ 5. Use multivariate testing on your least-popular product pages to increase purchases.
_____ 6. Lower your statistical significance threshold to 75% to see the results of your tests more quickly.