16
How To Run AB Tests In React
I was recently faced with a situation where I needed to run an AB test from within my React app. The platform that we use to run AB tests is Google Optimize, a free Google product that helps you run experiments on your website by varying parts of it and analysing the resulting metrics you care about.
Up until this point, the experiments we ran involved changes that were so wide-ranging and numerous that it made sense to just set up an entirely new route and split our visitors on the page prior to entering the React app. i.e. 50% sent to mywebsite.com/enteringreactnow/v1 and 50% sent to mywebsite.com/enteringreactnow/v2.
Setting up entirely new routes is super easy to set up, both on React and on Optimize, but isn't an efficient option if you only need to, say, test one component within your app. So what to do?
This option is more of a hack. I came across a very specific situation where the experiment was to either show / hide a particular screen route. I had set up my app so that the sequence of screens a user goes through is pre-set. I then use a custom hook to determine any conditional flows for navigation (useNavigator) e.g. whether to show or hide a particular screen as the user is clicking through this linear sequence of screens.
The essence of what I was trying to do was to show / hide a string (the path name) in an array of strings (the end to end flow), rather than showing or hiding a component. There didn't seem to be an obvious or easy way of doing this with Optimize. I therefore decided to implement the function of breaking users into random groups myself, then writing the logic that sends them down one experiment path vs. the other.
To create random groups,
const ALL_GROUPS = ['A', 'B']
type Group = typeof ALL_GROUPS[number]
const randomGroup = (): Group => ALL_GROUPS[Math.floor(Math.random() * ALL_GROUPS.length)]
The group allocation of each visitor is stored in the Redux state and persists to local storage. This ensures that visitors that revisit the app retain their original group allocation and thus have the same app experience. This is similar to how Google Optimize does this, where they use cookies instead.
From here, it's easy to write the logic of whether to show or hide a particular screen depending on the group someone belongs to. The harder thing is to have to monitor and calculate the experiment results yourself. You'd need to use something like Google Tag Manager + Google Analytics to set up event tracking, then be comfortable with calculating how statistically significant the variation in results are. In all likelihood, if you're already using Google Optimize, you'd likely also have set up Google Analytics, so it would just be the actual analysis of results you'd have to deal with.
Pros
Cons
In essence, what you need to do is:
<head>
tags in your index.html
file. This script will include your Google Analytics ID and Optimize ID.Run Diagnostics
to check that the script has been installed properly and is running on your app.- Ensure you've changed the activation event to "custom" and make a note of the event name. The default is
optimize.activate
but feel free to rename this. - Make a note of the experiment ID.
react-optimize
. Check its GitHub repo for more info.<Experiment id="<experiment-id>">
and <Variant id="0">
. Looking under the hood, it's pretty lightweight and does all the management of loading Optimize at the right time, for you. You will however, need to stick to the default custom activation event name of optimize.activate
for it to work.react-ab-test
package from Marvel App.For my experiments, I ended up going with Option 1 and Option 3. They both sufficed for what I needed, without needing too much set up time. Not knowing whether it was something I'd be wanting to do continuously, over the long term, also meant I needed something quick, and easy to understand and review (and potentially kill if not needed).
Any comments? Talk to me on Instagram @bionicjulia!
16