Proving words matter with
A/B testing.
What we did: Working with our Web Experimentation Sr. Manager, I submitted a proposal for a hero section, H1 headline-only copy test on one of our highest-traffic pages, Wellhub for Employees.
Why we did it: Our revenue marketing team revealed that using cost comparisons in their email campaigns were seeing the highest open rates and CTR. I wanted to validate if our cost value proposition would similarly drive conversion on our web page.
The challenge
My initial challenge was to get buy-in from our web manager to prioritize this test so we could launch quickly. I followed up on my test after submitting to reiterate the timeliness of the message (connecting the hypothesis to our revenue marketing campaign results), and that since this test required no design or dev changes, this was very low effort launch, but had high potential for a big win. This proved to be a convincing approach – we were able to launch within a few days of my test request submission.
The process
To simplify and speed project delivery, I proposed a UX Writing style guide (built in a google document with searchablt tabs) to address content design specifically for the web and to also heavily focus on brand voice and tone. More on process: By benchmarking style guides from companies like Slack and Mailchimp, I was able to effectively visualize and assess the pros and cons of the guide structure I had in mind.
Key contributions
-
Create initial request proposal (hypothesis & outcome) for a copy-only test
-
Write new hero H1 headline copy
-
Assess test results at stat sig with our web experimentation manager
-
Share results and recommendations to our entire marketing org channel
The result
Spoiler alert: this was an absolute homerun. Our team had anticipated this proposal as a low-effort test that would probably see positive results, but this tiny shift in messaging netted huge results. After reaching statistical significance, our new H1 variant showed a massive spike of 91% more signups post-click through of our primary CTA. Interestingly, our CTA click-through was only a small bump of 7.7% more clicks, which tells us that this new messaging also drove higher-quality leads that were more likely to convert.
The impact
Following this big win, I shared results with our full marketing team not only to inspire more test ideas, but also to reiterate that words truly do matter and that even small changes can make a big impact. Following this test, our team also launched a series of similiarly low-effort terminology tests which also helped us gain critical insights to improve our website's performance.