Website & CRO
Is A/B testing worth it for a small online store?
The short answer
Not until you have serious traffic. If only a thousand or so people see a page each month, a split test can't tell you anything you can trust. You'll get better results from applying proven conversion principles, writing great copy and improving one page at a time.
Why doesn’t split testing work at small scale?
A split test needs enough visitors on each version for the difference to be real rather than luck. With a thousand people a month landing on a page, you’d be waiting months for an answer, and by then your season, your ads and your offer have all changed. Most of the “winners” small stores declare are noise.
Strict split testing starts to pay off somewhere north of $200k a month in revenue, and it’s great when you’re doing a million. Below that, it mostly burns time.
What should you do instead?
Use the 80/20. Most of the gains on a small store come from things we already know work:
- A product page that asks for one click. Make the add-to-cart button contrast with your brand colours, make the price easy to read, and stop reviews tabs, chat widgets and points banners from competing with it.
- Copy that answers the questions people actually have before they buy.
- Offers and bundles that make the bigger purchase the obvious choice.
Then improve one key page at a time. You don’t need to change one thing and wait. Make a batch of sensible improvements, live with them for a few weeks, and judge the page on revenue per session. If you’ve raised your prices and conversion has held or gone up, you’re heading the right way. It isn’t as tidy as a lab experiment, but it moves much faster, and you’re changing things that matter, not button colours.
One of the biggest growth stories I’ve been part of didn’t involve a single split test. Just good conversion principles, great copy, and a run of sensible changes to one page.
When should you start testing?
When your traffic can give you an answer in a couple of weeks, and when someone owns the testing programme: picking tests, reading results and retiring losers every two to three weeks. If nobody is doing that, delete the tests you have running. A test nobody looks at is just clutter.