Testing as a guardrail

Most people see testing as a revenue-improving tool, but there’s another, less-discussed facet: testing as a protective strategy. It’s not just about forward momentum; sometimes, it’s about ensuring we don’t slide backward.

𝐓𝐞𝐬𝐭𝐢𝐧𝐠 𝐰𝐢𝐭𝐡 𝐍𝐨𝐰 𝐍𝐨𝐯𝐞𝐥:
At Now Novel, our testing and development resources are limited. As a data nerd I ensure that we have customer feedback via surveys and heatmaps, but our challenge was how to implement the insights. Ideally, iterative tests would have allowed us to gradually refine and shape our payment page and learn about customer behaviour and preferences. However, time is of the essence in a startup, and we have the issue of cost of iterative changes. Implementing a big bang change, but making sure it doesn’t break things was the answer.

𝐓𝐡𝐞 𝐁𝐚𝐥𝐚𝐧𝐜𝐢𝐧𝐠 𝐀𝐜𝐭:
Blindly integrating changes into our site was never an option. The objective wasn’t just to chase a win but to ensure our steps weren’t harmful. This validation method is one I’ve consistently used, especially when major shifts—like overhauling the checkout process or drastically altering navigation—are in question. While strategic initiatives are important, they shouldn’t jeopardize revenue streams. This also a factor in my consulting process. How do we check big ideas and inititiaves don’t have unexpected impacts and consequences?

𝐆𝐮𝐚𝐫𝐝𝐫𝐚𝐢𝐥 𝐌𝐞𝐭𝐫𝐢𝐜𝐬: 𝐀 𝐒𝐚𝐟𝐞𝐭𝐲 𝐍𝐞𝐭:
Guardrail metrics serve as the counterbalance. They’re a kind of key performance indicator (KPI) that keeps businesses from overemphasizing one metric. It’s the equilibrium between primary goals and secondary yet significant objectives.
Take, for instance, a company emphasizing viewing a product image. While this is their primary aim, conversion rate might be their guardrail metric. If forcing more views of the product images impacts conversion rate, it’s a clear signal: time to adjust and recalibrate.

In essence, as businesses, it’s essential to ensure our car doesn’t veer off course. Guardrail metrics help maintain that trajectory. To extend the metaphor, we’ve got to be the steady hand on the wheel (which is why I used the image with additional fingers, to ensure an extremely steady hand on the wheel 😀 🖐 🚗 )