On being profitably wrong
I loved this quote from Joseph Tussman:
“What the pupil must learn, if he learns anything at all,
is that the world will do most of the work for you,
provided you cooperate with it by identifying
how it really works and aligning with those realities.
If we do not let the world teach us, it teaches us a lesson.”
The final line is a bit clumsy, but I think it means he’d agree with me on this:
Every business and every design choice is part of an experiment — whether or not we want it to be.
What I mean is that if we don’t define our experiment variations and outcomes, the world will define some variations and outcomes for us — in the form of competition or changes in the marketplace.
We don’t get to choose whether or not what we put out there is part of an experiment. What we do get to choose is this:
A) we can choose to stick to doing only that thing that we believe should work. Lots of businesses do just this and, you never know, we may be one of the lucky handful that thrive for a while on that approach. (More likely, we’ll find ourselves outpaced by competitors, or pulling down the shutters after losing customers and revenue.)
Or:
B) we can invite the world to teach us what really works and align ourselves with it. We can define our own variations and measure their outcomes. We can challenge our cherished assumptions.
Now, the hard part about inviting the world to teach us is that we have to accept that our ideas about how it works might be — gasp! — wrong. This is against most of our training in the modern world: schools and corporations train us to have “the right answer” and our confirmation biases wire us up to look first for evidence that we’re right.
Until we truly get used to the fact that we’re all wrong about a lot of things, it’s painful finding out that we’re wrong as plainly as we do when we A/B test or conduct user research.
Let me give you an example
Some time ago, when I was younger and even more foolish, one of my clients decided to change a headline.
It was the headline on a key sales page.
I wanted to A/B test the change, even if only to measure how much better it was. But in the end, we didn’t. This was for two reasons:
it was technically tricky and so time consuming to test on this page and they didn’t want to waste any time because…
it was an idea they’d lifted from other websites in the industry. It was obviously better.
The product owner was adamant: “we should save our A/B testing for things where we aren’t sure. This new headline is a guaranteed licence to print money.”
And so we threw our A/B testing efforts into a part of the sales sequence where it was easier to test. (It was also a lot less effective, but that’s a story for another day.)