How to hear what your users are trying to tell you

Are you making the biggest mistake in interpreting customer feedback?

The two big mistakes you can make in design are:

  1. not listening to your users.

  2. listening to your users.

I don’t need to persuade you that not listening to users is a mistake. But perhaps it’s not as clear why listening to users is also a mistake.

It’s all down to how you listen. How you interpret what your users tell you. And how you interpret what they’re not telling you.

When I started out, I got this stuff wrong a lot. From asking bad questions to taking what people told me at face value, I made ‘em all. What’s more, even when I learned that I shouldn’t make these mistakes, it still wasn’t clear what to do instead.

The surprisingly simple secret to resolving the paradox came from someone who writes fantasy novels. I’ll share that later.

But first, why can’t we take what people say at face value? Why is research so tricky?

I don’t think I can put it more precisely than David Ogilvy:

"The problem with market research is that people don't think what they feel, don't say what they think, and don't do what they say."

We humans don't have conscious access to our decision-making brain-bits. We make 95% of our decisions completely unconsciously – even though it feels like we’re totally in control. It’s a trick: our conscious brain watches what we just did and then it makes up a plausible explanation for itself. It’s a master of post-hoc rationalisation, fabricating reasonable stories about what we just experienced, decided, or felt.

So when people tell you things in sessions …

  • “I like this”

  • “Hmm – it's a bit ... long”

  • “Needs more colour / animation / graphics (more "pop")”

  • “Yeah I totally understand this”

  • “I’d call support now”

… they're telling you how their conscious brain has made sense of the signals from their unconscious brain. This information is unlikely to reflect their actual thinking process. But it does give us useful clues.

The words they speak are rationalisations of what they’re feeling.

Weirdly, people are often mixed up about what their feelings actually mean. The fancy name for this is “misattribution of arousal” and it’s why — famously — scary films can make for good first dates because feelings of fear triggered by the film are easily misinterpreted as feelings of attraction for your viewing partner.

Our subconscious is always trying to guide our behaviour towards achieving its goals, and it does it in the only way it can: by making us feel things. Positive feelings are our subconscious telling us to move towards something it believes will help us achieve one of our goals. Negative feelings are when it’s trying to tell us the opposite: get away from that!

So while our emotional palette is nuanced and complicated, we can also usefully simplify emotions into intensity and valence. The intensity is the strength of a feeling. The valence describes the move-towards-ness or move-away-ness of the feeling.

For instance, fascination and desire are high intensity signals to move towards something. Revulsion and terror are high intensity signals to get away.

We can draw these on a line:

The challenge in user research is that you’re mostly dealing with valence shifts at low intensity: kind-of-towards-ish vs kind-of-away-ish. They can be hard to detect, so your job is to tease out these shifts and figure out what's causing them.

Let’s translate our examples of things people say:

  • “I like this” --> I feel slightly pulled towards this

  • “Hmm – it's a bit ... long” --> I feel bored by this

  • “Needs more colour / animation / graphics (more "pop")” --> I feel bored by this

  • “Yeah I totally understand this” --> my unconscious doesn’t feel it needs to expend more energy figuring this out. (Now look for whether the remaining feeling is "... and I'm interested to keep going" or "... and I feel no drive to keep going.")

  • “I’d call support now” --> I feel slightly frustrated by not understanding this, and slightly bored. There's not enough signal in the information I've seen to keep trying to figure it out.

The feeling of boredom is one of the most misinterpreted in all user testing, because people rarely tell you they're feeling bored. I don’t know why. They probably misattribute the feeling, but maybe they’re just being polite. What they do instead is come up with a proposed solution to their boredom: something they think will make them less bored. This usually comes down to one of two things:

  1. add entertainment or distraction (“needs more colour/pictures”)

  2. reduce their exposure to the boring thing (“make it shorter”)

These are lovely suggestions, but the real reason they're bored is because the thing doesn't connect with any goal their subconsious feels a positive valence towards. We all devote attention to activities that hook our brains – whether that’s reading books, programming, tweaking icons, or bingeing Netflix. We’re bored by everything else. And it varies from person to person. I'm sure you can think of activities you'll comfortably spend hours on that would bore other people to tears.

Now, to come back to that writer of fantasy novels: it was Neil Gaiman.

And all the fussy psychology stuff above is neatly wrapped up in his quote:

"Remember: when people tell you something’s wrong or doesn’t work for them, they are almost always right. When they tell you exactly what they think is wrong and how to fix it, they are almost always wrong."

- Tom x