The perils of design testing and trusting the instincts of experts

Where possible, managers should base decisions on reputable data. When it comes to investing in new product development, market research offers forecasts, scores and percentage points. But in his fascinating new book Blink: The Power of Thinking Without Thinking ‘author, journalist, cultural commentator and intellectual adventurer’, Malcolm Gladwell argues that it is, in fact, the first two seconds in many encounters that count far more than you would ever imagine. For creatives who live and die by their instincts, Blink casts valuable light on the pros and cons of trusting your gut. Gladwell makes clear that instincts can be both on the money or wildly off the mark. For example, relationship psychologists can review a 15-minute video of a married couple talking, and predict, with 90 per cent accuracy, whether they will still be married in 15 years’ time.

On the flip side, he recounts the tale of how Warren Harding, America’s worst-ever president, ascended the political ladder purely by making good first impressions. Like in his previous book The Tipping Point, Blink is based on well-researched and well-told stories from wildly eclectic fields: from speed dating to war games, and jam tasting to racial prejudice. Unlike The Tipping Point, this book moves beyond the merely descriptive to some genuine insights. The passage on the two-way street between facial expressions and emotions is enthralling. Although he sometimes lazily falls into computer analogies, the book paints vivid sketches of the inner workings of our minds. In most situations we apparently work best when we don’t process much information, but focus on ‘thin slices’ of vital information to spot familiar patterns.

Blink investigates the influence of ‘gut instincts’ and snap judgements in all walks of life. Here I focus on the lessons Gladwell draws from the stories of New Coke and the ‘Aeron’ office chair.

The New Coke disaster

In the early eighties market research indicated that Pepsi was gaining fast of the once dominant Coke, in terms of the number on consumers who said that they exclusively drank each brand – even though Coke had far better distribution and was spending $100m more on advertising. Pepsi were running their ‘Pepsi challenge’ campaign, in which 57% of people preferred their product in ‘blind sip’ tests.

‘The story of New Coke… is a really good illustration of how complicated it is to find out what people really think’.

In the early eighties market research indicated that Pepsi was gaining fast of the once dominant Coke, in terms of the number on consumers who said that they exclusively drank each brand – even though Coke had far better distribution and was spending $100m more on advertising. Pepsi were running their ‘Pepsi challenge’ campaign, in which 57% of people preferred their product in ‘blind sip’ tests.

Coke decided that they had a problem with taste and tasked the men in white coats with tinkering with the fabled secret Coke formula. They made it lighter and sweeter – more like Pepsi – until it tested better than Pepsi in blind tests. In a massive round of consumer testing in 1984 New Coke beat Pepsi in sip tests by 6-8%. At the press conference announcing its launch, the then CEO Roberto C. Goizueta called the new product ‘the surest move that company’s ever made’. New Coke was such a disaster, it provoked protests and forced Coke to bring back the original formula as Coke Classic – at which point New Coke sales evaporated. The biggest surprise was yet to come… Pepsi’s predicted catch-up never happened. Coke remained the world’s favourite cola.

Coke’s error was to believe the sip test. More realistic studies, which involved taking a case of cans home and drinking them over a number of weeks, showed very different results – Coke clearly came out on top. Gladwell argues that there are sensory and psychological explanations. Pepsi is sweeter and tastes good to many after a sip, but less appealing after drinking a whole can. Also, consuming the product at home is a more realistic context to reach a reflective decision. Finally, there is the more subtle influence of ‘sensation transference’, the concept that people convey impressions of the brand, including packaging, onto the product. For a brand like Coke, people make no distinction between the product and the packaging – ‘the product is the package and product combined’.

The reason the sip test results didn’t translate into real world sales, is that ‘in the real world, no one ever drinks Coca-Cola blind. We transfer to our sensation of the Coca-Cola taste all of the unconscious associations we have of the brand, the image, the can, and even the unmistakable red of the logo’. The lesson here is the develop an understanding of an issue’s context, before deciding on what the key parameters are that can be quantified through market research.

Aeron against the odds

The story of the ‘Aeron’ office chair illustrates a second, deeper problem with trying to measure people’s reactions: it is hard for us to explain our feelings about unfamiliar things. In the early nineties the renowned industrial designer Bill Stumpf, started work on his third task chair for Herman Miller. His previous ‘Ergon’ and ‘Equa’ had been successful, but Stumpf wanted to design the ultimate in ergonomic comfort. The result ‘looked like the exoskeleton of a giant prehistoric insect’. In 1992 Herman Miller tested them with local companies and asked each person to use them for half a day and then rate it out of 10 on comfort and appearance. The early prototypes scored badly. After some prototype development, comfort scores reach eight, but those for aesthetics never reached six. This was unusual, as experience has shown that consumers strongly relate comfort and looks. Generally, if people like the chair in the first ‘blink’, they will perceive it as comfortable – and vice versa.

In 1993, Herman Miller ran some pre-launch expert tests. Architects and designers, who might specify the chair, generally liked it – they understood why the chair was radical. However, facilities managers and ergonomic experts, the final purchasing decision makers, hated it – for mainly aesthetic reasons. Herman Miller trusted its instincts and launched the product. Before long it won design awards and got adopted by cutting edge creative and technology companies – ‘it matched the stripped-down aesthetic of the new economy’. By the end of the nineties, it became the best selling chair in Herman Miller’s history and scored eight for aesthetics – ugly had become beautiful!

Testing consumer reactions to the Aeron, failed for a different reason than those to New Coke. This was less a misunderstanding of context, than consumers misinterpreting their feelings. While they said they hated it, what they actually meant was that they didn’t understand it – they thought it was weird. The lesson here is that testing innovative products has to be handled very carefully and reactions need interpretation. If the Aeron has been a minor variation of what had gone before, consumer testing would have been far more straightforward.

The wisdom of experts

Deconstructing what happens in the first two seconds of ‘rapid cognition’ is near impossible, as it is a product of our unconscious – the part of our thinking that is outside our awareness. Gladwell advises against trying to explain snap judgements by asking consumers to remember, deconstruct and describe why they reached a decision. While Blink is long on problems and short on solutions, Gladwell does make a positive suggestion here; by drawing a distinction between consumer and expert instinct. He argues that instincts without experience in the subject matter should not be trusted. Whereas, expert instincts are fined tuned to get at the ‘gestalt’ or the essence of a subject: ‘with experience we become expert at using our behaviour and our training to interpret – and decode – what lies behind our snap judgements and first impressions.’ By definition experts spend a lot of time studying, reflecting and developing a precise vocabulary on their subject; and as a result they build a rich subconscious database’. In comparison consumer reactions tend to be shallow, as they are not grounded in real understanding.

Managers should still look to base investment decisions on data, but should ensure it is reputable. To do this they must understand the context, before working out what can be realistically measured; and include expert opinion into their assessment, especially when the product in question is innovative.