Script:

Farrell: This is A Penny for Your Thoughts, I’m Farrell Styers. A Penny for Your Thoughts is a show about the business of figuring out what people want and why. Because there is in fact a massive industry built around trying to understand human motivations.

[placeholder for “ever wondered why” questions]

Have you ever wondered why high-end retailers always play soothing music? I’ve wondered those things, too. And this show is for people curious about exactly this sort of stuff.

I don’t come to this quest empty-handed. I’ve spent the last decade helping all sorts of organizations understand what people think. I worked in the US embassy in Baghdad at the height of the Iraq war, I ran a social research company in Kyrgyzstan that looked at everything from perceptions of democracy to the impact of industry on a rural villages. I even ran a one-man user research operation for a major tech manufacturer. And now I live in the beautiful city of Gent, Belgium where I work for one of the most innovative market research companies in the world, InSites Consulting.

And this is all to say, I’m fascinated with this research. I’m fascinated by how academic theories in social science created a multi-billion dollar industry. I love seeing how changes in culture, in technology, even in the way we do business is shaping how we behave and how we understand ourselves. So I’m setting out on a mission to share these stories.And my first story starts a few months ago. I hit a point where I was disillusioned. I still loved research but I felt like the industry was broken in some pretty fundamental ways. So for this first episode, I want to look at the state of the industry and how we got here. We’ll hear a bleak assessment of how market research is like a dying horse.

[Hakim: dead horse]

And hear from one of the giants of the past

[Dichter: we don’t go out and ask directly]

But before we get to that, let’s look at where it all started.

You can find examples of market research going back hundreds of years depending on how loosely you define it, but where it really picks up is in the early 1900s in the United States. And initially it started from ad agencies who were looking for a way to measure how effective their advertising was. They would send some guys around town and stop people on the sidewalk. And they’d ask them which magazines they read and then which ads they remembered from those magazines. Then they would take all this information back to their office, tally up the results, compare it to the number of magazines sold and voila! the first advertising metrics were born. And while this research clearly had its limits, it laid the groundwork for how those measurements are made – even today.
Eventually an American psychologist by the name of Daniel Starch saw that he could add more scientific rigor to the way this research was conducted, and in 1923 he started among the first successful market research firms, Daniel Starch and Staff. They sold research as a service. And as businesses started seeing the advantages of market research, more of these research agencies began to pop up.

And for the next decade and a half this is how market research was done. Face-to-face quantitative surveys, mostly centered on advertising. That’s not to say there wasn’t progress. During this time George Gallup created the company that became synonymous with opinion polling, making the first scientific election predictions; Rensis Likert invented the Likert Scale, a tool for measuring attitudes which is still used by researchers today. But it was a very objective, dry, fact-based enterprise. In old news reels exploring this burgeoning new science, you get a feel for just how dry it was.

[Coffee and the American woman]

But then came a clever Viennese psychologist who flipped all of this on its head. His name was Dr. Ernest [Dick-ter]. Dichter was a young, Freudian-trained psychologist in Vienna who, like so many other Jewish intellectuals in the 1930’s, fled from the Nazis and eventually landed in New York. And that is where he started proposing a new way of understanding consumers, employing a concept called “motivational research” – a name which he also used for his research agency.

[Motivational research clip]

At the heart of his philosophy was a claim that people don’t know themselves particularly well (certainly not their motivations), so asking them directly what they thought just made no sense. Dichter thought that asking someone why they bought a given product was akin to your doctor asking you why your stomach hurt. That’s the doctor’s job to figure out, he said, not the patient. In market research, it was the researcher’s job to use the tools of psychology to understand people’s motivations, not the consumer’s job to tell them.
So Dichter would conduct long qualitative interviews or even just observe people, the way an anthropologist would. And like any good Freudian, he looked for the latent desires and fears that he thought drove most people’s thinking. Here’s an old clip of Dichter describing his work.

[Total personality clip]

So instead of the direct questions “do you like this one, or that one,” they were trying to look at deeper, more subjective drivers. This approach was fresh and his directness with clients was interesting, and soon he was the darling of market research. He’s even credited with coining the term “focus group”.
One of his first major successes was for Ivory Soap. Using his new techniques, Dichter came to the realization that bathing was a ritual and even more, it had underlying sexual implications. During their interviews women implied, according to Dichter, that their weekend bath was part of their preparation for a potential romantic encounter. In puritanical post-war America, such claims, especially in austere corporate environments, were revolutionary. Dichter even took it further by claiming that people unconsciously chose their soap based on the personality it exhibited and the values it stood for. Here’s this psychoanalyst with a thick German accent telling a group of executives that their soap needs a personality and that they need to focus on the secret sexual implications of the product. It doesn’t sound like a recipe for success, but…

The soap campaign built on his research was a success and Dichter’s new techniques were embraced by clients and researchers alike. It wasn’t just the techniques that made Dichter’s work revolutionary either. While not alone in this, he started going beyond advertising research and helping companies create new products based on consumer insights. For example, he recognized that by asking housewives to mix in an egg to Betty Crocker’s instant cake mix, women felt like they were preparing something homemade for their families. They were less embarrassed to buy it. And as a result of the new cake mix, sales improved.
But some of the boundaries Dichter and his adherents pushed went too far for many. Some of the more theatrical claims of deep-seeded drivers for power, inclusion, sex etc. motivating people’s purchases of everyday items were seen as suspect. He was also attacked by some who saw this work as driving needless consumerism. By the 1970s motivational research fell out of favor in boardrooms.

Regardless, the industry was forever changed. And while this seismic shift in qualitative research was happening, there was another change happening on the quantitative side too. Surveyors learned how to draw representative samples, allowing them to accurately measure huge populations. They were no longer just grabbing people from the sidewalk – they understood concepts like probability and mathematical randomization and applying this to their work. And they had also learned how to take advantage of a technology now found in every household – the telephone.

For the next 30 years, that’s how research was done: in-person, qualitative interviews and focus groups, and telephone-based surveys. What did change was the scale – more and more companies, governments, and organizations of all shapes and sizes realized that understanding people was key to improving their offers. So they poured more resources and more attention into it, creating a multi-billion dollar industry around trying to understand what makes people tick.
But remember we started this show with a dire description of the future of market research.

[Hakim: dead horse]

So who is this guy who thinks the research industry is a dying horse?

[Hakim: introduction]

I wanted to talk to Hakim because he witnessed the next major transition in research. As a general manager at an international research firm, he has his fingers in almost every aspect of the business. And he has been at it for a long time. Hakim started in market research as a student in the late 90s, working part time in a research agency call center. After a short time of climbing the ranks he eventually ran the entire call center.

[Hakim: randomized fixed lines]

This, as we already heard, was how research had been done for years. But market research was hit with the same technological shift that the rest of the world was in the late 90s.

[Hakim: internet]

Hakim’s experience at Market Response was similar to that of research agencies the world over. They realized the internet was going to change how research was done.

[Hakim: panels]

This is a really important point. As research migrated more and more to the internet, companies started building these research panels. The panels were really just databases with emails and demographic data on potential respondents.
Initially the internet was used only for recruiting, while the actual survey or focus group would be done offline – that is, in person or over the telephone. But it became clear even this wasn’t going to last. Phones died out and the research itself moved to the internet.

[Hakim: fixed lines]

And it makes sense then, that research would move the way so much activity would move – online. So why, then, the dying horse metaphor?

[Hakim: steep decline]
[Hakim: dying horse carrying many loads]

This, I think is worth digging into a bit. With the growth of internet research, a handful of massive panel brokers emerged. These were companies who had millions of email addresses and associated profiles for people. And remember, this was before anything like social listening, or public profiles with all of your personal data. So this was a goldmine.

So let’s say you have a new product and you want to research how to make your new product appeal to young, trendy city dwellers in the costal US. You go to a research agency and commission a piece of research looking at this specific group of people. That agency goes to one of these panel brokers who digs into their giant database to find people who fit that profile. And that broker connects the participants with the agency. The agency pays the broker for using their panel and the client pays the agency for the full research package. But most companies who are paying for the research don’t realize that essentially all the research agencies are using the same small number of panel providers. What that means is that whether your research is done by agency A or agency B, literally the same participants will likely be a part of the research. And more and more, those participants aren’t showing up.

[Hakim: sooner or later it’s going to be finished]

And that brings us to today. Market research has come a long way from its humble origins of pestering people on the sidewalk about magazine ads. We’re still pushing ahead, learning new things every day from people all over the world, but the current routine is unlikely to last. And that may seem scary, but what it really means is there’s just a chance for something new. For something innovative. That’s something we’ll explore in future episodes.
Here at InSites, we’re working on one approach to address this issue. It’s known internally as “the Square”. Here is a sneak peek

[Annelies: relevant]

Intriguing, right? We’ll hear more about that and other endeavors shaping market research in upcoming episodes, so check back in. We have a lot planned and a lot already recorded. We’ll talk to people here at InSites but also to clients, to partners, to academics, actually anyone with an interesting story to tell about market research.

This show was produced and mixed by me, Farrell Styers. It was edited by Katia Pallini. Production help came from Anke Moerdyck. Special thanks to Kaat Van Damme, Felix Rumpf and Hannes Willaert for help launching the show. For more on Ernest Dichter or other topics in this show check out our website at insites-consulting.com. That’s I-N-S-I-T-E-S dash consulting, and click on the podcast page. We’ll have links to some great articles that go in much more detail on Dichter’s history and work. Also check out Adam Curtis’ documentary “The Century of the Self” for a critical look into the emergence of marketing as a science, including some great footage of Dichter himself. 

If you’re new to podcasting, the best way to listen to this show is with a podcasting app on your phone or tablet. New episodes will show up automatically, and old ones will be deleted, allowing you to listen on the go without burning through your data. Check out our website for more info on how. And finally, please rate our show on iTunes or wherever you get your podcasts because not only does the feedback help us, it really helps other people find our show. Thanks for listening, see you next time

Episode Notes:
Check out these articles from the New York Times and the Economist for a detailed look at Ernest Dichter. For more information on the influence of psychology on business and marketing, check out Adam Curtis’ The Century of the Self.

Music in this episode:
Blue Dot Sessions Darby album
Johnny Dodds, “Blue Piano Stomp“