With shifting sources of competitive advantage, urbanization leading to new mindsets, behaviors and economies, advanced tech-connected living, the world around us is changing rapidly and change will never be as slow again as it is today. These changes create disruptions and uncertainties in many categories. Digital native companies benefit from them and seem to thrive in a world of behavioral data. Their ongoing decision making is built on analytics and insights. Traditional companies try to follow and look for versatile primary data collection and agile decision making. The good news in all of this is that ‘customer insight’ is the number 1 capability to develop according to the C-suite. Especially translating data and analytics into acted-upon insights are key.

The bad news is that many CMI departments are underwhelming. They remain order takers or are perceived as stuffy librarians, while being expected to be a strategic business partner. In addition, market researchers are no longer the exclusive collectors of customer data. Marketers and brand owners have alternatives with DIY tools, and are sitting on a fire hose of big and social media data. The digital disruption from within corporations makes that sales, digital and CRM functions all generate data and often connect with tens of thousands of people a day while traditional research scrambles to recruit 10,000 in a month. Not surprisingly, data-driven marketing and self-service reduce the perceived need for ‘market research as we know it’ among business stakeholders.

So how do CMI departments at the client side deal with this? Based on end-client round-table discussions and an industry literature study, we observed that CMI departments at the end-client side deal with these changes largely in three ways.

1. Reorganizing structure and processes for maximum impact

Several CMI departments (are trying to) gain an independent place in the organizational design. Independent from the commercial marketing function, to operate in full objectivity and serve business stakeholders cross-functionally. Companies like PepsiCo, Diageo and Unilever have developed a ‘purpose’ for their insight function and make insights accountable for business outcomes and impact. These CMI departments become indispensable internal business partners, as they create inclusion and become a part of the business solution in itself. When ‘insights’ is attached to the actual business challenge, it becomes business-ready and a natural thing to do, rather than a last-minute mandatory chore.

To make this work, there is a need for explicit talent management and higher investments in action skills. Talent management refers to managing and positioning ‘insights as a career’. It implies explicitly attracting as well as retaining high-level talent and catching them young, but also composing teams with multi-functional roles (e.g. analytics, execution, production, communication). Some CMI functions develop centers of excellence which innovate and pilot internally and develop knowledge in specific domains. The long-term effect is that we connect with stakeholders while they are still open, and the importance of insights spreads as executives move into new roles in the business. ‘Action skills’ refers to e.g. storytelling or crafting and delivering configurable insights, to expand people’s capabilities beyond the expected functional skills of project management, research and analysis, into communicating, persuading, facilitating, creating and leading. ‘Insights skills’ need to be scaled throughout organizations to help all executives become better at turning data into insights and business results.

This goes hand in glove with building a brand and developing an identity that stands for being assertive, taking ownership and being provocative. At Philips, for example, CMI is redefining its role in the persona of an inspirer, explorer or navigator to show its value and re-assert its expertise.

The basic table stakes for insight professionals in this is that the delivery and communication of insights need to be relevant and appealing for business stakeholders. This can be achieved by delivering multi-media streams of tiny insights, and also by making stakeholders part of the insights initiative.

The insights function that will endure is one that builds the business by being assertive, taking ownership, building belief, showing impact and repeat this!

2. Implementing smart and adaptive research operations

To address the changed processes that come with ‘agile management’, certain end clients develop research capabilities in-house (e.g. booking.com) and automate their repetitive and evaluative quantitative research (e.g. concept testing). PepsiCo for example has developed its own proprietary digital insights and intelligence platform, called ‘Ada’. The automation platform not only generates time and cost gains, it also allows PepsiCo to get smarter over time and unlock meta insights through machine learning. End clients such as Nestlé and Unilever also expand their scope to where the data is, such as customer contact or care data. As consumers have become more outspoken and assertive, these open communication channels provide a wealth of information for companies to mine from. These are not only complaints, but often free ideas and suggestions, and primary data collection suffers from this self-generated data.

All in all, insight professionals must be pragmatic and manage the risk for decision making and the business of those insight projects that are executed faster, at a lower cost and potentially along mediocre quality standards. Especially when dealing with behavioral and social data, there is a need for reconciling rigor and provenance of this data with the business goals. The key question is always: “what does one want to use it for?”. When used as a concrete option by stakeholders, insight professionals have a lot of skepticism regarding social media listening and feel it is their task to educate about the caveats internally. Many clients feel it is oversold and not a true reflection of reality, because there is a lack of profiling information and many conversations are not public. In order for social media analytics to create added value, it is important to look for what people say on these fora that they do not say on other platforms. It is particularly relevant for monitoring and crisis communication, assessing product launches and product reviews, or as an exploratory starting point.

What objective, behavioral or social media data does not do, is fill in the blanks and getting to the ‘why’. To connect the dots and inform decisions along the way, many end clients do add versatile and immersive primary data-collection methods such as structural online communities and digital consumer connect methods (e.g. based on messaging). These allow to conduct more research in less time while saving budget, but also to provide data with a soul. The upside of communities can also lay outside of the digital or online aspect, as senior management sometimes just wants to talk to people, ‘touch and feel’ them. Online communities seem to be in full sync with the current management practices of ‘agile’ and ‘scrum’.

Business stakeholders are looking for integrated solutions which provide a holistic understanding, independent of whether it is qualitative or quantitative. When we deliver empathy, the data matters less.

3. Developing innovative capabilities

Clients want and need to experiment with AI, bots, virtual and mixed reality, geo-location. With these technology innovations, clients are seeking different forms of value. Increased speed and cost reduction are table stakes and always need to be there. Innovative approaches should lead to gains in information quality and generate transformative insights that can be game changers.

As collecting data directly and explicitly from people is harder today, primary data collection methods also need to be aligned with the ‘next gen’ and what people do every day. Hence, agile consumer connections through messaging, chat and video are in vogue.

However, end clients need to separate the wheat from the chaff. Business partners, e.g. agencies and suppliers, have to assist in this by delivering actionable insights and bring innovations that are relevant for the business and not just executional gimmicks. Innovations for insights must always link to business performance (e.g. sales, NPD) and therefore business partners must understand what decisions clients have to make. The core value lays in those new approaches that integrate claimed and behavioral data which help to connect the ‘what’ and the ‘why’.

Admittedly, many innovations require time to mature. Only now are many neuroscience or implicit techniques maturing, and has it become clear what these methods add on top of more traditional methods. Experimenting with qualitative approaches to generate immersive consumer insights is also easier than technologies which impact validation work, and hence the latter take more time.

Vetting the right innovations is a challenge as well as a need for end clients. Suppliers all shout they have the next big thing, but the propositions are often executional or methodological, and do not focus on the business relevance for clients.

Our shared ultimate goal is to build insight capability engines that can paint a true picture of human behavior. Algorithms and data need an overlay with direct consumer input (e.g. contextual observations or opinions) as no algorithm can completely analyze a human mind, and consumers cannot process all propositions made to them. While there is a skew nowadays to predominantly rely on big data or analytics, which are influenced by forces external to the insights industry or profession, we should also avoid creating ‘eco-chambers’. Because if there is one thing, it is that the fundamental need to connect with, understand and experience people holistically is prevalent as never before!