Computational Consumer Insights
Transcription
Computational Consumer Insights
Computational Consumer Insights: Using AI and Machine Learning to Understand Consumer Preferences Summary The globalization and digitalization of the world have given consumers more information and choices than ever before. At the same time, it has become easier for businesses to collect consumer data from online surveys and other sources. Thus, Consumer Insights has become a key pillar of business strategy. The objective of consumer data analysis is to find valuable business insights. We view Consumer Insights as an information game where the goal is to uncover as much relevant information as possible about consumers, with the least costs and efforts. We believe that human-driven technology is the key to enhancing human capabilities in data insights, by leveraging computational and algorithmic power. We use Artificial Intelligence (AI) and Machine Learning to create the next-generation tools for Consumer Insights. AI and Machine Learning must be used in conjunction with Human insights to create value, and we call this Human-Driven Artificial Intelligence (HD-AI). HD-AI can also be used to optimize survey design and data collection. Asking the right questions increases the quality of consumer data, which in turns improves business insights. Our technology can also automatically process free-text data from open-ended survey questions. February 2016 c 2016 Business Interactive Games Inc. All rights reserved. http://www.bi-games.com Copyright We view Consumer Insights as a game where companies try to uncover as much relevant information as possible about consumers, with the least costs and efforts. Consumer Insights as a Game 1 thods, such as web surveys - with multiple-choice questions, openended questions, or choice tasks have become common ways to obThe globalization and digitaliza- tain consumer data. While busition of the world have given con- nesses can in principle greatly besumers more information and choi- nefit from more data, this is conces than ever before. To remain tingent on their ability to extract competitive in today’s landscape, valuable business insights from businesses have to become more these data. We use Artificial Inconsumer-focused to better meet telligence (AI) and Machine Learntheir needs. At the heart of a ing to create the next-generation consumer-centric approach is the tools for Consumer Insights. quest to understand consumer preWe view Consumer Insights as ferences, perceptions, and behav- a game where companies try to iors (which products they want, uncover as much relevant informahow much they are willing to pay, tion as possible about consumers, how much they like a brand, etc). with the least costs and efforts Consumer Insights is therefore ne- (Fig. 1). One needs to solve an incessary for a successful business formation maximization and cost strategy. minimization problem using new At the same time, it has be- computational techniques. The come easier for businesses to col- ultimate goal of the game is to lect large and diverse sets of con- capture consumer utility functions sumer data from sources such as (perceptions, preferences, choices transaction data, loyalty programs, and behaviors). These functions or consumer surveys. Online me- are predictive quantitative mo- CONSUMER INSIGHTS AS A GAME Collect Data Consumers Integrate & Visualize Data Analyze Data & Derive Insights OBJECTIVE HD-AI Consumer Analytics Capture consumer utility functions Automatic Free-Text Analysis • Perceptions • Preferences • Choices & Behaviors ? Ask questions Dynamic Survey Design Fig. 1: Our innovations in Consumer Insights dels of consumer choices, with which we develop Computational Economics models and Computational Business Games. For more information, refer to our white paper “Computation Strategy: Using Games for Decision Making”. We believe that human-driven technology is the key to enhancing human capabilities in data insights by leveraging computational and algorithmic power. Using AI and Machine Learning, we propose three new techniques for Consumer Insights: Human-Driven AI (HD-AI) Automatically discover all significant relations in data to improve human insights Dynamic Survey Design Leverage HD-AI to adjust survey questions during data collection and increase data quality Automatic Free-Text Analysis Automatically extract normalized and quantitative data from freeform text inputs Human-Driven AI The objective of consumer data analysis is to find valuable business insights. One type of output from statistical data analysis is correlations (‘Variable A is dependent on Variable B’), which yield insights about cause/effect relationships (‘someone who buys shampoo is likely to buy condi- c 2016 Business Interactive Games Inc. All rights reserved. http://www.bi-games.com Copyright We use Artificial Intelligence (AI) and Machine Learning to create the next-generation tools for Consumer Insights. tioner’). Another type of output is clusters (‘Values of Variables A, B, and C occur together’), which yield insights about consumer segments. We use custom AI and Machine Learning algorithms to automatically identify all significant correlations and clusters in consumer datasets. While correlation does not imply causation, the lack of correlation always implies the lack of causation. This means that if a variable does not exhibit a significant relationship to another variable, it can be ignored. Our technology is based on cutting-edge research in hierarchical Bayesian networks, tree learning algorithms, and constraint preference encodings. Our tools automatically identify a small set of potentially interesting data relationships from many thousands of possibilities, so that human analysis can quickly hone in on the most valuable business insights (Fig. 2). Fundamentally, no algorithm or machine can yield data insights on its own, without human intelligence. We believe that AI and Machine Learning must always be used in conjunction with Human insights to create value, and we call this approach Human-Driven Artificial Intelligence (HD-AI). Dynamic Online Surveys Beyond data analytics, HD-AI can also be used to optimize the design of online surveys used for Consumer Insights. We view survey design and data collection as an information-maximization and cost-minimization problem. The objective is to learn information from each survey question, but each question has a cost. It generally costs money to pay people to answer questions and longer surveys take more time to complete. Moreover, the attention of respondents also typically diminishes with the length of the survey. Therefore, it is important to only ask questions that give valuable insights, rather than wasting valuable respondent time on useless questions. Asking the right questions increases the quality of 1 Artificial Intelligence Use range of Machine Learning techniques to discover ALL significant variable correlations and clusters in datasets. Sets of correlations between variables consumer data, which in turns improves business insights. We propose a new dynamic approach to online surveys where HD-AI analytics are applied on the data after an initial set of responses have been collected, to refine survey questions. The goal is to automatically test for potential variable correlations (or the lack thereof) and adjust the set of survey questions based on the results. Questions that are highly unlikely to yield insights are removed and replaced with other questions. Consider a question where 97 out of the first 100 respondents selected answer A. At this point, we can be very confident that the answer is almost always A. Therefore, asking this question again to other respondents would be unlikely to yield any new insight, and another potentially valuable question could be asked instead. 2 Human Intelligence Use business expertise to explain causation relationships underlying correlations found by Machine Learning. Business insights and recommendations Fig. 2: HD-AI Consumer Analysis c 2016 Business Interactive Games Inc. All rights reserved. http://www.bi-games.com Copyright 2 Our technology automatically turns free-text survey responses into normalized quantitative data that can be analyzed by statistical tools. In essence, the automated approach of HD-AI reduces the cost and time of data analysis, making it easier for humans to test the quality of survey data “on the fly” and adjust survey questions accordingly. This testing process can be repeated throughout data collection to hone in on the most valuable set of questions – leading to higher quality consumer data. Better data begets better insights. 3 time and labor-intensive to analyze even for small sample sizes. Typos, multiple-word inputs, lack of word space all make free-form text questions hard to analyze with common statistical tools. It requires human workers to manually read, classify, and count text responses. Our technology based on Natural Language Processing (NLP) and Computational Linguistics can process free text fully automatically. It turns free text inputs into normalized quantitative data Automatic Free-Text that can be analyzed by statistical tools. Consider the example Analysis of asking respondents to name Consumer surveys often ask open- their favorite brand. From the ended questions where respondents list of misspellings, abbreviations, enter their own answers as free unnecessary words, or incorrect text. While free-form text ques- phrases, our algorithm automatitions remove the bias of multiple- cally identifies the intended brand choice questions, as there is no names and groups them for frepre-defined set of answer choices, quency analysis, with no prior they are also traditionally very knowledge of brands expected in the data (Fig. 3). In many respects, AI and Machine Learning are powerful tools to extract insights from data. In the game of Consumer Insights, the better you are at using these tools, the closer you get to the goal of understanding consumer utility functions. Authors Aurélie Mei-Hoa Beaumel is the CEO of BIG. She has BA/MS degrees from Stanford University in Economics and Cognitive Science, and has multi-year experience in International Business Strategy. Thomas Dillig is the President of BIG. He has a PhD from Stanford University in Computer Science, and has a decade of experience in cuttingedge Computer Science research at top academic institutions. For more information, visit our website at www.bi-games.com and contact us at info@bi-games.com. “What is your favorite brand?” 50 shades of Louis Vuitton • • • • • • • • • • • • • • • • • louie v louie vaton louie vatton louie vitton louie voutton louie vuitton louie vuttion louie vutton louisvutton louis vatton louis veton louis vetton louis vinton louis viotton louis vitaan louis vouton louis vouttan • • • • • • • • • • • • • • • • • louis vouttion louis voutton louis vuiton louis vuittion louise vuitton lous vouitton lous vuitton lousi vuitton louis vitan louis viton louis vittan louis vittion louis vitton louis vittuon louis viuton louis viutton louis voiuttan • • • • • • • • • • • • • • • • luis vuttion louis votton louis vouitan louis vouiton louis vouitton louis voutain louis voutin louis vouton luie viton luis vatton luis vitton luis vittons luis vouton luis vuiton luis vuitoon luis vuitton BIG Automatic Free-Text Technology Output Louis Vuitton n = 50 Algorithm groups all (mis)spellings of a brand for frequency analysis Brand Name Frequency Fig. 3: Example of Automatic Free-Text Analysis c 2016 Business Interactive Games Inc. All rights reserved. http://www.bi-games.com Copyright