Consumers, marketers, and new product developers increasingly rely on LLMs for ideation. Yet ideas generated by independent LLM sessions tend to be less diverse than ideas generated by independent samples of humans, raising concerns that widespread reliance on LLMs could homogenize marketplace ideation and decrease consumer welfare. We identify two mechanisms undermining LLM idea diversity, informed by theories of creative ideation from new product development and cognitive psychology. First, LLMs aggregate knowledge into a unified distribution rather than exhibiting the knowledge partitioning inherent to human populations, where each person occupies a distinct region of the knowledge space. Second, at the individual level, LLMs exhibit fixation just as humans do, where early outputs constrain subsequent ideation. Two product-ideation experiments demonstrate that targeted prompting interventions can address both mechanisms. Persona modifiers based on ordinary individuals improve knowledge partitioning but amplify fixation. This fixation can be mitigated by using chain-of-thought (CoT) prompting, and combining both strategies produces the highest idea diversity, outperforming humans. These findings offer a theoretically grounded framework for understanding LLM idea diversity and practical strategies for leveraging AI's efficiency without compromising the idea diversity that sustains marketplace variety and consumer welfare.
Yuting Deng, Jingxuan Liu, Olivier Toubia, Naman Jain
原文摘要
AI-moderated interviews are emerging as a scalable market-research method for generating consumer insights and building consumer "digital twins." Yet it remains unclear whether they match human-moderated interviews and whether they improve on simpler, static data collection methods. In a pre-registered study with three industry partners, the authors randomly assign participants to AI-moderated interviews (N = 139), static interviews (N = 154), or interviews conducted by professional human moderators (N = 24). AI moderation matches human moderation in depth, covers more themes, and, at equal budget, recovers more customer needs than either alternative, although participants sound more emotionally engaged with a human interviewer. The authors then build a digital twin of each participant and evaluate it against the participant's own held-out responses to six real-world marketing stimuli. Twins built from AI-moderated interviews predict consumer responses better than demographics-only personas, but no better than twins built from static interviews: the richer conversation does not necessarily yield more accurate predictions. Comparing the open-ended explanations provided by participants and their twins shows that prediction errors arise when twins reason differently than the people they represent and when questions fall far outside what the interview covered. AI moderation thus pays off most for discovering customer needs, and less for predicting customer responses.