Why you shouldn’t ask chatbots to act like an expert
A new study from Wharton’s Generative AI Labs suggests prompting chatbots to act like a subject matter expert can actually hurt accuracy.
2 min. read
By testing six large language models on graduate-level questions in science, engineering, and law, researchers from the Wharton School’s Generative AI Labs (GAIL) find that so-called “expert personas” deliver no consistent boost in accuracy, and in some cases worsen the results. The pattern holds across most models, suggesting organizations may get more value from how they use AI than from how they prompt it.
This cuts against widely-promoted guidance from leading chatbot makers—including OpenAI, Google, and Anthropic—encouraging users to assign roles, with prompts that cast the model as a “math teacher” or “tax expert.” But the evidence suggests otherwise.
The study is the fourth in Wharton’s “Prompting Science” series examining how different prompting techniques shape AI performance. It is co-authored by Savir Basil, Ina Shapiro, GAIL senior fellows Dan Shapiro and Lilach Mollick, and management professor Ethan Mollick.
The findings point to a different way of working with AI, coming after a period in which “prompt engineering” was widely seen as a way to get better results, even spawning a new role focused on crafting the right instructions.
Instead, the paper suggests organizations are likely to get more value from how tasks are framed for AI, what gets fed in, and how results are checked, rather than from layering personas on top of prompts.
When the Schuylkill swallowed the city: Lessons from Hurricane Ida’s historic flood
New Penn research shows that Hurricane Ida wasn’t a once-in-a-century anomaly but a preview of how climate change, urbanization, and aging infrastructure are rewriting flood risk.
The Fed explained: What it does and why it matters
Former Philadelphia Fed President Patrick Harker and financial historian Peter Conti-Brown, both Wharton professors, unpack the central bank’s origins, its unusual structure, and the quiet ways it shapes the economy
How population changes are impacting primary education worldwide
Research from Penn sociologist Emily Hannum and colleagues reveals regional trends in whether school-age populations are increasing, plateauing, or decreasing—and shows how different countries are responding.
A bioengineered bean gum from the lab of Penn Dental’s Henry Daniell is found to reduce the levels of three microbes associated with head and neck squamous cell cancer to almost zero, without affecting the beneficial bacteria normally found in the mouth.
Fighting oral cancer with bioengineered chewing gum
Research led by Penn Dental’s Henry Daniell shows that antiviral and antibacterial chewing gums reduce the levels of three microbes linked to worse outcomes in oral cancers, paving the way for more effective and affordable therapies.
