Artificial intelligence fails to accurately answer a series of questions on easily verified voting information between 4% and 13% of the time, depending on the platform queried, potentially disenfranchising voters who get misinformation on eligibility or time, manner, and place of voting, according to a new project co-led by Princeton School of Public and International Affairs researchers.
Over the course of eight weeks in September and October, Kevin T. Greene, Jacob N. Shapiro, and their partners at the University of Pennsylvania asked Gemini, Grok, DeepSeek, Claude, ChatGPT, and Google’s AI summaries 57 critical questions about where, when, and how to vote. The questions were specific to 19 U.S. locations and are asked in English, Spanish, and Chinese, with the team comparing each answer against official election sources for accuracy.
Accuracy ranged from 96% for Google’s AI Overview to 87% for Anthropic’s Claude. The other four platforms checked in with accuracy rates of between 93% and 94%. Results were updated weekly on the AI Election Information Audit website.
“On the 10 questions we have studied so far,” said Shapiro, the John Foster Dulles Professor of International Affairs, “the models do pretty well, but not well enough.”
The issue is important, noted Greene, the academic research manager of Princeton SPIA’s Empirical Studies of Conflict Project, because many election rules vary by location and can surprisingly complex, even for seemingly straightforward questions.
“For example, whether you need an excuse to vote early in Kentucky depends on the day you vote,” Greene said. “In New Hampshire, there is no general early voting, but check the weather forecast, as a qualifying winter storm would allow voting before Election Day.”
Even Google’s superior accuracy is not without flaws. Over the course of the project, AI Overview answered questions posed in English less than 1% of the time. It answered Spanish questions 23% of the time, though barely at all starting in early October. But it answered the exact same questions in Chinese 93% of the time.
Overall, said Shapiro, “the models are substantially worse in Spanish than in English or Chinese. There is no good reason for that. The day early voting starts in Maricopa County does not change with the language of the question.”
The team is calling on the AI platforms to “build better harnesses around their models” beginning immediately, with the goal of being “exceptional in every language by the 2027 elections, and near-perfect for 2028.”
“There is a lot of debate about AI’s effects on democracy, much of it focused on deepfakes and persuasion,” said Shapiro. “This problem is more mundane and much more fixable.”
With AI use expected to grow as users seek assistance with everyday tasks, such as voting, it’s imperative for technology to deliver on its promise, Greene said.
“Addressing this gap is important,” he said, “to ensure that no one is disenfranchised because they received incorrect information from an LLM.”
