• Authors: Kevin T. Greene, Sean T. Norton, Jacob N. Shapiro (Princeton University)
• Title: “Evaluating Text-to-Image Platforms' Content Moderation During the 2024 US Presidential Election”
• Journal: Journal of Quantitative Description: Digital Media

The use of artificial intelligence was widespread in the run-up to the 2024 U.S. presidential election. That fact gave many people pause, as AI is capable of generating text and images that are virtually indistinguishable from reality.

What were text-to-image (T2I) platforms — which use artificial intelligence to generate visual imagery from copy — doing to calm people’s fears? Jacob N. Shapiro and his colleagues at the Princeton School of Public and International Affairs were curious.

“Going into the election, many people were worried about the potential impact of political deepfakes,” said Shapiro, the John Foster Dulles Professor in International Affairs and cofounder of Princeton SPIA’s Empirical Studies of Conflict Project, “so we simply wanted to understand what mechanisms different platforms put in place to address that potential issue.”

Shapiro called the team’s research approach “quite straightforward.” They deployed an automated pipeline to change headlines from prominent news sources into prompts for T2I platforms OpenAI, Stability AI, and StarryAI.

“This process allowed us to create prompts that covered a variety of political viewpoints in a highly realistic manner — that is, that asked for images we thought people might request based on the news of the day,” he said. “We focused on headlines where the subject was either the then-president or -vice president, a member of the Republican presidential ticket, or one of the then-frontrunners for the Democratic vice-presidential position.”

Their timeline comprised 18 straight weeks, covering the final three months of the 2024 campaign and the month after the election, “allowing us to track how T2I systems responded to changes in the political environment in real time,” Shapiro said.

The team then assessed the T2I platforms’ content moderation policies by measuring the rate of prompt blocking and rewriting over time and comparing the consistency in moderation outcomes among the platforms based on their agreement in prompt blocking.

“The basic idea here was to automatically figure out how they responded on a common set of requests for images,” Shapiro said.

The research team identified three key findings:

First, each of the three platforms studied took a different approach, with one doing nothing, one blocking prompts by name, and the third rewriting prompts to avoid asking for named politicians.

Second, one of the platforms significantly altered how it moderated the ability to produce images of politicians. The platform went from not moderating at all to blocking nearly all the images two weeks out from the election – but never announced the change.

Third, there was no consensus among the platforms on what images to block.

“The most important thing to note here is that these platforms have huge power over what you can or cannot do with their tools,” he said, “but may not disclose their policies, and have no obligation to do so.”

With respect to policy follow-ups, Shapiro recommended “ongoing public-interest audits of the systems with large user bases so that people can know how these powerful tools work, and also what values and perspectives companies are expressing through their moderation policies.”

The paper is part of a series from Shapiro and Greene that audits different dimensions of how AI systems work. Last fall, they studied the extent to which four different AI tools help one make arguments on both sides of hot-button issues.

“There, too, we found huge difference across platforms and over issues in the extent to which they will help you make both positive and negative arguments,” he said.