In early June 2020, as Black Lives Matter protests flowered across the United States following the murder of George Floyd, businesses and other institutions rushed to enhance their diversity efforts. Chief diversity officer hires tripled among the largest publicly traded companies, enhancing diversity, equity and inclusion offerings for which U.S. companies paid an estimated $3.4 billion to outside firms that year.

What have we achieved with all this effort? In 2022, this question has special significance, as measures to increase diversity and racial equity have come under political attack, often by people who believe those shouldn’t be goals in the first place. But even among people who believe in the basic mission, common questions about diversity training have shifted from “Which training is best?” to “Is the training even a good idea?” and “Does the training have negative effects?”

The problem is that the real answer to all three of these questions is: We don’t know.

As a behavioral scientist who studies prejudice and behavior change, I can tell you that the situation really is that bad. Last year, my colleagues and I published a comprehensive review of the prejudice reduction literature. We included only program evaluations that used random assignment and control groups, as you would use to check the effectiveness and safety of a drug. Out of hundreds of studies evaluating prejudice reduction programming from the past decade, only two large studies tracked the effects of diversity training. Most diversity training evaluations look like customer satisfaction surveys (“How much did you appreciate this?”) or elementary school worksheets (“Tell me what you learned today about stereotyping”).

Since our review, despite the surge in diversity programming, there have been only a handful of additional studies. In sum, we don’t have good evidence for what works. We’re treating a pandemic of discrimination and racial and religious resentment with untested drugs.

Why don’t institutions evaluate their diversity training programs? Well, take one well-intentioned CEO for a large global corporation who almost got there in 2020. At the peak of the Floyd protests, this CEO sat for a listening session with employees, and in response commissioned more programming from his diversity team. The team hired a new company to train thousands of workers, using a documentary-film-based program that was highly creative but completely untested. After presenting our credentials and a proposal to the corporation’s diversity team, my graduate student and I worked pro bono for months to design a rigorous evaluation of the training.

Capturing the impact of a diversity training might sound impossible, but there are sophisticated ways to do so. One technique is to gradually roll out the program to randomly chosen divisions of the company, comparing trained vs. untrained workers while eventually providing it to everyone. You can track whether the training is followed by more hiring, promotion and retention of underrepresented workers; short of that, you can also measure the behaviors or perceptions of employees around those goals. You can measure how much minority and female employees are heard in meetings by tracking speech and chat contributions with online meeting software. You can also survey employees about racial or gender attitudes among their colleagues, rather than asking them to report their own biases.

We were all ready to roll out a randomized trial of the diversity training. But weeks away from the launch, our agreement with the company fell apart. We’ll never know exactly why the company shut it down, but the company’s lawyers told us that sharing company data was simply too sensitive. This was despite the fact that we agreed to use Princeton’s world-class data security system, to keep the company anonymous, and to report any changes observed as percentages (“a 5 percent increase in hiring women) instead of absolute numbers (“100 more Black employees hired”). Still, it wasn’t enough. The diversity training went ahead, eating up thousands of employee work hours without any clarity as to its effect.

Diversity training’s uncertain impact is a collective action problem on an enormous scale. The need is obvious: Pay inequities are rampant, companies face constant public relations crises in which non-White clientele are treated badly, and 51 percent of employees who recently quit their job cited a lack of belonging as a central reason. But CEOs don’t want to risk a lawsuit by sharing data, or to find out their training has failed. And diversity trainers selling untested programs might not want to risk negative results from a study.

Who can break this impasse? Someone in diversity management once told me that trainings happen when companies either “see the light or feel the fire” — that is, when leaders are either inspired or under pressure. Clearly, the project begins with the former group — or at least with the subset that wants results, not window dressing, and cares about whether resources are well used. But decision-makers should also realize that ignoring whether training works has political costs, too. Diversity training has become a political football, and companies should want to know whether it’s worth the pushback they might get.

Collective action problems require collective solutions. Studies that combine multiple corporations and trainings could shelter participants from legal and PR risks. My research team and other behavioral and social scientists are eagerly waiting to help design these kinds of trials. Because if we don’t study what works when it comes to diversity initiatives, we know what will almost surely follow: another crime of hate, followed by a surge in diversity trainings that might not help at all.

Betsy Levy Paluck is a professor of psychology and public and international affairs, and the deputy director of the Kahneman-Treisman Center for Behavioral Science & Public Policy, at Princeton University.

Read full article in The Washington Post.