Pre FPO - Auditing Platforms for Transparency and Accountability


Madelyne Xiao
Click here for CITP event listing.

Department(s)
Topic(s)

How should online platforms verify identity and information at scale? Many social media platforms and online services have turned to automated technical systems to verify user identity, determine eligibility for access, and evaluate information credibility. Though these systems promise accuracy and ease-of-use, the signals they detect are often imperfect proxies for the properties they claim to measure. Using adversarial audits, user and measurement studies, and observational analyses, this dissertation examines the technical mechanisms by which online platforms establish identity and information credibility and how well these systems work.

We first examine the relationship between on- and off-platform notions of identity verification. In a survey and measurement study of Twitter/X’s paid blue checkmark program, we identify disparities between users’ perceptions of the verification mechanisms required to obtain a blue check and Twitter/X’s actual identity verification practices. We then investigate the black box biometric systems used to enforce age gates online. In addition to designing a novel privacy-preserving methodology for conducting these audits with in-person participants, we find that lightweight facial occlusion attacks can successfully circumvent these systems. We also highlight serious gaps in existing evaluations of these systems conducted by NIST (U.S.) and eSafety (Australia), which do not consider adversarial robustness or liveness performance.

Turning from identity to information integrity, a survey analysis and evaluation of automated misinformation detection pipelines will be presented. We find that the supermajority of automated pipelines are not performant in late breaking news scenarios, and that, absent a unified definition of misinformation, these systems often detect proxies for the presence of misinformation (for instance, virality or writing style). 

The talk will  conclude with an overview of two in-progress works that address disparate aspects of automated information verification and retrieval: first, in an audit study of Twitter/X’s Community Notes platform, we describe a mixed-methods audit framework that compares on-platform measures of polarity to users’ self-reported political polarities; second, in a qualitative analysis of GEO (generative engine optimization) practices, we present some preliminary findings on GEO consultants’ mental models for working: namely, how practitioners conceive of their job roles, implement GEO strategies, and measure performance.

Together, these studies illustrate the challenges of reducing context-dependent social judgments to standardized technical assessments and provide recommendations for improving both these systems and the batteries used to measure their performance.

Bio:

Madelyne Xiao is a graduate student at CITP advised by Jonathan Mayer, with support from a NSF Graduate Research Fellowship. She is broadly interested in online speech, platform governance, and applied cryptography. Current work includes the design and implementation of privacy-preserving systems for content analysis (joint with the Digital Witness Lab), robustness testing of age assurance systems, and an audit of Twitter/X’s Community Notes.

Prior to coming to Princeton, she was an editorial staff member at The New Yorker. She graduated from Stanford University with undergraduate degrees in mathematics and comparative literature.

This talk will not be livestreamed or recorded. 

If you need an accommodation for a disability please contact Jean Butcher at butcher@princeton.edu.

Sponsorship of an event does not constitute institutional endorsement of external speakers or views presented.