Mitigating Label Bias

Abstract: Statistical decision algorithms are increasingly deployed in domains where ground-truth labels are hard to obtain, such as hiring, university admissions, and content moderation. In these settings, models are typically trained on historical human decisions, for example using past hiring decisions as a proxy for true applicant quality. However, if past decisions unjustly penalize certain groups, models trained on those labels may inherit those biases. To address this problem, we propose rubric embeddings, a representation framework that replaces standard dense embeddings with interpretable features derived from expert-defined criteria that align with the underlying construct of interest. By anchoring predictions to semantically meaningful dimensions, this approach guards against biased proxy signals. We provide both theoretical and empirical evidence that rubric embeddings mitigate label bias under plausible conditions. Empirically, we evaluate our method on a novel dataset of over 10,000 applications to a large public policy master’s program. We find that models trained on rubric embeddings reduce group disparities while improving measures of cohort quality. Our results suggest that basing predictions on interpretable, domain-grounded representations offers a practical approach to learning in the presence of biased labels. Speaker Bio: Sharad Goel is a Professor of Public Policy at Harvard Kennedy School. He looks at public policy through the lens of computer science, bringing a computational perspective to a diverse range of contemporary social and political issues, including education, the delivery of public benefits, and the equitable design of algorithms. Sharad is the founder and director of the Harvard Computational Policy Lab, an interdisciplinary team of researchers and engineers that use technology to solve public problems. Prior to joining Harvard, Sharad was on the faculty at Stanford University, with appointments in management science & engineering, computer science, sociology, and the law school. Sharad holds an undergraduate degree in mathematics from the University of Chicago, as well as a master’s degree in computer science and a doctorate in applied mathematics from Cornell University.