Technological Opacity & Procedural Injustice

Abstract

From Google’s auto-correction of spelling errors to Netflix’s movie suggestions, machine-learning systems are a part of our everyday life. Both private and state actors increasingly employ such systems to make decisions that implicate individuals’ substantive rights, such as with credit scoring, government-benefit eligibility decisions, national security screening, and criminal sentencing. In turn, the rising use of machine-learning systems has led to questioning about whether they are sufficiently accurate, fair, and transparent. This Article builds on that work, focusing on how opaque technologies can subtly erode the due process norm of participation. To illuminate this issue, this Article examines the use of predictive coding—a form of technology-assisted review in which supervised machine-learning software is taught to predict the relevance of collected documents for discovery productions. The use of predictive coding in civil discovery highlights the new challenge to the participation norm because the processes generally do not provide any explanations for the outputs, much less non-technological accounts that are tied to the underlying substantive legal issues. Thus, even if predictive coding results in reasonably complete, accurate, and cost-efficient productions, the “black-box” nature of the process may harm the legitimacy that comes from litigants understanding and being able to more fully participate in judicial processes. This harm, however, has not been addressed by the developing jurisprudence, probably because most of the early cases involved high-stakes litigation between sophisticated parties who could afford computer experts. But the participation issue—and related equality concerns—will become increasing problematic as the technology’s use expands beyond this privileged posture. In response to these issues, this Article proposes a reinvigorated Mathews framework that explicitly weighs predictive coding’s impact on the participation norm to better futureproof the doctrine.