Disability justice Rather than attempting to promote fairness solely through technical adjustments to training data and input–output relations, data scientists should take up a disability justice approach. This view acknowledges that factors such as error rates and accuracy are necessary but insufficient to understand, identify, and address algorithmic bias. Disability justice serves as a framework for reasoning about how ableist structures and norms subtly configure and restrict the ostensibly objective aspects of AI design. A deeper analysis of disability and capacity not only draws attention to the value-laden nature of design processes but illuminates how data analytics and machine learning help to define and redefine those concepts. Instead, a disability justice approach suggests drawing a wider circle of considerations relevant to AI ethics; centering analysis of assumptions and values in design; closing gaps between stakeholders; and pursuing policies that empower further activism. Disability justice calls into question fair-ML’s dogged quest for objectivity and failure to confront broader socio-political norms, demanding that data scientists attend to the subjective choices intrinsic to algorithm design. This shifts from a sole focus on technical aspects of design—such as input–output relations and training data—and expands the problem space to include the social responses and outcomes that algorithms generate. While data scientists need not abandon technical considerations, these decisions must follow from an analysis of the normative choices that go into, e.g., generating classifications, formulating problems, labelling data, and selecting proxy variables. That is, designers should take a broader socio-technical perspective, which considers systems as not merely technological but enmeshed with embodied, organizational, social, and political factors (Simon et al., ; van der Bijl-Brouwer & Malcom, ). By considering how these dynamics interact and affect AI design together instead of assessing them in isolation, data scientists can better conceptualize, identify, and respond to biased algorithms. For instance, by reflecting on values woven into targets like ‘fit,’ one can parse out legitimate indicators of employability, such as word choice, from ones steeped in ableist norms and unrelated to performance, such as speech patterns and tone. Widening the circle of factors considered relevant to ethical AI design beyond data and input–output relations not only alters design processes but also renders thinkable a wider range of alternatives. Looking beyond the optimization of technical fairness criteria intervenes in solutionism and reveals a variety of other mechanisms for combatting bias and promoting social transformation. Notably, this gives us the tools to situate algorithms in a socio-technical context and raise the question of whether algorithms are appropriate instruments to measure and shape certain domains in the first place. For instance, given concerns about algorithms’ ability to capture shifting, contextual aspects of disability, what amount of discretion should they have over disabled people’s hiring outcomes, if any? While simply omitting groups from algorithmic models would exacerbate inequality, there may be reason to be much more skeptical of their inferences. Already, protests by Google employees about collaboration with the Departments of Defense and Homeland Security on Projects JEDI and MAVEN (Wakabayashi & Shane, ) and pledges by computer science students to refuse work at Palantir in light of its cooperation with Immigrations and Customs Enforcement (Birnbaum, ) point to the ways in which data scientists can affect change beyond the algorithm itself. As such, disability justice indicates that we ought to expand the scope of AI ethics and situate algorithms in their socio-technical context. Moreover, a disability justice approach suggests that data scientists should critically analyze the ways in which disability is defined, measured, and labelled. Given that most data about disability are medical, inferences from them typically are limited to the medical model, reducing disability to a biological dysfunction. Yet, by drawing on disability scholarship, we can uncover the ways in which these categories shift and trace their roots to ableist assumptions. For instance, consider proposals to increase the amount and quality of disability data by drawing from people’s electronic health records. While well-intentioned, these efforts would further entrench the medical model of disability, leave little room for self-identification, and underrepresent disabled people who lack access to health care. Making explicit the ways in which algorithms model disability creates space to imagine it differently through methods like value-sensitive design (Friedman & Hendry, ; Simon et al., ). Such an approach melds conceptual, empirical, and technical elements, e.g., assessing models of disability in terms of their assumptions, impact on data gathering, relationship to outcomes, and so on. In so doing, data scientists could better attend to the ways that different models influence people’s claims about their disabilities and how those claims are mediated by intersecting hierarchies of class, race, and gender. By investigating how different definitions and measures of disability shape who ‘counts’ as disabled, data scientists can intervene in these ongoing omissions. This requires altering how disabilities are operationalized so they are defined not solely as medical diagnoses but also in terms of how people navigate institutions, norms, and each other. While fair-ML idealizes abstraction so that data can be applied across contexts, (in)capacities emerge from a multitude of contextual factors like competing values and built environments. Instead, a disability justice framework suggests that an algorithm’s accuracy does not always trade off with its ability to accommodate contingency because many disabilities are intrinsically situational and resist quantification. Implementing this approach might involve, e.g., assigning more weight to contextual factors, assessing how aspects of a given disability shift over time, and designing scales that are sensitive to social values and attitudes. For example, some researchers have demonstrated that disability data can be disaggregated to more easily study the variables that cooccur with certain disabilities or contexts (Brown & Broido, ; Peña et al., ). Others have called for analyzing how people’s capacities expand and contract in relation to built accessibility, office culture, and the type of labor at hand (Bennett et al., ). This suggests that data scientists could model disability differently by altering processes of data collection, label generation, and proxy selection so as not to abstract away its context. In addition, disability justice demands more fundamental shifts in access, communication, and ownership across the AI design pipeline. Indeed, the biases that hiring algorithms exhibit are inseparable from developers’ sole focus on employer values and widening asymmetries between firms and applicants. Although applicants’ ability to secure jobs is shaped by these AI, they cannot examine or offer input on them easily, if at all. Expanding opportunities for these engagements and removing barriers to access is essential to designing more equitable algorithms. Already, initiatives like Access Computing offer accessible training in computer science (Trewin et al., ) and groups like (Dis)Ability in AI provide support to disabled people at conferences about AI (Whittaker et al., ). Despite efforts to increase disability representation at design forums and in training datasets, the mere presence of disability is not enough to transform existing institutions and restore trust. That is, vendors must design with disabled people, not merely for them. As the scope of design expands, data scientists ought to draw insights from other disciplines, including disability studies and socio-technical design, and team up with a wider range of stakeholders. This process cannot simply tack on these perspectives but must integrate and consult with them throughout the design pipeline. By collaborating with a diverse array of disciplines and voices, data scientists could better identify stakeholders in AI hiring, explore their values and needs, study how AI might implicate them, and involve them in design processes. This bears similarities to a participatory design perspective, in which workers are empowered by directly influencing the design of technologies they will take up (Korsgaard et al., ). Using a variety of techniques—from prototyping and mock-ups to future-oriented workshops and scenarios—stakeholders reflect on the situation, imagine innovative solutions, and transform those speculative approaches into realistic, concrete alternatives. This might involve, to give an example, asking stakeholders to evaluate whether automated interviews that assess affect and gestures align with their values, what they communicate about disabled applicants, and how they might be navigated. In parallel, insights from disability studies suggest granting disabled people’s perspectives significant weight in debates about, e.g., generating labels and selecting proxies for disability status. For instance, developers might look to data platform co-ops operated by people with rare disabilities, in which they oversee the use of their data and share profits (Scholz, ; Treviranus, ), to inform their own decisions about responsible data use. By working alongside these other disciplines and voices from the start of the design process, data scientists could help bridge gaps in knowledge, foster a common conceptual language, and develop new research methods. Beyond expanding the scope of AI ethics concerns, redefining design values, and changing stakeholder dynamics, a disability justice approach suggests pursuing policy measures that can empower data activism. While legislation often lags behind advances in computing, existing regulatory uncertainty allows AI vendors to selectively endorse transparency and fairness criteria in ways that preserve existing arrangements. Greater transparency could help promote the conditions necessary for activists and data scientists alike to study, uncover, and contest algorithmic bias. Despite the explanatory difficulties associated with the black box, policies ought to be introduced mandating that AI developers regularly audit and disclose the purview, inputs and outputs, and limitations of their algorithms. For example, some have suggested requiring employers to make public the automated inferences they make about applicants (Hoffman, ) and tasking them with retaining detailed records of their bias mitigation procedures (Hoffman, ). Developing legislation to establish minimum standards for data transparency and disclosure of bias mitigation measures—even for opaque machine learning AI—is necessary to identify and make claims against discriminatory algorithms. In addition to setting baselines for transparent design and implementation, recognizing how ableist structures can affect people’s identification with and disclosure of disability status highlights the importance of more robust privacy regulations. Existing protections are ill-equipped to grapple with the ethical risks posed by automated hiring tools or predictions of disability status. The ADA (1990) only shields those who are disabled, treated as disabled, or were disabled, while GINA (2008) and HIPAA (OCR, ) protect health data while it is being stored, transferred, and analyzed by certain classes of covered entities. However, since these laws neither cover AI vendors and data brokers nor explicitly forbid discrimination based on inferred risk factors for disabilities, AI can circumvent them (Givens & Morris, ). Similarly, while the ADA prohibits discrimination in places of public accommodation, it does not explicitly include algorithms, social media platforms, and so on (Rothstein, ). Expanding existing privacy protections would help make disability disclosure less precarious and, in so doing, assist efforts to gather more representative datasets.