As artificial intelligence (AI) systems assume more decisionmaking roles in government, the economy, and society, a question emerges: Will humans retain the capacity to shape collective outcomes? Several theories suggest that, once human decisionmaking erodes past a certain threshold, the skills, institutions, and political standing needed to reclaim that decisionmaking capacity may no longer exist. However, no widely accepted metrics exist for tracking this erosion. In this report, the authors draw on social choice theory to develop a formal model of how AI erodes collective human agency; they also model decisionmaking in terms of coalitions and propose quantitative metrics for tracking shifts in the distribution of decisionmaking power to identify the point beyond which those shifts could become irreversible.
Key Findings
- Agency erosion is measurable across domains. Three metrics—distribution of decisive coalitions, minimal coalition size, and composition of minimal coalitions—are applicable to a wide variety of decisionmaking processes. These metrics provide a framework for tracking human agency impacts across domains, comparing changes, and detecting nonlinear acceleration.
- Distinct mechanisms drive agency erosion. There are three pathways through which AI reduces human agency: human disenfranchisement (fewer humans in decisionmaking roles), AI enfranchisement (AI entities gaining decisionmaking power and changing the composition of decisive groups), and AI agenda control (AI systems shaping which alternatives reach human decisionmakers to consolidate power in unintended ways).
- A terminal state exists. The mathematical structure of the model identifies a formal end state of agency erosion: a single minimal coalition that is decisive for all choices. This provides a target for monitoring how far the trajectory is from this point of irreversibility.
Recommendations
- Develop agency evaluations. Existing AI evaluations assess capabilities, safety, and alignment, but they do not assess structural effects on human decisionmaking. Researchers should design benchmarks that measure when AI systems reduce the number of humans in decisive coalitions, influence outcomes, or shape which alternatives reach human decisionmakers.
- Establish human participation thresholds. For high-stakes domains (such as democratic governance, military applications, and critical infrastructure), policymakers should consider minimum requirements for human presence in decisive coalitions. These thresholds should reflect domain-specific requirements for legitimacy and reversibility. Organizations, such as the National Institute of Standards and Technology, could incorporate coalition composition as a measurable dimension of AI risk alongside existing metrics for reliability and security.
- Monitor coalition composition longitudinally. The danger of gradual disempowerment is that no single change appears catastrophic. Organizations and governments should track the human composition of decisive coalitions across domains over time.
- Benchmark reversibility capacity. Organizations should assess whether they could restore human decisionmaking if AI-driven agency loss accelerates. Doing so would require maintaining the human expertise, institutional knowledge, and deliberative infrastructure needed to reverse course.
– Alvin Moon, Benjamin Boudreaux, Published courtesy of RAND.
