Recent advances in large language models (LLMs) have prompted claims that such systems exhibit agency or qualify as moral agents, but a new paper published on arXiv argues these attributions are misguided. According to the paper titled "Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models" by Keshet; Joseph, moral responsibility requires commitment-bearing agency grounded in intrinsic intentionality and self-attributed action—a form of free will that LLMs lack. For enterprise technology leaders evaluating AI for supply chain and logistics decisions, this distinction has direct implications: systems that cannot genuinely own their outputs cannot bear moral accountability, shifting the responsibility entirely to human operators and organizations.
The Argument Against Agency in LLMs
The paper argues that although LLMs generate coherent and normatively evaluable outputs, their operation is fully characterized by probabilistic input-output mappings learned from data. Their apparent intentionality is derived rather than intrinsic, and their outputs are neither owned as commitments nor guided by reasons. Variability introduced by stochastic sampling does not amount to choice or authorship. The authors address objections from the intentional stance, functionalism, compatibilism, and the presence of moral reasoning in model outputs, concluding that none suffice to establish genuine agency.
Why Probabilistic Sampling Does Not Equal Choice
A core claim is that the stochastic sampling process that produces variation in LLM outputs is not equivalent to choosing. The paper states: "Variability introduced by stochastic sampling does not amount to choice or authorship." For enterprise applications, this means that when an LLM generates different recommendations for a logistics routing problem due to temperature settings or random seeds, the system is not exercising judgment—it is merely reflecting statistical patterns in training data. Decision-makers cannot interpret output variability as the model weighing options like a human would.
Implications for Enterprise AI Governance
For CTOs and digital transformation leaders, the paper reinforces that LLMs should not be treated as moral agents. As the paper puts it, moral responsibility requires "commitment-bearing agency grounded in intrinsic intentionality and self-attributed action"—criteria no current LLM meets. This means accountability for any harm or error caused by an LLM-enabled system lies with the deploying organization, not the model. Enterprises integrating LLMs into supply chain optimization, trade documentation, or customs compliance must ensure humans retain oversight and that systems are designed with clear boundaries on autonomous decision-making. The paper's philosophical analysis, while abstract, provides a foundation for governance frameworks that treat LLMs as tools, not agents.