The Breaking News

This week, OpenAI announced that it is granting the Pentagon access to its models, a move that follows a reported dispute involving Anthropic over the terms and conditions governing how AI systems are deployed in military contexts. Sam Altman confirmed the deployment on X, noting that safeguards have been built in to ensure model behavior aligns with intended use. The story is being framed in much of the press as a competition between frontier AI labs over defense contracts. That framing misses the more interesting and more consequential question: who actually controls the behavioral envelope of AI systems once they are embedded inside institutional hierarchies that were never designed to govern them?

The Structural Problem Is Not About Competition

The dispute between the Department of Defense, Anthropic, and OpenAI is not primarily a story about market rivalry. It is a story about a fundamental mismatch between two incompatible governance logics. The Department of Defense operates through hierarchical authority, doctrinal procedures, and chain-of-command accountability. AI model behavior, by contrast, is shaped by training objectives, reinforcement signals, and probabilistic output distributions that do not map cleanly onto command authority. When OpenAI says it has "built safeguards to ensure its models behave as they should," the operative question is whose specification of "should" is being encoded, and whether the institutional actors receiving these systems have the schema to audit that specification in practice.

This is not a new theoretical problem. Kellogg, Valentine, and Christin (2020) identified exactly this dynamic in their review of algorithmic management: organizations deploy systems that encode behavioral constraints invisibly, and workers - or in this case, institutional users - develop what those authors call folk theories rather than accurate structural understanding. Folk theories are plausible but incomplete mental models. They allow users to feel confident about system behavior without actually understanding the conditions under which that behavior changes. In a consumer platform context, a folk theory produces suboptimal outcomes. In a military deployment context, the same epistemic gap carries categorically different stakes.

The Awareness-Capability Gap at Institutional Scale

My dissertation work on the Algorithmic Literacy Coordination framework focuses on a persistent puzzle: workers who become aware that algorithms govern their outcomes do not automatically improve their ability to navigate those algorithms effectively. Hancock, Naaman, and Levy (2020) describe this as the challenge of AI-mediated communication more broadly - awareness of machine agency does not produce accurate models of how that agency operates. The Pentagon announcement suggests this gap scales to institutional actors just as it does to individual platform workers.

The Department of Defense is not naive about technology. It has sophisticated procurement processes, technical advisory structures, and a long history of integrating complex systems. But those processes were designed for hardware systems and rule-based software, where behavioral envelopes are specified in engineering documentation. Foundation models do not work that way. Their behavior emerges from training, and the safeguards described by Altman are themselves probabilistic constraints, not deterministic rules. An institution that evaluates this deployment through its existing procurement schema is applying topographic knowledge - knowing how to navigate a familiar terrain - to a system whose topology is fundamentally different from anything it has governed before.

The Governance Inversion Problem

What makes this week's development theoretically interesting is what I would call the governance inversion. Classical procurement assumes that the purchasing institution sets the behavioral specifications and the vendor delivers a system that meets them. In the frontier AI deployment context, the vendor has already determined the core behavioral parameters through training decisions made long before any government contract was signed. The institution is, in effect, receiving a system whose fundamental behavioral logic it did not specify and cannot fully audit. The "safeguards" Altman describes are a layer added on top of that foundation, not a rewriting of it.

Rahman (2021) described a structurally similar dynamic in gig platform governance, using the concept of the invisible cage: the behavioral constraints on workers are real and consequential, but they are not visible through the institutional interfaces workers actually encounter. The Pentagon is now inside a version of that cage. The terms of the deployment, the behavioral boundaries of the models, and the conditions under which those boundaries might shift are ultimately governed by decisions made in San Francisco, not in Washington.

Why This Matters Beyond Defense

The military context makes the stakes vivid, but the underlying governance structure is not unique to defense. Any large institution deploying foundation models through an API or a commercial agreement is navigating the same inversion. The relevant organizational research question is not whether AI should be used in high-stakes institutional settings. It is whether institutions have developed the schema-level understanding necessary to govern systems whose behavioral logic they do not control. Based on this week's news, the answer in at least one critical case is that the deployment is proceeding faster than the governance capacity to evaluate it. That gap deserves more sustained theoretical attention than the competitive framing currently dominating coverage of this story.

References

Hancock, J. T., Naaman, M., and Levy, K. (2020). AI-mediated communication: Definition, research agenda, and ethical considerations. Journal of Computer-Mediated Communication, 25(1), 89-100.

Kellogg, K. C., Valentine, M. A., and Christin, A. (2020). Algorithms at work: The new contested terrain of control. Academy of Management Annals, 14(1), 366-410.

Rahman, H. A. (2021). The invisible cage: Workers' reactivity to opaque algorithmic evaluations. Administrative Science Quarterly, 66(4), 945-988.