This week, the Secretary of Defense gave Anthropic1Anthropic is the company behind Claude, one of the most capable AI systems in the world. an ultimatum: allow the military to use their AI for “all lawful purposes” by Friday, or he’ll invoke the Defense Production Act to force them.
Anthropic’s position is that it won’t allow its systems to be used for two purposes:
- mass domestic surveillance
- autonomous lethal weapons without human oversight.
The Pentagon’s position is that it can’t be “beholden to a private company” for access to AI capabilities during a crisis. A senior Pentagon official reportedly said they’d make Anthropic “pay a price for forcing our hand.”
I’ve been watching this unfold with a kind of recognition that’s hard to describe. Not because I have any inside knowledge of Anthropic’s situation, but because I’ve been inside a version of this story before.
Twice, I’ve spent time working inside defense – once explicitly on AI for defense. I joined because I genuinely believed in its mission. I still do. I helped build teams and products focused on cybersecurity, big data analysis, and tools designed to protect the US and our allies. I left when the mission I’d signed up for and the mission the organization was actually pursuing drifted apart.
I’m not writing this to name names or settle scores. The people I worked with were talented and sincere, and many of them are still doing work they believe in. I’m writing this because the pattern matters.
It’s the same pattern that’s playing out between Anthropic and the Pentagon right now.
The Mission That Drew Me In
I joined a defense AI startup because the pitch was compelling and, I believe, genuinely meant. Our mission was to help protect US allies. We aimed to strengthen cyber defenses and make sense of the overwhelming volume of data that modern intelligence operations generate.
The team was talented and mission-driven. Leadership truly cared about the work. Our products were defensive in nature. We had a suite of cyber security tools that helped identify and respond to threats. We deployed a data analysis platform that helped allied nations make sense of complex intelligence. I led engineering teams working on these products, and I was proud of what we were building.
Let me be clear, as this matters for the rest of the story: this isn’t a tale about a company that was rotten or corrupt. The mission was real. The people were good. What happened wasn’t about bad intentions. It was about gravity.
The Gravity Problem
Defense AI companies face a structural force that pulls them from defensive applications toward offensive ones. It’s not unique to any single company. It’s an industry-wide dynamic, and understanding it is critical for anyone thinking about the future of AI in national security.
Here’s how the gravity works:
Defensive products (cybersecurity, data analysis, threat monitoring) have diffuse value. They protect broad categories of assets and operations. They’re genuinely important, but they’re hard to attribute direct ROI. The customer (usually the DoD or an allied military) values them, but with modest budgets and glacial procurement cycles.
Offensive and targeting applications have concentrated value. The one primary customer with enormous budgets, and that customer’s most urgent operational needs tend toward capabilities that are further along the kill chain. Target identification. Sensor fusion. Disposition recommendations. Strike coordination.
The pipeline from “monitoring” to “targeting” is shorter than most people outside the defense world realize. The same sensor fusion architecture that detects threats can identify targets. The same data analysis platform that summarizes intelligence can build pattern-of-life profiles. The same AI that monitors anomalies can recommend actions against them.
Including lethal ones.
The technical infrastructure is dual-use by nature. What changes is the intent. It shifts gradually, through a series of scope expansions that each seem reasonable in isolation.
How the Drift Happens
Nobody walks into a conference room and says “let’s pivot from defense to offense.”
First, the defensive products work well. The cybersecurity platform catches real threats. The data analysis tool surfaces real intelligence. The teams are delivering value.
Then someone asks a reasonable question: “Could we extend this capability to cover a broader set of use cases?” The answer is almost always yes, because the underlying technology is flexible.
The scope expands…
Partnerships form with companies whose core business is further along the offensive spectrum. These partnerships make strategic sense; they open new markets, they provide distribution, they bring credibility with defense procurement offices. But they also shift the center of gravity. The partner’s priorities become your priorities, gradually, through the normal mechanics of business alignment.
Then the white papers get requested. Use cases that weren’t in the original pitch start appearing in planning documents. “Monitoring” becomes “monitoring and interrogation of anomalies.” “Threat assessment” becomes “threat assessment with disposition recommendations.” The language shifts by degrees, each feeling small.
The product lines that were working – the genuinely defensive ones – gradually become deprioritized. Not because they failed, but because they’re not where the money is pointing. Resources are reallocated. Teams reassigned. The people who joined for the defensive mission find themselves working on something they didn’t sign up for.
I watched this happen. I participated in it. I’m not claiming I stood apart from it and saw clearly while everyone else was compromised. I was inside the machine, making decisions, managing trade-offs, trying to steer toward the mission I believed in while the ground continued to shift.
The Hardest Part
At some point, the gap between the original mission and the current trajectory became too wide for me to bridge.
We shut down product lines – including the cybersecurity platform that was my whole reason for joining. The whole division. Good engineers doing good work on a product that mattered. Gone, because the organizational priority had shifted elsewhere.
We’d already done the same thing with our data analysis platform. More good people, more mission-aligned work, more capability that didn’t survive the gravitational pull.
I tried to reorient what was left around a product I still believed in. I spent months building a path forward that I thought could work. I hired and trained my replacement.
Then I left.2The decision to stay, though, is also a legitimate choice. People have families, mortgages, teams they care about, and a genuine belief they could make things better from the inside. I respect that choice. Mine was different, but I don’t think it was more virtuous. I’d reached a point where I believed my own leverage to change things had been exhausted.
What I’d Do Differently
I’ve thought about this a lot. There are a few things I’d change about how I approached the experience. I don’t want to relitigate old decisions; I think these lessons are relevant for anyone working in AI right now, defense or otherwise.
I’d ask harder questions during the interview process. Not because leadership was being dishonest – I don’t think they were. The gravitational pull I’m describing isn’t always visible at the start. I’d want to understand: what happens when the DoD asks you to extend a defensive product into an offensive use case? What’s the company’s actual decision-making process for that? Who has the authority to say no, and have they ever used it?
I’d build ethical guardrails into the infrastructure, not just the policy documents. This is the big one. We had policies about appropriate use. Everyone in the defense AI space has policies. But policies are documents that live in a knowledge base and are overridden by operational urgency. What I wish we’d had was technical constraints: infrastructure-level enforcement that required deliberate, auditable action to modify. Policy documents drift silently. Infrastructure constraints create friction, and friction creates accountability.
I’d establish clearer tripwires for myself. Pre-decided criteria for when the mission has drifted too far. It’s harder to draw a line from within a situation and every individual step seems like a small concession. The time to draw your line is before you’re standing on it.
I’d name the drift earlier. In the “I notice we’re having conversations this quarter that we weren’t having last quarter” way. Not adversarially. Often the drift happens because no one says it out loud until it’s already happened. By the time someone raises the question, the answer feels predetermined.
The Pattern Playing Out Again
Which brings me back to Anthropic and the Pentagon.
If you’ve read this far, you can probably see why this week’s headlines feel familiar to me. The pattern is playing out in public, in real time, at an unprecedented scale.
An AI company enters the defense space with genuine guardrails and a sincere commitment to responsible use. The initial contracts focus on broadly defensive applications: intelligence analysis, data processing, decision support. The company partners with a defense data intermediary to deploy on classified networks.
Then the pressure mounts. The customer wants broader access. The language shifts from “specific authorized uses” to “all lawful purposes.” The company is asked to trust that the customer will self-regulate. The company’s position – that pre-negotiated technical constraints are more reliable than trust – is framed as obstructionist, or worse, unpatriotic.
I’ve seen how this plays out when the company gives in. The slide is gradual. Each individual concession seems reasonable. The endpoint is somewhere nobody planned to go at the start.
What makes Anthropic’s situation different and genuinely important is that they’re saying no. Publicly. With hundreds of millions of dollars on the line, with the threat of being designated a “supply chain risk” usually reserved for foreign adversaries. With the Defense Production Act being invoked to potentially force compliance.
They’re holding the line.3Even as I write this, that line is shifting. Earlier today, Time reported that Anthropic is dropping the central pledge of its Responsible Scaling Policy. This is their 2023 commitment to halt training if safety couldn’t be guaranteed in advance. The gravity I’m describing doesn’t spare anyone. Not even the company most publicly committed to resisting it.
The other two items—fully autonomous weapons and AI for strategic decision-making—are harder lines to draw since they have legitimate uses in defending democracy, while also being prone to abuse. Here I think what is warranted is extreme care and scrutiny combined with guardrails to prevent abuses. My main fear is having too small a number of “fingers on the button,” such that one or a handful of people could essentially operate a drone army without needing any other humans to cooperate to carry out their orders. As AI systems get more powerful, we may need to have more direct and immediate oversight mechanisms to ensure they are not misused, perhaps involving branches of government other than the executive. I think we should approach fully autonomous weapons in particular with great caution, and not rush into their use without proper safeguards. – Dario Amodei
I don’t know how this ends for them. I don’t have any inside information. I don’t presume to advise a company with far more resources and context than I have. But I’ve seen what happens when the line isn’t held.
Anthropic’s stance matters more than most people realize. It sets a precedent about whether any AI company can maintain ethical boundaries in the face of government pressure.
An Engineering Problem, Not a Political One
What frustrates me most about the current standoff: it’s being treated as a political argument. It’s actually an engineering problem.
The Pentagon says they can’t call a tech CEO every time they need to use an AI system in a crisis. They’re right. That’s an absurd operational model and no military should be dependent on a phone call to a private company during a missile defense scenario.
Anthropic says they can’t allow unrestricted use of their systems for mass surveillance or autonomous lethal action. They’re also right. Those are legitimate red lines that exist for good reasons.
These positions aren’t actually incompatible. The defense world already solved this kind of problem in other domains. Classified networks have automated access controls that enforce compartmentalization rules without requiring a phone call to the NSA director. Nuclear launch requires dual authorization: pre-negotiated rules, enforced by infrastructure, not by real-time human negotiation. SCIF access is governed by pre-set policies that execute automatically.
The same pattern can apply to AI deployments.
Pre-negotiate the policies. Encode them as infrastructure-level constraints. Deploy them as automated enforcement that neither side can unilaterally modify. Missile defense goes through at machine speed – pre-authorized, no phone calls. Mass domestic surveillance gets blocked automatically – no negotiation, no ambiguity. Both sides get what they need while neither side is “beholden” to the other.
This is a solvable problem.
The conversation hasn’t yet included enough people with understanding of both the technology and the operational reality. That’s why it’s being fought through ultimatums and threats rather than engineering solutions. Both sides need engineers in the room who’ve been there, done that, and understand policy-as-code patterns. The folks who’ve personally seen what happens when the guardrails exist only on paper.
Who Needs to Be in the Room
Two groups are dominating the conversation: people who build the models and people who deploy weapons systems. Both are necessary. Neither is sufficient alone.
What’s missing is the engineering leadership layer between them.
I want to be clear: I’m not anti-defense. I believe AI should be used for national security. Cyber defense, missile defense, intelligence analysis, threat assessment. These are legitimate and important applications. I’ve built products in this space.
I’d willingly do it again.
I also know, from direct experience, that “defensive” and “offensive” are not stable categories in this industry. They drift. The gravitational pull is real, it’s structural, and it doesn’t require bad intentions to operate. Without deliberate engineering and deliberate leadership to maintain the boundary, things drift in one direction only.
The companies that hold the line need people who’ve seen what happens when the line doesn’t hold. They need engineers who can build the technical guardrails, not just write the policy documents. They need leaders who’ve made tough decisions to combat mission drift. Those who understand viscerally why infrastructure-level constraints matter more than good intentions.
This week, watching Anthropic navigate the exact forces I’ve felt firsthand, is a reminder that the work of building responsible AI isn’t theoretical. It’s happening right now, under real pressure, with real consequences.
AI will be used in defense. There’s no question there. What is undecided is whether the people building and deploying these systems have the tools, the infrastructure, and the organizational courage to ensure ethical boundaries hold.
I know what happens when they don’t.
- 1Anthropic is the company behind Claude, one of the most capable AI systems in the world.
- 2The decision to stay, though, is also a legitimate choice. People have families, mortgages, teams they care about, and a genuine belief they could make things better from the inside. I respect that choice. Mine was different, but I don’t think it was more virtuous. I’d reached a point where I believed my own leverage to change things had been exhausted.
- 3Even as I write this, that line is shifting. Earlier today, Time reported that Anthropic is dropping the central pledge of its Responsible Scaling Policy. This is their 2023 commitment to halt training if safety couldn’t be guaranteed in advance. The gravity I’m describing doesn’t spare anyone. Not even the company most publicly committed to resisting it.
