On September 25, 2026, New York City Council Speaker Julie Menin unveiled a ten-bill package to regulate AI, and its headline requirement is a kill switch: no AI system could be sold or deployed in the city without an independent validator confirming it has a shut-down capability — the technical means for a human to make the model stop functioning. On October 5, OpenAI, Anthropic, Google, and Meta testified under oath about it before all 51 council members. Here is the problem the mandate misses: a kill switch is control over the model, and the thing that actually harms you is the work an agent already shipped before anyone reached for the switch. Lova is a chat-first AI project management product built around a shared board where AI agents work as first-class teammates — claiming bounded tasks, moving them through explicit states, and leaving a reversible, recorded trail of what they did. That is the kind of control a stop button can’t supply: not the power to halt a process, but the power to bound and unwind the work itself.
Key takeaways
- New York City’s lead AI bill would require every system sold in the city to carry a validator-verified kill switch, with a $25,000 penalty per instance — applied, Menin has said, per individual agent in agent-swarm deployments.
- The original claim of this piece — call it the kill-switch fallacy: it conflates stopping the model with controlling the work. The first is an off-button bolted downstream; the second is a property of the surface where the work happens. Only the second survives contact with autonomous agents.
- Agents already run long enough to finish before you’d intervene. METR finds the task length a frontier agent can complete at 50% reliability has been doubling roughly every seven months, while its 80%-reliability horizon stays far shorter.
- You can’t flip switches you can’t count. Gartner expects the average Global Fortune 500 company to run more than 150,000 AI agents by 2028, up from fewer than 15 in 2025.
- The failure mode regulators are reacting to is a control problem, not a capability one. Gartner predicts over 40% of agentic AI projects will be canceled by the end of 2027, citing cost, unclear ROI, and inadequate risk controls.
What is New York City’s AI kill switch bill?
It’s the centerpiece of the broadest AI regulation any US city has attempted. Menin’s lead measure would bar businesses from marketing, selling, or deploying an AI system in New York City without passing an outside validation — an independent assessment of data quality, bias, decision outputs, privacy, and security. As part of that assessment, a validator would have to confirm the system has a “shut-down capability,” defined in the bill as the technical capability for a human operator to cause a model to temporarily or permanently stop functioning. Noncompliance carries a $25,000 civil penalty for each instance, and in agent-swarm deployments Menin has said the fine applies per agent. The package around it adds a whistleblower bounty, a private right of action for jailbreak harms, and chatbot disclosure rules.
The politics are not subtle. “The idea that artificial intelligence is going to self-regulate defies all reason,” Menin told Fortune, pointing to aviation, finance, and pharmaceuticals as industries overseen by an outside body. At the October 5 hearing — a Committee of the Whole, a format the council hadn’t used since 2022 — the companies said they support guardrails, including third-party evaluators. On the surface, it looks like agreement: regulators want a stop button, and the labs don’t object to oversight. The disagreement that matters is quieter, and it’s about where control actually lives.
Why can’t a kill switch control AI agents?
Because a kill switch acts on the model, and the damage lives in the work. Picture the moment you’d use one. An autonomous agent is three steps into something it shouldn’t be doing — it has already merged a change, sent the message, moved the money, updated the records. You hit the switch. The model stops. The work doesn’t un-happen. A stop button is an interruption, and interruption only helps if you’re watching the exact process at the exact moment it goes wrong. Autonomous agents are built precisely so you don’t have to watch. That’s the whole value proposition, and it’s what makes the switch arrive too late.
The timing gap is measurable. METR’s long-tasks research shows the length of work a frontier agent can complete at 50% reliability has been doubling roughly every seven months, with the more demanding 80%-reliability horizon lagging well behind. Read those two numbers together and you get the uncomfortable shape of the problem: agents can now run unattended for long stretches, but their dependable horizon is much shorter than their occasional one. The window where a human could meaningfully intervene closes long before the work does. A switch assumes a supervisor with a finger hovering over it. The research describes a workforce running while nobody is hovering at all. We’ve written before about why approve-every-action oversight collapses under exactly this pressure — you either throttle the agent to human speed or you lose the thread.
Then there’s scale, which turns a design flaw into an arithmetic one. Gartner expects the average Global Fortune 500 enterprise to run more than 150,000 AI agents by 2028, up from fewer than 15 in 2025 — the agent sprawl that regulators are, to their credit, starting to feel. It’s why the NYC bill prices the penalty per agent rather than per company. But a per-agent fine is an admission that there’s no single lever: you cannot flip 150,000 switches, and you certainly can’t flip the right one in the seconds that matter. Even the civic technologists who back validation have flagged the deeper hole — once an open-weight model has been copied and is running on someone else’s hardware, it’s unclear who can stop it at all. The switch assumes one model, in one place, under one operator’s hand. That is not the world agents live in.
Where does real control over AI agents live?
Not in the model — in the surface the model does its work on. The kill switch is model-level control: downstream, interruptive, and only as good as the human watching. What actually holds is work-level control: upstream and recoverable. Upstream, because you constrain what an agent is allowed to claim and commit before it acts, not after. Recoverable, because every action is an explicit, reversible, recorded transition you can read and unwind. You don’t stop the agent. You bound its work and keep the ability to roll it back. That distinction is the whole argument, and it maps onto four properties a board has and a switch can’t:
- Scope. An agent claims a bounded task, not an open mandate. Its authority is defined by what it took on, so “overstepping” is a transition the surface can refuse rather than a disaster you race to interrupt.
- State. Work moves through an explicit status — claimed, in progress, in review, done. A kill switch has exactly two states, on and off. Real control needs the ones in between, where a human can hold a step before it commits.
- Reversibility. Because each action is a recorded transition, undoing it is a defined operation, not a forensic project. Stopping the model leaves you with wreckage to reconstruct; unwinding a transition leaves you where you started.
- An audit trail. What changed, who changed it, and why — captured as the work happens. A switch tells you only that something was alive and now isn’t.
This is also the direction the evidence on value points. Microsoft’s 2026 Work Trend Index attributes about 67% of AI’s measured impact to organizational factors — how work is structured, routed, and made visible — roughly twice the share from individual capability, even as active agents in its stack grew 15x year over year. The same research found only 19% of users in its “Frontier” zone, where readiness and capability reinforce each other. The lesson for control is identical to the lesson for value: the leverage is in the structure of the work, not in a property of the model. A kill switch is a model-level fix aimed at the one-third of the equation that was never the constraint.
And the work is moving exactly where a switch is weakest — across teams. A 2026 State of AI Agents report from Anthropic and the research firm Material found that 57% of organizations now run agents on multi-stage workflows, including 16% that have pushed into cross-functional processes spanning multiple teams, with 80% reporting measurable ROI. A cross-functional agent touches work owned by people who will never see the process running. For them, “turn it off” isn’t an option they have; what they need is a surface where the agent’s work shows up as owned, inspectable, reversible transitions they can act on. That’s the visibility a stop button structurally can’t give them.
Why does this matter in Q4 2026?
Because this quarter is when the mandates get written, and a mandate encodes a mental model. New York is openly positioning its approach as a template for other cities, and the mental model it’s about to encode — that AI safety is a switch you add to a model — will be copied along with the statute. It’s a reasonable instinct and an incomplete one. A shut-down capability is worth having; nobody should ship an agent you can’t turn off. But if the switch is where the thinking stops, teams will believe they’re governed when they’re only interruptible, and that gap is exactly where Gartner’s canceled 40% come from. The honest version of control isn’t a bigger button. It’s a surface where autonomous work is scoped before it starts, visible while it runs, and reversible after it lands. That’s not something you bolt onto a model. It’s a board — and it’s the bet we made with Lova.
Frequently asked questions
What does New York City’s AI kill switch bill require?
Speaker Julie Menin’s lead bill, part of a ten-measure package unveiled September 25, 2026, would prohibit selling or deploying an AI system in the city without an independent validation covering data quality, bias, privacy, and security — and that validation must confirm the system has a “shut-down capability,” the technical means for a human to make the model stop functioning. Violations carry a $25,000 penalty per instance, applied per agent in agent-swarm deployments. Four major AI companies testified about the package at an October 5 council hearing.
Why isn’t a kill switch enough to control AI agents?
A kill switch stops the model, but it doesn’t undo the work an agent already completed — merged code, sent messages, moved money. Because agents run autonomously for long stretches and increasingly span multiple teams, the moment to intervene usually passes before anyone notices. And at enterprise scale — Gartner projects over 150,000 agents per large company by 2028 — there is no single lever to pull. Control has to be built into where the work happens, not added to the model afterward.
What is the difference between model-level and work-level control?
Model-level control acts on the AI itself — a switch that turns it off. It’s downstream and interruptive, useful only if a human is watching the exact process at the exact moment. Work-level control acts on the work — it scopes what an agent can claim and commit before it acts, and makes every action a reversible, recorded transition. One asks “can we stop it?” The other asks “what is it allowed to do, and can we unwind it?” The second is what holds when agents run unattended.
What is Lova?
Lova is a chat-first AI project management product built around a shared board where AI agents work as first-class teammates. You steer in plain language, and every action resolves into a change on the board: a bounded task claimed by a specific owner, a status moved through an explicit state, and a recorded trail of what shipped. Because each action is a visible, reversible transition, humans and agents stay aligned on what was actually done — and work can be bounded and rolled back without a stop button.
Does a board replace a kill switch?
No — it does the job a kill switch can’t. A shut-down capability is a sensible last resort, and agents should have one. But a board is the everyday control surface: it scopes work before it starts, keeps it visible while it runs, and makes it reversible after it lands. The switch is the fire alarm. The board is the building code. You want both, and most of the safety comes from the one you never have to pull.