Who Can Stop the Agent?
Designing Accountable Authority for Consequential AI

October 12, 2026

The Question Behind the Authority

Imagine a bank’s AI-assisted fraud workflow. One service flags a transfer for review. Another recommends release because the payment is time-sensitive and the customer may suffer if it is held. Both recommendations likely contain useful information. Neither answers the governance question: who has the legitimate authority to hold, release, or override the payment, and under what circumstances?

This question is a large part of why we’re developing Piscys. As AI systems gain more tools, access, and autonomy, it becomes easier to delegate action. But being capable of doing something and being authorized to do it are not the same. The space between those two states can be where some of the most consequential governance failures begin.

An AI agent cannot become the ultimate authority over another AI agent simply because we put it in charge of monitoring. We would merely have moved the problem up one level. If human agency is missing from that chain, if no accountable person can set boundaries, inspect decisions, revoke permissions, or intervene, then the control surface is already compromised. That isn’t assurance. That is automation acting as its own oversight.

Responsibility Without Authority Is More Consequential with AI

We’ve seen versions of this problem in large organizations. An executive makes a consequential decision about a team. People with direct operational context understand that the decision is likely to damage the work, and they have a responsibility to advise or advocate for affected stakeholders. But responsibility to speak up is not the same as the authority to stop or correct the decision. By the time negative consequences become visible, the opportunity to intervene may have passed.

The analogy has limits, of course: an internal reorganization and an autonomous payment system don’t have equal risk, but the lesson carries across scenarios. Naming an accountable owner AFTER a decision does nothing to prevent harm and doesn’t empower anyone with authority when it was most needed. With AI, consequential actions occur repeatedly and at lightning speed so controlling these scenarios at runtime will distinguish the category-defining players.

Precision around four terms is important: responsibility identifies who is answerable. Authorization defines what a person or system is permitted to do. Human agency is the ability of accountable people to make meaningful choices and exercise control. Enforceability means the operational system actually abides by those permissions and limits. These controls belong together but are not interchangeable. A governance platform that confuses them has the potential to risk documenting accountability without exercising it.

Human Agency Must Always Be the Starting Point

It goes without saying that human agency is foundational to legitimate AI governance. Without it, authority, accountability, and enforceability become impossible to distinguish from decisions the automated system happens to make. People, and the institutions accountable to them, must retain the ability to establish constraints, define delegation, challenge outcomes, intervene, and withdraw authority as circumstances dictate.

Of course, that doesn’t mean a person should approve every model response, tool call, or low-risk transaction. Done indiscriminately, mandatory human approval is equally as worrisome as uncontrolled AI decision-making but in the opposite direction: a disengaged human reviewer produces delay, fatigue, and ultimately rubber-stamping. An intervention control that pauses a clearly unauthorized action under a previously approved policy may be far more protective than waiting for someone to click a button. The important question for our industry is whether that automated control is acting within an authorized, reviewable boundary and whether people retain meaningful power to change it.

This is not an argument that the rest of the industry has overlooked agent identity. NIST’s 2026 work on software and AI-agent identity and authorization reflects serious efforts to concretize these boundaries; its implementation work is emerging, not a mandatory standard. [1] Our question is how to grow that control from access permissions into consequential decisions and accountable intervention.

Access Control Is Necessary But Not the Entire Decision

Modern identity and policy infrastructure already does important work. Microsoft Entra Conditional Access can evaluate supported agent identities before issuing access tokens to protected resources, while AWS AgentCore policy can enforce authorization for requests routed through its gateway. Their scope and limitations are documented. An API key route outside an identity provider, for example, may bypass that provider’s controls. We must build on these capabilities, not pretend they don’t exist. [2][3]

Conversely, access to a resource is not always authorization for every consequential action taken through that resource. A fraud-review agent may be allowed to read transaction data while lacking authority to release funds. An employee may hold a valid role but not have permission to approve an exception above a particular threshold. And an override that is legitimate for one transaction may be unacceptable as a standing permission across an entire payment portfolio.

This is the distinction we want Accountable Authority to make operational: who establishes the relevant operating rules? Who may delegate a decision, and to whom? What is the scope, duration, and basis for that delegation? Who can approve an exception, who can revoke it, and what happens when the required approver is unavailable? Just as importantly, what evidence connects the permission to the action that actually occurred? Those are design requirements for Custody.

When the Payment Decision Becomes Consequential

Here’s another hypothetical scenario: a transfer is flagged as potentially fraudulent shortly before a customer’s payment deadline. Fraud investigators favor a temporary hold. Operations are concerned about harm from a false positive. The AI model supplies a plausible explanation for releasing the transfer. This is not a simple contest between ‘safety’ and ‘speed;’ the bank may face legal or financial obligations, contractual duties, customer harm, and internal rules that determine which decisions are even available.

If the rules require a hold until specific evidence is reviewed, an AI system’s argument for speed cannot waive that requirement. If a bounded exception is permitted, the bank must know which role can authorize it, what evidence that person must examine, and when the permission expires. The ultimate record must distinguish the recommendation from the approval, the approval from the executed action, and the executed action from the actual outcome. Those distinctions are essential if anyone needs to later reconstruct why the funds did or didn’t move.

Now let’s examine two harder cases that will arise in real deployments: two authorized humans disagree. Or a severe incident requires containment while the normal approver is unreachable. We cannot resolve either with a generic ‘human in the loop’ reply. Governance needs a clear escalation chain, permitted emergency actions, limits on exceptions, separation of duties where appropriate, and post-action review. For Piscys, these are explicit design and validation questions. We are developing and testing them before a customer is forced to answer them under pressure.

Transparency Is More Than a Checkbox

Transparency is indivisible from, and essential to, stakeholder trust, but transparency alone is not proof of a good decision. A human may approve something without understanding it, or outside the authority that the organization actually delegated. An automated safeguard may act correctly without a person involved in that individual decision event. Both situations require distinguishing the legitimacy of the decision process from the mere presence of a human signature.

The supporting evidence must answer more than ‘who clicked approve?’ It should link the governing policy and version, relevant context, authorized role, any delegated scope or exception, the time of the decision, the executed action, and the observed outcome. An authorization record that cannot be connected to the actual payment release is weak evidence of operational governance. OpenAI’s foundational agent governance paper and OWASP’s Agent Control Standard provide useful related context. [4][5]

There are also limits to what technical evidence can prove. A complete audit trail may establish what permission was presented to a system and what the system did. It does not automatically establish that a decision was prudent, lawful, fair, beneficial, or in the customer’s interest. Those judgments depend on applicable rules and facts outside the technical record. The right claim is that better, more multi-faceted evidence makes authority more inspectable and contestable; not that cryptography settles every question of responsibility.

Authority As Part of the Control Surface

Piscys is being designed as an independent assurance layer connecting Provenance, Intervention, and Custody engines. The Custody capability is intended to make the origin, scope, exercise, and revocation of decision authority operationally explicit and decisions accountable. It should be possible to see which decisions were within delegated authority, where the boundary was reached, and when a human or institutional decision was required. Constraint Polyphony may inform that broader evaluation, but cannot substitute for legitimate authorization.

For executives and risk owners, this is not merely a technical access-control question. It is a question of whether their organization can still govern its own systems. For customers and regulators, it is a question of whether a harmful or disputed outcome can be challenged with evidence, rather than accepted on the strength of an AI-generated explanation. And for Piscys, it is a development standard: if we cannot show that human authority remains meaningful in the workflow, we have not achieved the assurance we set out to build.

Human agency cannot be reduced to a ceremonial checkpoint at the end of an automated process. It must be the source of that process’s authority and the means by which that authority can be constrained, challenged, and revoked.

Research status

Piscys’ Accountable Authority approach and related Custody capabilities remain in development. The payment scenario is hypothetical, and the organizational example is generalized. No deployed authorization controls, validated escalation or override procedures, demonstrated outcomes, or regulatory compliance results are claimed.

Bibliography

NIST (September 29, 2026), “Comments on Software and Agentic AI Identity Concept Paper.” Original source ↗ 

Microsoft Learn, “Secure autonomous agents with Conditional Access.” Original source ↗ 

AWS Security Blog, “Why Policy in Amazon Bedrock AgentCore chose Cedar for securing agentic workflows.” Original source ↗ 

OpenAI (2023), “Practices for Governing Agentic AI Systems.” Original source ↗ 

OWASP, Agent Control Standard. Original source ↗