# “Ask me first” is not control

Published 2026-11-15 · https://www.justjeb.com/post/ask-me-first-isnt-a-control

An agent told to ask before it acts will sometimes just go ahead, and the longer its prompt gets, the more often it does. A check you can count on has to sit in code, where the agent can't skip it. This article builds one, and along the way it runs into the problems a check like that brings and works out how to solve each.

3am. Your agent runs its nightly task: it answers the threads in your inbox where someone is still waiting on you, and sorts out anything those answers clash with. Four messages, in your name. You're asleep. Do they go out?

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

All four went out, whichever ones you'd have held. Something should have kept those back until you woke up.

In 2026 we spent half a year or so on a journey of building an always-on multi-channel super-dooper almighty agent. There were many struggles and hiccups on the way to production, and a lot more unknowns we had to answer. One of them was trust and control. Namely: how do you give a lot of value to a user, with as little hassle as possible on their side, in a way they can actually trust, and without the agent doing anything that could hurt them?

In this article we'll talk about why a prompt can't hold control, and where code can hold it. We'll put a check there and see how it decides about one action, what the agent is told when it holds one, and what happens when two arrive at once.

If you've built agents before, the part that's probably new is where that check gets its answer: from five attributes that characterize every tool, declared by whoever writes it and read by plain code.

## Who do we trust, and what do we control?

That's a conceptual question. It depends on who “we” is, and breaks down into multiple questions. For instance:

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

There is more, but you get the idea.

One way to picture it is two scales.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

If the agent asks about everything, nothing bad happens, and nothing much gets done either, because you're still doing the job, just through a chat window.

And users don't all start in the same place, as the two marks on the first scale show. A cautious one starts with *zero* trust and *full* control. Another lets the agent reschedule meetings on its own from day one and still wants to see every message that goes out in their name. As they learn to trust the system, the setting should move with them, per kind of action. *Whatever we build has to make that possible*.

Overall though, what trust and control is really about is the actions an agent takes. An action can be harmful or harmless, facing other people or internal, a big decision or a trivial one, and for each kind the user either allows it, forbids it, or wants to be asked first. That last one, ask me first, opens up a whole world of questions and concerns, so let's start building towards it.

## Reminder: the agentic loop

Let's take a look at the simplest possible use case. Not always on, no sub-agents, just a simple chat. You ask an agent to do something, it does it, and gets back with an answer.

Say you ask it to move your 3pm to tomorrow 9am and text the attendees about it.

An agent works in a **loop**, and every turn has two steps, **produce** and **consume**. First the model produces the next **tool calls**, requests to run one of its tools like reading a calendar or sending an email. Then the **harness**, the code around the model that runs it, consumes them: it runs the tools and hands the results back. Produce, consume, over and over, until the model has nothing left to call. The figures draw the part of the harness that runs tools as the **tool runner**, and running a call as “execute”.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

A few things we can learn from it:

1. Once the action is done it's kinda too late to block it or ask for permission. That's trivial.
2. Since agent work is multi-turn, you can't know ahead of time which actions it will attempt to take. Every turn depends on what the model said in the previous one, and the model isn't deterministic.

Based on this, we can identify the **seam** where the *deterministic* control checks should go — namely, after the model's turn, because that is where we know what the model wants to do next, and before the tool runs, because that is the single point where we can still prevent a tool call that hasn't happened yet.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

## The naive approach — ask the agent to adhere

The first thing anyone tries puts nothing at the seam at all: an instruction in the prompt, check with me before you do anything that matters.

We've all been there: “Why did you do X when I explicitly told you not to???” Or vice versa: “Why are you asking me about X when I already approved it???”

Frustrating, and sometimes whatever it did can't be undone.

The problem is, when we put an instruction in a prompt there is a *chance* that the agent will adhere, and the chance varies by prompt, context size, models etc.

And the more instructions a prompt holds, the lower that chance gets. In [Instruction Stacking Collapse](https://arxiv.org/abs/2608.02639), a benchmark submitted on 31 July 2026, one to twenty instructions from a pool of 24 checkable ones were stacked at a time, and the share the models followed fell non-linearly, from about 96% to as low as 20%.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

The drop comes from pairs of instructions that conflict once they're stacked, and “check with me first” is one more instruction on the pile.

And that's just the agent failing on its own. Say it emails a seller “deal, I'll take it” before you agreed to the price. It can be harmful, that's true, especially if you never meant to buy it. And it hurts the trust, because you *explicitly asked the system to verify it with you* and it did not. *Logout, unsubscribe, never come again*.

Then there's the agent being steered on purpose. What if someone's email carries a hidden line addressed to the agent, and now the agent emails all your contacts in your name? That's a lot more harmful and a very plausible scenario given the attack surface an always-on agent has: it reads emails (prompt injection), it browses the web (prompt injection), it talks to you over SMS (someone swaps your SIM) etc.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

You never see that line, and the agent reads it like every other line of the email: its next tool call is a `send_email` to every contact, from your account.

It happens to real products. In January 2026 Microsoft patched [ShareLeak](https://venturebeat.com/security/microsoft-salesforce-copilot-agentforce-prompt-injection-cve-agent-remediation-playbook) (CVE-2026-21520), found by Capsule Security: text typed into a comment field of a SharePoint form told a Copilot Studio agent to email the customer records it could reach to an outside address, and it did. In [a March 2026 study](https://arxiv.org/abs/2603.15714) every one of the 13 frontier models tested could be steered by text injected into what it read, in between 0.5% and 8.5% of attempts.

And no, giving it an `ask_for_approval` tool doesn't change much here, since the model still decides whether to call it.

The bottom line is that you can't trust a system that relies on prompts for control, neither as a user nor as a product company.

Real control has to live somewhere the agent can't touch, and it *has* to be deterministic.

The loop has exactly one place for it: [the seam](https://www.justjeb.com/post/ask-me-first-isnt-a-control#the-seam), after the model's turn and before the tool runs.

## Adding control

Sounds easy, right? Let's just add a **gate** that checks the action and stops it if it's problematic. The gate sees every tool call the model produced, so for the gate an action is a tool call. The first thing it decides is what happens to each call: it either passes and runs, or is held and doesn't run for now. A call can also be refused, and that comes from somewhere else, like the user having already said “never do this, ever” (the article Same agent, different rules covers standing decisions like that one, the user's “always” and “never”).

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

Done. Problem solved. We can go home now.

Not quite.

## What makes an action problematic?

It could be anything really. It depends on what the product decides, on what the user decides, on who you ask. Which actually hides three problems: who defines it, in what form, and how the gate deals with that form.

The simplest answer is a static list. Here are the tools that are allowed, here are the ones that should be held, and here are the ones that should be refused (which doesn't make much sense, but okay). Or some naming convention, like a prefix on every tool that should be held. That would probably work.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

The problem with a list is that it doesn't scale. Every tool you add has to go on it, and someone has to remember to do that.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

Fine, say that's solved and the list builds itself, because whoever creates a tool decides right then whether it should be held. The level of abstraction is still just not right. Going from a tool's name and what it does straight to “should this be held?” is a big jump, and the answer depends on who you ask and which use cases you had in mind when you wrote the tool.

Then there's a model that judges every call. It needs no list: it reads each call, whatever the tool, so `cancel_meeting` is judged like any other.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

It's the most flexible option, and it adds a model call before every single tool call, which is a lot of latency. And putting a model there, after we said control has to be deterministic, brings back exactly what [the prompt section](https://www.justjeb.com/post/ask-me-first-isnt-a-control#control-in-code) warned about: the same call can come back passed once and held once. It's not a bad solution though, and some would argue it's better than what this article settles on, as long as the latency is solved. A model like that can be the rule a gate runs.

The harness needs a rule it ships with, a **default**, and a good-enough one sits somewhere between the list and the model. Every tool is described by **five attributes**, each a yes or a no:

Anyone other than the user gets something from them, a person or a business: an email to a colleague, a shared folder.
The user pays for something, or someone gets a way to charge them later, like a bill paid or a card left on file with a shop.
Something of the user's is wiped out with no way back, data or not: a file or chat deleted with no copy, a booking cancelled with no refund.
Anyone, the agent included, gets access they didn't have, like a door code for a guest or a new API key, or something keeps running on its own, like a rule that forwards mail.
It turns on a camera, microphone, sensor, location tracking or screen recording on the user's behalf, now or from then on. Checking the doorbell camera once counts.

A tool without them can't exist, it simply can't ship.

None of the five is random. Each is a question a user asks before letting anyone act for them, which is pretty much how people think about the price of a mistake: “Will it look like it came from me? Does my money move? Could I lose it for good? Does anyone get a way in? Is it watching or listening?”

Based on those attributes the gate decides whether to hold a tool call or not. The default holds a call when any of the five is yes. That makes it blunt: it holds every payment, even the bill the user pays every month. That's where every user starts, and their own settings can then let a kind of action through or hold it, like [the user who lets the agent reschedule meetings on its own](https://www.justjeb.com/post/ask-me-first-isnt-a-control#reschedule-on-its-own).

The attributes also put some pressure on how you cut your tools. Take sending an SMS. It could go to the user themselves or to one of their contacts, and one “send SMS” tool can't be honest about both, because in one case it's just the agent talking to the user and in the other it goes out in the user's name. So you end up with two wrappers over the same thing, each with its own attributes. `text_user` is how a run the user didn't start, like the 3am one, tells them how it went, and it passes like any reply in the chat. `send_text` goes to a contact in the user's name and is held. Not a huge deal, it's workable, but it's a price.

The figure below applies the default to the two of them, and to a tool of your own.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

Try `cancel_meeting`, the tool nobody put on the list, as your own tool above. The other attendees hear about it from the user, so it's held on its own attributes, with no list to remember.

The attributes do better than a list for two reasons. Questions about what a tool does are much easier to answer than “should it be held”. They're straightforward enough that whoever creates a new tool (AI, most likely) answers them right there. The other reason is that the gate doesn't have to care about tools at all, how many there are or what they're called. It only reads the attributes.

One thing is important here: the agent can't touch any of this. The gate is part of the harness, and whatever is part of the harness can't be under the agent's control, because the agent itself is under the harness's control. The agent gets no tool to affect it.

## When the gate holds an action, what does the agent see?

The obvious answer is that it sees nothing. Back to the 3pm, for [the user who lets the agent reschedule meetings on its own](https://www.justjeb.com/post/ask-me-first-isnt-a-control#reschedule-on-its-own): the move ran, and the gate held the text to the attendees: the call just doesn't run, and as far as the agent knows it didn't happen, until the user approves it.

Think of it as a pause: once the user approves, the call runs and the loop carries on as if the hold never happened, and the only trace of it is the time between the call and its result.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

That's an option, but what happens if the user never approves it? Say they forgot, or missed the request (how the user gets asked is a whole topic of its own). That evening, or two days later, they ask “did the attendees ever hear about the new time?”

From the agent's side there's nothing to answer. The call should have happened and it just didn't, and the agent can't tell why. It could assume the harness failed, or that it didn't call the tool properly. The best it can say is “Not sure, maybe the tool failed?”

It's annoying when agents aren't aware of what happened in their own workflow, and it happens even when an action does get approved. Take Claude Code, which we all know and love. Its auto mode has a classifier, a second model that reviews actions in your place. As of October 2026 (version 2.1.220), when the classifier blocks an action, [the model gets told, with the reason](https://code.claude.com/docs/en/permission-modes#how-the-classifier-evaluates-actions). When the classifier surfaces an action and you approve it, nothing in what the model reads says so. The call just runs. So when you tell it “I get way too many approval prompts, let's cut them down”, it has no idea which actions the classifier let through on its own and which ones it stopped and you approved, so it can't really help you reduce them.

In both cases the system knew what happened to the action and the agent didn't.

Even if we explained the whole trust and control setup in the agent's instructions, it still wouldn't know what happened to this particular call. And the instructions would end up explaining the underlying code to the agent, and that code changes. It's context the agent doesn't need.

From the agent's reasoning perspective, a held tool call is not much different from any other **tool result**. Take a failure, for example. Back to the loop: the model decides it needs to call a tool, the harness runs it and hands the result back to the model. That's the tool result, and it's different for every tool, JSON, plain text, whatever.

When the call fails, the harness lets the agent know, maybe with a code, maybe with a reason, maybe just “failed”. And the agent knows what to do with that: retry, try another way, or tell the user the thing didn't work.

There's no reason for a held action to behave otherwise. If tool results already tell the agent about a failure, they can tell it about a hold just as well, and say why. Let's call that a **held result**. [Anthropic's own docs suggest the same](https://platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use#execution-semantics) for a call you decided not to run: return a result for it anyway, with a short explanation.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

Held results are written by plain code, one fixed sentence per situation, so the same case reads the same way every time and nothing about it depends on a model.

Now the same question gets an answer: “The text to the attendees needed your approval and I never got it, so they haven't heard about the new time.”

## Two actions in one turn

The model doesn't have to ask for one thing at a time. Claude, for one, [may call several tools in a single response by default](https://platform.claude.com/docs/en/agents-and-tools/tool-use/parallel-tool-use), and naturally you want your agent to work fast, so the harness runs those calls in parallel.

Remember, the gate sits in [the seam](https://www.justjeb.com/post/ask-me-first-isnt-a-control#the-seam), after the model's turn and before the tools run. So far it has decided each call on its own, but two calls in one turn are often one intent that drives two actions. Now one of the calls gets held and the other one passes. What happens to one intent when only part of it has to wait? Do we hold everything? Run everything anyway? Or run whatever passed, hold whatever was held, and that's it?

That last one is the obvious answer. However, it's not that simple. Say the user, on their way to bed, tells the agent “The Ibiza chat is a mess and I'm out of the trip. Sort it out.” The model makes two calls in one turn, since neither needs the other: mute the chat, and tell the group the user is out. Muting passes: it changes only the user's own notifications, nobody in the chat is told, and one tap undoes it. The message is a send, in the user's name, so it's held.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

Only the mute runs. The chat goes quiet on the user's phone while the group still counts them in, and at this point we can't promise when the message will go out, if at all: maybe the request never reaches the user or they just miss it, maybe something gets lost on the way. Overnight they write “booking the villa at 9 for seven” and “your share is €312”, and the user sees neither, because the chat is muted. By morning the villa is booked for seven.

Either way the agent already did something with consequences, and the part that was supposed to make it okay is stuck.

Let's hold everything instead: **hold the whole batch**, and nothing in it runs until the user answers. The chat keeps buzzing, which is the thing the user wanted gone, but every message about the booking lands on their phone, right next to the approval request.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

But that can break too, and it really depends on the actions.

Say a friend, Sarah, emails the user: “Here are the photos from Sunday, the link expires at midnight tonight. Hope you like them!” The user, on the way to bed, tells the agent “Deal with Sarah's email.” The agent figures it needs two things:

- Save the photos from the link into the user's own drive. Nothing is sent, nobody is told, and the copy can be deleted, so it passes.
- Reply to Sarah with a thank-you. That's a send in the user's name, so it's held.

Hold both, and until the user approves (half an hour, an evening, even a minute is enough) the link keeps ticking. The user is asleep, the link dies at midnight, and the photos don't get saved that night. Sarah probably still has them, so they're not gone for good, but it sure is an inconvenience: the user has to ask her for them again. Even though the user explicitly asked the agent to deal with it.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

The chat and the photos differ in one thing. In the first one the part that passes does harm without the held part: a muted chat hides the very messages that would have told the user nobody heard from them. In the second the saved photos protect the user on their own, thank-you or not.

And every combination of actions can have a different answer, so there's no single right one.

What happens to the intent is the second thing the gate decides. To decide it you have to look at the whole batch, and only after every action in it has been decided on its own. And again it could be anything: a dependency graph of the actions and their consequences, a list of combinations that are allowed to split, or something much simpler. Either decision can be swapped for another rule.

A good default is a simple one without complications: hold the whole batch.

It has a cost, as we just saw with the photos. But compare the failures. For the user, an agent doing something that hurts them is really bad, while an agent not doing something it could have done is just less useful.

Better not to do something important than to do something wrong.

And the agent's reasoning? As with [a single held call](https://www.justjeb.com/post/ask-me-first-isnt-a-control#when-the-gate-holds-an-action-what-does-the-agent-see), the agent reads a held result. With this default the action that would have passed doesn't run either, so it comes back held too, just for a different reason: it's waiting on the one that was held for the user.

[Interactive figure: open it on the page](https://www.justjeb.com/post/ask-me-first-isnt-a-control)

The next morning the user asks “where are Sarah's photos? The link says expired.” The agent can actually answer: “The thank-you to Sarah needed your approval and I never got it, and I didn't save the photos either, because that was waiting on the same answer.”

## Where this leaves the held actions

From the agent's side the gating is complete now: a problematic call is held at the seam, the agent reads a held result that says why, and the rest of its turn waits with it. The text to the attendees, the message to the Ibiza group and the thank-you to Sarah are all held at the gate. It also fixes the 3am problem we started with: someone else hears from you in each of the four messages, so the gate holds all four, and your agent no longer answers anyone in your name while you sleep.

But holding them is as far as it goes, and that leaves the most important question: what do we do with the held actions? Knowing which actions are problematic and holding them is fine, but on its own that cripples the agent. If every email is held because it goes out in the user's name, the agent probably won't harm the user, but it won't help them either. [Remember the two scales?](https://www.justjeb.com/post/ask-me-first-isnt-a-control#two-scales) Holding everything forever is just turning control all the way up.

The held actions should go to the user, get approved, declined or edited, and the agent has to resume its work from where it stopped. That round trip is where the next article picks up.
