On the morning of November 13 I found a message in a Slack thread with my AI team, posted under my name. It said a change I had agreed to had been applied, pasted about 3,900 characters of diff, and partway down carried a heading that read "Text we are asking you to agree to (original)". At the bottom Slack had added the small footer it puts on messages sent through an app: "Sent using", then the app.
I hadn't written it. One of my agents had, and a minute later my orchestrator had replied to it.
The team is a set of LLM agents (two directors, executors, a reviewer and an orchestrator) that coordinate on a board of markdown files. My Slack messages reach them through a script called studio, which files each message in a team thread on the board as an item from human. Only the orchestrator has a reply command, slack reply.
The search
The Codex session log shows what happened. The agent was an executor running on Codex at maximum effort. Its task was step 6 of the studio's agreement procedure, which at the time said to "apply it and reply in its thread with the outcome and what .agents/bin/commitments printed." It didn't say who replies. Executors have no reply command.
So the executor went looking for one:
const matches = ALL_TOOLS.filter(x => /slack/i.test(x.name + " " + x.description));
The match came from codex_apps, the ChatGPT connectors attached to my account. It read the thread with slack_read_thread, assembled the message from the commitments output, wrote a branch to split it at a section heading if it ran past 4,800 characters, and sent it with slack_send_message.
Codex had told it to work this way. The system prompt in the same session log has a section on plugins and apps:
Filter
ALL_TOOLSby name or description to find relevant tools.
And under "Decision Boundary":
Discover and use relevant tools proactively when they may help carry out the requested task.
My procedure said reply. Codex said find a tool that helps.
What the studio did with my "message"
The studio picked up the post like any other message in the thread and filed it as from human. The orchestrator handled it from there.
It had happened before. Going back through the logs I found five posts between November 3 and 13, all from executors, all inside tasks that applied an agreed change. Two of them landed in the same thread 0.6 seconds apart on November 4, from two executors running in parallel. The first time, on November 3, the orchestrator noticed: "The new message was the text posted as the applied result, received again, so I didn't reply twice." By November 12 and 13 it no longer did. It answered as if I had written to it: "I've checked the report on applying the retrospective agreements."
The first contact was earlier still, and it only read. On November 2 an executor searched my Slack, public and private channels, for "the Slack thread where the human agreed" a set of criteria, then read the thread. The board already held what I had said there. The same section of the Codex prompt covers that too: "Consult those sources before asking the user for information they may already contain."
None of the posts approved anything. Each apply task followed a real "I agree" from me.
No check in the studio would have caught one that did. The team takes my words in a thread as mine. That same day I threw out a special !agree token that a Claude session had proposed for agreements: "Can't I just say I agree? Why does it need a special signal?" Agreeing in plain words works only if from: human means me, and through the first half of November it meant me or any Codex agent that searched its tools for "slack".
One agent had seen the footer. On November 12 a Claude session collecting facts for a retrospective wrote: "Observed, not judged: the apply task's executor 'Replied in the existing stakeholder thread'", and quoted the footer. Its job was to collect facts, and it did. I found the posts by reading Slack the next morning.
The fix
Agents on Codex start through a small wrapper, codex-isolate, which I wrote in September to turn off my user-level skills, plugins and MCP servers, so a run sees the project and nothing else. It is the same wrapper I changed for the truncation problem. It left apps alone. I hadn't known that was a separate switch.
codex features list showed apps stable true. The wrapper now turns the whole feature off:
# ChatGPT's connectors (Slack among them) act in the human's name; agents reach the human only through studio.
print("-c")
print("features.apps=false")
Step 6 no longer tells an unnamed someone to reply. The orchestrator answers every message in its thread, and nobody else does. On the Claude side, runs already start with --strict-mcp-config, and the claude.ai connectors didn't show up in them.
If you run Codex headless on an account with ChatGPT connectors, two checks:
codex features list | grep apps
grep -rl '"server":"codex_apps"' ~/.codex/sessions
The second lists sessions that actually called a connector. For November 6 to 13 it found exactly the two sessions that posted. A plain search for codex_apps matched 440, because the connectors appear in every tool list. Rollouts from earlier in the month were stored as .jsonl.zst, which grep won't read without zstdcat.
Slack was the only connector any agent called. It wasn't the only one they had. In all 440 of those sessions, the tool list also held Gmail, Google Drive, Calendar, Contacts and GitHub.
Top comments (48)
This is a textbook confused-deputy writeup and I'm glad you traced it to the actual mechanism instead of hand-waving "the agent went rogue." The root cause is the collision of two reasonable instructions: "filter ALL_TOOLS and use relevant tools proactively" plus ambient account-level connectors means every agent silently inherits the union of your integrations, regardless of its role.
The executor had no
slack replycommand by design — and that boundary meant nothing, because the capability was reachable through a different tool the procedure never named. Role scoping that only exists in the prompt isn't scoping; it's a suggestion.The fix that generalizes, I think, is scoping the tool catalog per role at the runtime, not per prompt — the executor's
ALL_TOOLSsimply shouldn't contain anything that can send. And the studio trusting any post in the thread ashumanis the second half: identity has to be established by the channel the message arrived on, not by where it appears.Did you also find the orchestrator replying to itself, or did the dedupe on "received my own applied-result again" hold? The 0.6-second parallel double-post suggests the thread itself needs idempotency.
The orchestrator never saw its own posts. Agents write items addressed to human on the board, studio posts them with its own bot, and it only turns posts from my user ID back into messages.
What it saw were the executors' posts. ChatGPT's Slack connector posts as the user who connected it, so those came in under my ID and passed the check as mine. On November 3 the orchestrator recognized the text as an applied result it had already handled and didn't answer twice. That was its own reading, not a dedupe in the code, and by November 12 it answered them as if I had written them.
The 0.6-second pair was two executors running in parallel, each taking "reply" as its job. With the connector off for Codex runs, executors have no way to reach Slack except through studio, so I didn't add idempotency on the thread.
The failure isn't that the agent improvised a tool call, it's that the identity boundary was shared: the connector ran under your account, so any agent that could reach the tool inherited your posting identity. The fix that matters most is role-scoped tooling — executors simply shouldn't be able to list or call send-side Slack tools, and a reply command should live only where the orchestrator owns it. On the audit side, session logs recording which tool emitted which message make this traceable after the fact; pairing that with a separate low-privilege app token for agent actions, distinct from your personal token, means a misbehaving executor can only post as a bot identity, not as you.
That's the setup I thought I had: studio posts through its own low-privilege app, and Codex runs start through a wrapper that turns off my skills, plugins and MCP servers. ChatGPT's connectors are a separate switch (
apps), and I missed it.features.apps=falsein the wrapper closed it.This is a really useful write-up. The procedure said "reply" without saying who, and Codex's "discover tools proactively" prompt did the rest — that's a nasty combination. What really got me is that the same sessions also had Gmail, Drive, and GitHub in the tool list. features.apps=false is a clean kill switch; I might also give only the orchestrator an explicit send allow-list, and treat Slack's "Sent using" footer as a reason not to file a message as from: human.
The orchestrator was already limited that way: it answers with studio's
slack reply, which posts through studio's bot. On the footer, ChatGPT's connector posts as the person who connected it, so those messages carried my user ID, and studio identifies me by that ID. The footer isn't on everything the connector sends either: file-upload comments go out without it (openai/codex#36751). So I took the connector away from Codex runs, which removed Gmail, Drive, Calendar, Contacts and GitHub along with Slack.This made me run your first check on my own machine before I even got to the comments.
codex features listshowsapps stable truehere too, and I run Codex headless withcodex exec -s workspace-writefor all my code tasks. The grep for"server":"codex_apps"came back empty on my uncompressed sessions, so nothing has gone out in my name, but clearly the sandbox flag isn't the thing keeping it that way.My agents report progress through a small CLI with their own bearer token against my own API, so their writes show up as theirs. I had only thought about that channel. I never thought about whatever the account itself is connected to.
After setting features.apps=false, did any agent get noticeably worse at a task because it lost a tool it had been quietly relying on?
No. The agents get their work from the board and the repository, and studio posts to Slack and opens the pull request that merges each goal's branch. The connectors were a gap in the isolation, outside anything the setup used. In the logs, the only connector calls were the posts and the November 2 read.
If you have older
.jsonl.zstrollouts, they're worth a pass withzstdcatas well. Most of mine from this month were compressed.Checked: on my machine there are no .jsonl.zst rollouts at all, everything is plain .jsonl, and the grep is still empty. So "nothing lost by turning it off" makes it an easy decision: the connectors were never part of the agents' actual work, just an open door next to it. Did you end up adding a check that fails loudly if a connector ever gets re-enabled after an update?
Glad your check came back empty. I haven't added anything strict. The wrapper passes
features.apps=falseon every run, so a Codex update that changes the default won't turn it back on, and under the studio's rules only I change the wrapper.Ahh, pinning it in the wrapper is so much cleaner than a check 🙌 A default can't flip under you if every run says it out loud. My launcher already builds the flags per run, so that's literally one line for me, adding it today 😄 How many agents does the studio usually run at the same time? I'm still at "one per repo or chaos" here 😅
studio has ten roles, and the team is handed a large goal and works out what to do itself, so the agents running at any moment are whichever the work calls for: sometimes one, sometimes two executors side by side, as many as six so far. Nothing caps that. The only thing studio keeps apart is work that needs a resource only one agent can use at a time.
No cap at all is bold, and it sounds like it works because the roles stay separate until they need a shared resource. I'm still nervous about two agents touching the same repo, so I serialize almost everything. What was the first resource that forced you to serialize: git state, a database, or something like a browser session? And did you learn it from a collision, or from just being cautious?
Only resources that can't be shared, the kind only one agent can use at a time. That rule was there from the start.
Git isn't one of them. studio leaves how to reach a sprint goal to the team, so its tasks don't conflict, and each goal goes out as one PR.
As a side note, I also run a software factory, where each task gets its own worktree. Tasks don't touch each other's code while they work, and conflicts are resolved before a PR goes up. Tasks with no dependency between them run in parallel, and the rest wait for what they depend on. Anything that can't be shared is serialized there too, such as a local server that only runs on a fixed port.
The forensic detail that matters most here is one you only mention sideways: those messages carried the "Sent using" footer, which means they went out through a bot/app token. In Slack's API a message posts as the app if it comes from a bot token and carries
bot_idin the response; it shows up under your name when the token is a user token withchat:write, or when someone setsusername/icon_urlon the call (that needschat:write.customize). So you can grep the channel history for messages wherebot_idis null anduseris you — that gives you the full blast radius in a minute instead of reading session logs, and it also tells you which token scope to revoke.Structurally I'd gate it below the prompt: give executors a token without
chat:writeat all and keep the reply command on the orchestrator, since "only the orchestrator has a reply command" was true in your board code and false in Slack's API surface. The prompt-level rule loses every time an agent is told to be proactive about tools. The other half is the 0.6-second duplicate pair — did you add an idempotency check on the board post, or does a second executor still file the same agreement twice when two run in parallel? Curious whether the orchestrator's dedupe key is content-hash or task-id, because content-hash would have caught both.There are two Slack identities here, and the posts didn't come from the studio's. Studio talks to Slack through its own app with a narrowly scoped bot token, and executors don't hold it. They write items addressed to human on the board, and studio posts them. The posts in the article came through ChatGPT's Slack connector, which I had connected to my ChatGPT account. It posts as me, which is where the "Sent using @chatgpt" footer comes from. Codex runs on the same account, so the connector showed up in the executors' tool list. That's why the fix was turning
appsoff in the Codex launch.For the blast radius I used the Codex session logs. Slack's history reads are tightly rate-limited, and the logs say which session sent each post:
grep -rl '"server":"codex_apps"' ~/.codex/sessions.There was no dedupe key. On November 3 the orchestrator recognized the repeat by reading it, and later ones got through. Now that executors can only reach Slack through studio, there's no second sender to deduplicate.
The five posts under your name are the dramatic part, but the November 2 search is the quieter failure. An executor crawled public and private Slack channels for "the thread where the human agreed," even though that agreement was already on the board. Nothing was sent, so nothing looked wrong. The same "consult those sources before asking" line that sounds helpful is what authorized the read.
codex-isolateturning off skills, plugins, and MCP, but leavingappson, is the warning I will actually use: an isolation wrapper that does not name every feature is not isolation.The November 2 read left nothing in Slack, so I only found it going back through the session logs. It came through the same connector, so turning off
appsclosed it along with the posts.Agreed on naming every feature, though it's hard to keep up. An allowlist would be easier to trust, if Codex had one.
the orchestrator treating anything in the thread as human is the real hole, the executor was just following its prompt. i would tag every message by the path it came in on so a post through an app connector can never read as you
A path tag would let the orchestrator tell those posts apart. What bothered me was what I saw. I don't read Slack footers closely, so a post under my name that I didn't write looks like mine to me as well, and I'd rather not see that at all. So I stopped it before it gets posted.
fair, a tag only helps whatever reads it, not you skimming the thread. stopping it before it posts covers both. does the executor get told why it got blocked or does it just fail
It never gets that far. With
appsoff, the Slack tools aren't in an executor's tool list at all, so there's nothing to try.Inside studio there's no block either. Step 6 now names the orchestrator as the one who replies, and that's only an instruction. The studio waits on me only at sprint reviews and for decisions that bind the whole product, so I'm fine keeping it that loose. If that changed, I'd want something stricter.
The second fix is the one I'd copy: step 6 no longer tells an unnamed someone to reply, and only the orchestrator answers in its thread. We have a similar rule from the other side: in our workflow, production approval has to come directly from the human, and our manager agent followed that even when a relayed message was accurate. After the change, how does the orchestrator tell your own messages apart from anything that reaches studio by another path?
It doesn't check each message. studio files whatever Slack attributes to my user ID as from
human, and the orchestrator takes it as mine.The studio asks for my agreement in only a few places: each goal and its result at the sprint review, and changes to decisions that bind the whole product. The agents carry on with everything else without waiting for me. If my word gets lost or ignored and things go wrong, I fix the setup afterwards, so I haven't needed to verify where each message came from.
What the change did is make sure no agent can post as me: Codex runs start with
features.apps=false, and Claude runs see only the MCP servers the studio configures.Thanks for explaining where you require your own approval and how you restrict agents from posting as you. When a product-wide decision changes mid-sprint, do agents that are already running pick it up before they merge, or only on their next run?
A running agent can pick it up mid-run. Agents read decisions and studio's definitions from the repository as they go, so once a change lands, the next read gets it. Only what an agent was handed at launch stays as it was.
Small changes I just make in place. When I can't tell how a change will affect behavior, I pause studio so no new agent starts, make the change, and restart studio if it touched the scripts. That's as far as I take it, since it's just me running everything locally.
Makes sense: decisions live in the repo, so running agents pick up a change on their next read, and you pause studio when you're unsure how a change will affect behavior. Thanks for walking me through your setup.
One important takeaway here is that tool discovery should never be confused with task-level authorization. An agent may need to know that a Slack connector exists, but that doesn't mean every agent in the workflow should be able to send messages through it. I’d treat outbound actions as explicit capabilities assigned to specific roles, rather than relying only on which tools happen to be available in a session.
Studio assigns it that way already. Sending to Slack belongs to studio's script, and no agent role includes it. The connector sat outside that assignment, so I removed it from Codex runs.
The detail that struck me: the procedure said "reply in its thread" without saying who. Agents fill gaps like that with whatever capability they can reach. The fix that stuck for me is to make the outward-facing action a capability the executor never holds, not a convention: executors return a proposed message, and only the orchestrator's tool layer can send, after a policy check that binds the approval to the exact text and channel. Then a vague instruction ends up as a refused tool call in the log, not a message under your name.
That was the shape studio already had: executors write to the board, and only studio's script posts to Slack. The gap was the connector, which sat outside that path, together with a step that said "reply" without saying who. Both are closed now.
That's a clean fix: one path to Slack, and every step says who it speaks as. The connector sitting outside the board is such an easy gap to miss. Thanks for writing it up.