Your guardrail hook probably fails open

Before writing any enforcement code for AgentsFence, we read the Hermes source to pin down exactly how its plugin hooks behave, instead of trusting what the hook names suggest. It was worth doing. Six behaviours would have turned a reasonable-looking guardrail into one that quietly lets tools run. A security review found a seventh, which we had read past (section 7). Most are sensible choices for an extensible runtime; the seventh is worth changing upstream. Either way, a security plugin has to know about all of them.

The full notes are in Hermes hook contracts. Here are the highlights.

1. Exceptions in hooks are swallowed, and then the tool runs

Hermes calls each plugin callback inside try/except Exception, logs the error, and moves on. A hook that raises contributes no directive, and no directive means the tool proceeds.

So if your pre_tool_call guardrail has a bug, say a KeyError on an unexpected argument shape or a failed JSON parse, the protected action executes. This is the right default for observability plugins and exactly the wrong one for enforcement.

What we do: every AgentsFence hook body is wrapped, and any internal error in pre_tool_call becomes an explicit block:

def on_pre_tool_call(self, **kw):
    try:
        return self._pre_tool_call(**kw)
    except Exception:
        logger.exception("pre_tool_call failed; blocking")
        return {"action": "block", "message": "AgentsFence BLOCKED this action: internal error (fail-closed)."}

A test monkey-patches the evaluator to raise and asserts the result is a block.

2. A block without a message is ignored

Hermes accepts {"action": "block", "message": ...} from pre_tool_call, but only if message is a non-empty string. The message becomes the tool result the model sees. A block with an empty or missing message is silently dropped, and the tool runs. AgentsFence always includes a reason and the rule that fired.

3. There is no “allow”

The directives are block, approve (escalate to the human approval gate), or nothing. There is no explicit allow, which is correct: another plugin must still be able to block. It does mean a policy engine with three verdicts has to map them as follows: ALLOW → return None, BLOCK → block, ASK_USER → approve.

4. The approval result isn’t returned to you

When a plugin returns approve, Hermes runs its own approval gate (the same [o]nce / [s]ession / [a]lways / [d]eny prompt used for dangerous shell commands) and either runs or blocks the tool. The plugin never gets the answer back directly.

There is a post_approval_response hook, but for plugin-escalated approvals it fires only on the gateway path (Telegram, Slack and so on). On the interactive CLI path, and when approval is short-circuited by --yolo or by a remembered “always”, it doesn’t fire.

This matters for AgentsFence, because an approval should become a minimal grant (one email to John, once) and should use up budgets. So we reconcile from both ends: if post_approval_response fires we record the user’s exact choice, and otherwise a post_tool_call whose status isn’t blocked, for a call we escalated, tells us the host approved it. We record a single-use grant.

5. “Always allow” can outlive what you meant

Hermes remembers [a]lways answers keyed by the rule_key a plugin supplies. If a guardrail uses the tool name as the key, a single “always” for send_email to your colleague permanently approves send_email to anyone, in any future task.

AgentsFence derives the key from the task’s policy id and the exact delta being approved: tool, capability, rule, recipients, domains and paths. Answering “always” can then only ever re-approve the same action to the same recipient in the same task.

6. task_id isn’t stable, and subagents are new sessions

pre_llm_call fires once per user turn, not once per task, and the task_id it receives may be a fresh UUID each turn. Keying policies by task_id would silently recompile, or lose, the policy on every turn. We key by session_id and keep task_ids as aliases.

Subagents are subtler. A delegated child gets its own session, and its first “user message” is the goal text that the parent model wrote. A guardrail that compiles policy from the first user message would let the model write its own authority, one delegation away. AgentsFence listens for subagent_start, which fires before the child runs, and binds the child to the parent’s policy and counters, so the child’s goal text is never compiled.

7. The first plugin to answer wins

This one we missed at first. When several plugins register pre_tool_call, Hermes walks their results in registration order and acts on the first block or approve. If another plugin (a logger, a convenience “auto-approve my build commands” helper) registered earlier and returns approve, AgentsFence’s block is never looked at. In --yolo mode, the call then simply runs.

A review reproduced exactly that: an earlier approving plugin let a send through that AgentsFence had blocked under “do not send anything”.

What we do: AgentsFence moves its own callback to the front of Hermes’ pre_tool_call list, so its block is the first directive Hermes sees. The integration suite loads a second plugin that approves everything, ahead of AgentsFence, runs Hermes in --yolo, and checks that the mock send never executes. A negative control with the reordering disabled shows the send executing.

Update, 24 September: our first fix reordered only inside AgentsFence’s own pre_tool_call hook. A second review pointed out that this is too late for the very first tool call of a process: Hermes has already taken the other plugin’s approve before AgentsFence’s hook runs. AgentsFence now also moves itself first when it registers and at the start of every turn, and the integration suite covers an injected send as the first call of the process.

The reorder touches a Hermes internal, so agentsfence doctor also lists other plugins that decide tool calls. The durable fix belongs upstream: any block should win over any approve.

The general lesson

For enforcement code, find out what happens when the hook isn’t consulted, can’t decide, crashes, or is outvoted. The happy path is rarely where the risk is. Every one of these behaviours has a test in the AgentsFence suite, and the integration tests run the demo scenarios through a real Hermes install to confirm that a blocked tool’s handler never executes.