Arcus Docs

Enforcement

Threat detection

Gate 1 counts what each agent is doing and quarantines the ones that spike. A Redis counter, a 60-second window, a 300-second quarantine, and a release that is an operator decision rather than a timer.

What it stops#

An agent in a retry loop is the most common self-inflicted outage in agent systems. A tool call fails, the agent retries, the retry fails the same way, and within a minute it has sent ten thousand requests to a downstream service that has no idea it is being attacked by something on your own side.

Gate 1 is a per-agent rate limit that turns that from an incident into a 429.

The mechanism#

SettingValue
Window60 seconds
Limit60 dispatches per window
Quarantine on breach300 seconds (5 minutes)

A counter in Redis, keyed on (tenant, agent). On the 61st dispatch inside a window, the agent is quarantined:

json
{
  "error": "Rate spike: 61 dispatches in 60s (limit 60) — quarantined for 300s",
  "status": "THREAT_DETECTED",
  "retryAfter": 300
}

429, with a Retry-After: 300 header, recorded as dispatch.blocked_by_threat. Every subsequent dispatch from that agent gets the same response, with retryAfter counting down to the live TTL, until the quarantine expires.

One dispatch per second, sustained, is the effective ceiling — comfortably above what a working agent does and comfortably below what a runaway loop does.

Why the quarantine, and not just the window#

A plain sliding window lets a looping agent send 60 requests every minute forever. The quarantine converts a breach into a real pause: five minutes of silence is long enough for a backoff to matter and long enough that a human notices the log entry.

Isolation#

The counter key includes the tenant. intake-agent in your account and intake-agent in someone else's are separate counters that cannot affect each other — see Core concepts.

It counts the bound agent#

Where the key has an agentId, the counter uses that rather than the from on the request. Otherwise a quarantined agent could escape its own quarantine by changing one string in its payload. This is why identity binding is strongly recommended — without it, the rate limit is keyed on a value the caller chooses.

Where it sits in the pipeline#

Gate 1 — first, before identity binding, policy, DLP and guardrails.

The reasoning is cost. A flooding agent should cost one Redis read, not a policy evaluation followed by a full payload walk. Putting this check last would mean an abusive agent consumes the most expensive path on every request it makes, which is exactly what an attacker would want and exactly what a runaway loop accidentally achieves.

It fails open#

If Redis is unreachable, the rate-limit check is skipped and the request proceeds to gate 2.

This is a deliberate, stated tradeoff:

  • Rate limiting is a throttle, not an authorization boundary. Its job is protecting a downstream service from volume, not deciding whether a request is permitted.
  • The checks that are authorization boundaries — identity, policy, DLP, guardrails — live in PostgreSQL and continue to run. A Redis outage does not open a route, expose a credential or bypass a guardrail.
  • The alternative is a Redis outage taking your entire gateway down. For a component whose purpose is availability protection, that is the wrong failure mode.

The skip is logged loudly. Note the asymmetry: checkAgent fails open, quarantineAgent does not. Imposing a quarantine is a decision that must persist, so if Redis cannot record it the operation fails rather than silently succeeding.

This is the one fail-open in the pipeline

Everything else fails closed. PostgreSQL unavailable means requests fail — Arcus will not forward a request it cannot record. The full asymmetry is set out in the Security model.

Seeing your quarantines#

bash
curl -sS "$ARCUS_URL/v1/admin/threats" \
  -H "Authorization: Bearer <clerk-session-token>"

Returns every currently-quarantined agent in your account with its live remaining TTL, read from Redis rather than computed from a stored timestamp — so what you see is what the enforcement path will do on the next request.

bash
curl -sS "$ARCUS_URL/v1/admin/threats/intake-agent" \
  -H "Authorization: Bearer <clerk-session-token>"

Per-agent detail: quarantine state, remaining seconds, and the recent enforcement history that led to it.

In the dashboard this is the Anomaly Detection view.

Releasing early#

bash
curl -sS -X POST "$ARCUS_URL/v1/admin/threats/intake-agent/release" \
  -H "Authorization: Bearer <clerk-session-token>"

Clears the quarantine immediately. The agent's next dispatch proceeds normally.

Release appends threat.released to your audit chain with the actor. This is the part worth noticing: the release is the audited event, not the quarantine. Anyone can trip a rate limit by accident; deciding that an agent which tripped one should be allowed to continue is a human judgement, and it is sealed.

Fix the cause before you release

Releasing an agent that is still in a retry loop restarts the flood, and the next quarantine is five minutes later. Stop or patch the agent, then release. The Retry-After value is not a suggestion to poll against — it is the earliest point at which a fixed agent can resume.

Operator-imposed quarantine#

Beyond the automatic limit, a platform operator can quarantine an agent deliberately from the abuse review queue — for a duration between 60 seconds and 24 hours.

The important implementation detail: an operator throttle writes the same Redis keys the automatic path writes. There is exactly one enforcement mechanism, so:

  • /v1/dispatch refuses a throttled agent through the same gate, with the same 429;
  • /v1/admin/threats shows an operator throttle exactly as it shows an automatic one;
  • release works identically for both;
  • the two mechanisms cannot disagree about who is blocked.

An operator throttle appends abuse.throttled to the affected tenant's audit chain, and a release appends threat.released. An action that changes what your agent is allowed to do appears in your own sealed record, whoever took it — you are never in a position where your traffic was blocked and your chain does not say so.

See Platform endpoints for the operator side.

Handling a 429 in your agent#

python
import time
import requests

def dispatch_with_backoff(session, url, body, max_attempts=3):
    for attempt in range(max_attempts):
        r = session.post(url, json=body, timeout=15)

        if r.status_code == 429:
            # Arcus tells you exactly how long. Respect it — do not poll.
            wait = int(r.headers.get("Retry-After", 300))
            if attempt == max_attempts - 1:
                raise RuntimeError(f"quarantined, {wait}s remaining")
            time.sleep(wait)
            continue

        return r

    raise RuntimeError("exhausted attempts")

Three rules that matter more than the code:

  • Read Retry-After. It is the live TTL. Retrying sooner produces another 429 and another audit event.
  • Never retry a 429 in a tight loop. That is the behaviour the gate exists to stop, and it will keep the quarantine alive rather than shortening it.
  • A 429 is a bug report about your agent. In normal operation an agent should never see one. If it does, something is looping, fanning out unexpectedly, or being called far more often than intended.

Designing around the limit#

Sixty dispatches per minute per agent is generous for one worker and restrictive for a batch job. If you legitimately need more throughput:

Split by agent identity. The counter is per (tenant, agent), so import-worker-1, import-worker-2 and import-worker-3 each get their own 60/minute. This is the intended shape — it also means each worker's baselines, policies and logs are separately attributable, and a misbehaving shard is quarantined without stopping the others.

Batch inside one dispatch. One dispatch carrying an array of 500 records costs one against the limit; 500 dispatches cost 500. Guardrail paths like data.records[*].id are built for exactly this shape.

Do not spread one workload across keys to evade the limit. Multiple keys bound to the same agent id share a counter, which is the correct behaviour. If you need parallelism, use distinct agent identities — that is a design decision with an audit trail, not a workaround.

What is recorded#

EventWhen
dispatch.blocked_by_threatEvery dispatch refused while quarantined
threat.releasedA quarantine cleared early, with the actor
abuse.throttledAn operator imposed a quarantine, with the reason

Every refusal is a sealed audit event, so the history of a flooding incident — when it started, how many dispatches it attempted, when it was released and by whom — reconstructs from the chain rather than from an application log that could have rotated away. See Audit chain.