Platform

Troubleshooting

Symptom to cause to fix. Every status code, every non-obvious behaviour, and the diagnostic order that finds the answer fastest.

Start here: what the response told you#

Almost every question has its answer in the status code and the status field.

And the general diagnostic order, which is faster than guessing:

1. Read the status field — it names the gate.
2. Read the error string — it names the rule or the reason.
3. Open Transfer Logs, filtered to that status, and read decisionReason.
4. If the decision looks wrong, read Audit Trail — configuration changes are in the same chain, so "the policy was different at the time" is checkable rather than arguable.

401 Invalid API key#

Check in this order.

The header format. It must be exactly:

Authorization: Bearer ark_live_xxxxxxxx

Not ApiKey, not the raw value with no scheme, not Bearer: . A missing Bearer prefix is the single most common cause.

Whitespace. A key read from a file or copy-pasted through a terminal often carries a trailing newline. os.environ["ARCUS_KEY"].strip() costs nothing.

Was it revoked? Check API Keys — a revoked key keeps its row with revokedAt set, so it is visible rather than absent. Revocation is immediate; there is no grace period.

Is it from this instance? A key minted against local development does not work against production. Compare the prefix shown in the dashboard with the first 15 characters of the key you are sending.

Are you calling the right surface? An ark_live_ key returns 401 from every /v1/admin/* endpoint. That is not a bug — see the two credentials.

401 Agent identity mismatch#

The key is bound to an agentId and your from field is a different string. Recorded as dispatch.spoof_rejected.

{ "error": "Agent identity mismatch" }

Cause, nearly always: a typo or a drift — intake_agent versus intake-agent, or a key bound during a rename. Open API Keys and compare the Agent ID column against the from you are sending. They must match exactly: case-sensitive, no trimming.

Fix: send the bound value, or issue a key bound to the identity you actually want. Do not "fix" it by unbinding the key — the binding is what makes policies, guardrails and rate limits mean anything.

If from is omitted entirely and the key is bound, the binding is used and this error does not occur. If the key is unbound and from is omitted, only a wildcard policy can match — which usually surfaces as an unexpected 403.

403 BLOCKED_BY_POLICY#

{
  "error": "No policy allows intake-agent -> https://api.example.com/webhook",
  "status": "BLOCKED_BY_POLICY",
  "policyId": null
}

Read policyId first — it splits the diagnosis in two.

policyId: null means nothing matched. This is default-deny, working. You have no ALLOW policy covering that sender and that target.

• Brand-new account with no policies? Everything is refused until you create the first one. Expected.
• Have a policy? Check the target string. https://api.example.com/webhook is not matched by https://api.example.com/webhooks/*, and a trailing slash counts. * is the only wildcard.
• Check the sender. A policy for intake-agent does not cover intake-agent-v2.

A populated policyId means a policy matched and its action was BLOCK. Look it up in Policies. If you expected a different, more permissive policy to win, this is specificity: the more specific rule wins regardless of creation order, and a long literal BLOCK beats a broad ALLOW. Make your ALLOW more specific, or delete the BLOCK.

Fastest test:

curl -sS "$ARCUS_URL/v1/admin/policies" \
  -H "Authorization: Bearer <clerk-session-token>" | python3 -m json.tool

Then compare, character by character, against the target you sent.

422 BLOCKED_BY_DLP#

{
  "status": "BLOCKED_BY_DLP",
  "violations": ["AWS_ACCESS_KEY"],
  "matchCount": 1,
  "detailAvailableOn": "pro"
}

violations names the category. That is usually enough to guess the field. On Pro+, the dlp.configure capability adds matches with field paths — the matched value is never returned on any plan.

Common causes by category:

The most common shape by far: an entire configuration or environment object serialised into data. DLP walks recursively to depth 12, so a credential nested four levels deep is found.

Fixes, in order of preference:

1. Send less. Include only the fields the receiver needs. This is right more often than any exemption.
2. Rename the field if it is a false positive — a field called token holding a non-sensitive correlation id trips a field-name detector. Call it correlationId.
3. Exempt the narrowest possible route by setting dlpEnabled: false on a policy whose targetEndpoint is one specific path. Never a whole host.

Full detail: Data loss prevention.

422 BLOCKED_BY_PARAMETER#

{
  "status": "BLOCKED_BY_PARAMETER",
  "severity": "critical",
  "findings": [ { "label": "No destructive commands", "path": "data.command",
                  "condition": "matches_destructive", "observed": "DROP TABLE patients" } ]
}

findings lists every matching rule, not only the deciding one, and each carries label, path, condition, expected and observed. The diagnosis is usually complete in the response.

If the block is correct: the agent tried something it should not. The finding tells you which field and what value.

If the block is wrong:

Path debugging. data.amount matches only at that exact position. data.items[*].price matches every element. A bare name — amount — matches at any depth, which is the forgiving option when you are unsure of the payload shape.

Cannot create a rule?

{ "code": "PARAMETER_RULE_LIMIT_REACHED", "tier": "free", "limit": 2, "activeRules": 2 }

Only enabled rules count. Set enabled: false on one you are not using — the definition is kept. On Free that is the intended way to maintain a library and run two.

429 THREAT_DETECTED#

{
  "error": "Rate spike: 61 dispatches in 60s (limit 60) — quarantined for 300s",
  "status": "THREAT_DETECTED",
  "retryAfter": 300
}

60 dispatches in 60 seconds per agent, per tenant. Quarantine is 300 seconds. Identical on every plan — it is a safety mechanism, not a billing one.

Honour Retry-After. retryAfter is the live remaining TTL, so it counts down. Retrying sooner produces another refusal and another audit event, and hammering it is indistinguishable from the behaviour that triggered it.

Then find the cause. A quarantine is almost always one of:

Fixes:

• Batch inside one dispatch. One dispatch carrying 500 records costs 1 against the limit; 500 dispatches cost 500. Guardrail paths like data.records[*].id exist for exactly this shape.
• Split by identity. import-worker-1, -2, -3 each get their own 60-per-minute budget, their own baselines and their own audit trail. Do not spread across multiple keys bound to the same agent — counters key on the bound agent, not the key, so that changes nothing.
• Back off properly. There is a ready-made dispatch_with_backoff in Threat detection.

Releasing early: Anomaly Detection → the agent → release. Or /v1/admin/threats/:agent/release. It appends threat.released with your name on it — the release is the audited decision, because tripping a limit is an accident and deciding to let the agent continue is a judgement.

Was it an operator? A platform operator can impose a quarantine of 60 s to 24 h. It writes the same Redis keys, so it appears identically — but it also appends abuse.throttled to your audit chain with a reason and an actor. If a quarantine appeared with no matching traffic burst, check Audit Trail.

400 validation errors#

The message names the field and the problem.

Not retryable without changing the request. Content-Type: application/json must be set, and the body must be valid JSON — a trailing comma produces a parse error, not a field-level message.

202 but nothing arrived#

Three different outcomes return 202. Branch on queued, not on the status code.

queued: false with status: "PENDING_APPROVAL"#

Held for human review. Nothing is delivered until someone decides, and holds do not expire.

Open the approval queue on the Policies page, or /v1/admin/messages/pending. If nothing is there, it was already decided — check Transfer Logs for REJECTED.

Cause: a REQUIRE_APPROVAL policy matched the route, or a hold_for_approval guardrail matched a field. The decisionReason on the log row says which.

queued: false with error: "Queue unavailable"#

{ "ok": true, "queued": false, "error": "Queue unavailable" }

Redis was unreachable when Arcus tried to enqueue — the 2-second enqueue timeout expired. The decision was made and recorded; the transfer log row is written FAILED with Queue unavailable: … rather than the request being silently dropped.

Nothing was delivered and nothing will retry automatically. Fix Redis, then resend. Check /v1/admin/platform/health if you operate the instance.

queued: true and still nothing arrived#

Delivery was attempted. Open Transfer Logs and read the row's status:

PENDING that never moves is the informative one: the gateway accepted and recorded the dispatch, so the problem is downstream. Check that the delivery worker is alive and that Redis is reachable.

Delivery behaviour, for reference:

500 errors#

A genuine server error. Retryable with backoff, unlike everything else on this page.

If they persist and you operate the instance, check /v1/admin/platform/health — PostgreSQL unreachable is the usual cause, and every gate is on that path. If you are on hosted Arcus, note the time and the transferLogId if you have one, and contact the operator.

Control-plane problems#

Every admin endpoint returns 401#

You are sending an ark_live_ key. The control plane requires a Clerk session token — see Authentication & keys. This is a deliberate boundary, not a missing feature: an agent must not be able to rewrite the rules that constrain it.

403 on an admin endpoint#

Two possibilities, and the body distinguishes them.

A tier limit, with a code:

{ "code": "PARAMETER_RULE_LIMIT_REACHED", "tier": "free", "limit": 2,
  "activeRules": 2, "upgradeAvailable": true, "requiredTier": "pro" }

A missing capability — verification, reports, baselines, DLP paths, policy authoring. Check Account Settings for your capability set. See Pricing & plans.

404 with code: "NO_ACCOUNT"#

No tenant record resolves for the session. Sign in to the dashboard once — the account row is created on first authenticated access.

The verify button is disabled#

audit.verify is a Pro+ capability. On Free the button explains what would enable it rather than failing silently.

Audit history stops 7 days back#

Free shows the most recent 100 events within the last 7 days. Every event is written regardless of tier — Free limits reading, not recording. Upgrading reveals history that was recorded all along.

My tier looks wrong after paying#

A webhook was probably missed. Re-read your state from the provider:

curl -sS -X POST "$ARCUS_URL/v1/admin/subscription/sync" \
  -H "Authorization: Bearer <clerk-session-token>"

Entitlement holds while your status is active, trialing or past_due — a failed payment does not strip your configuration on the same day.

Chain verification failed#

{ "valid": false, "brokenAtSeq": 872, "reason": "hash mismatch: recomputed hash does not match stored hash" }

Treat it as an incident. Do not delete, re-seed or "repair" anything — you would destroy the evidence of what happened. The procedure is in Audit chain: note the brokenAtSeq, export what you have, compare against your last good archive, and investigate who had database write access.

Self-hosting problems#

The API will not start#

Missing required environment variable: DATABASE_URL — the only variable whose absence prevents boot. In development, apps/api/.env is loaded regardless of the working directory; in a deployment, set it in the platform environment.

[api] Redis unavailable … running without a delivery worker#

The API boots and serves; dispatches are evaluated and recorded but not delivered. This is deliberate — an evaluated, recorded, undelivered request beats a gateway that refuses to start. Fix REDIS_URL and restart to pick the worker up.

While Redis is down: rate limiting fails open (the other four gates still run) and every dispatch records FAILED with Queue unavailable.

The dashboard cannot reach the API#

CORS. DASHBOARD_ORIGIN defaults to http://localhost:3001 and must be your real dashboard origin in production. Only that one origin is permitted — deliberately never *. Also confirm NEXT_PUBLIC_API_URL points at the API the browser can actually resolve.

The control plane 404s for me#

SUPER_ADMIN_EMAILS must contain your address, on the API and on the dashboard — the two are checked independently, because the dashboard decides whether to render it and the API decides whether to answer it. The alternative and primary mechanism is Clerk publicMetadata.role, settable only with the Clerk secret key. Either way it is configuration, not a database role: granting access needs a deploy or a Clerk-side change, which is why an application bug cannot grant it.

Checkout returns an error#

The billing provider is not configured. Billing is optional by design: with no credentials the instance boots normally, every account is treated as free tier, and checkout says so rather than inventing a link. For an internal deployment, use /v1/admin/tenants/:id/tier to provision your own account.

Still stuck?#

Collect these before asking anyone — they identify anything in the system without exposing a payload or a credential:

• The transferLogId from the response
• The HTTP status and the status field
• The error string, verbatim
• The key prefix — the ark_live_ first 15 characters, never the whole key
• Your tier and capability set from Account Settings
• The relevant audit event seq if a configuration change is involved