Troubleshooting
Something broke at runtime and you want the fix, not a tour. There are two fast paths:
- Your problem belongs to one feature — a subscription that won’t deliver, a stuck action run, a token-scope question. Jump straight to its runbook in the table below.
- It’s a cross-cutting runtime error — a write conflict, shape mismatch, auth failure, empty query result, or a rate/payload limit. Those don’t belong on any single feature page, so they’re covered here, each as symptom → cause → fix.
To look up what a specific error code means, see the error-code table and the SDK ErrorKind reference.
Feature-specific issues
Section titled “Feature-specific issues”Troubleshooting that belongs to one feature lives on that feature’s page:
| Symptom | Go to |
|---|---|
| A subscription stopped delivering, is retrying, or dead-lettered | Debugging a Failing Subscription |
An action webhook failed, stalled, or hit dead_letter | Delivery Lifecycle |
You need to know what an error code means | Error codes · SDK ErrorKind |
| Request-rate limits and per-tier quotas | Rate Limiting |
| Installing or repairing a component (e.g. Veritas) | Installing Veritas · Authoring Components |
| Token scopes, expiry, and access | Personal Access Tokens · Access Reference |
Cross-cutting runtime errors
Section titled “Cross-cutting runtime errors”The fixes below use the CLI for brevity; the SDK, HTTP API, and MCP expose the same operations where each surface supports them (token management, for example, is CLI/SDK only). WarmHub returns errors in a consistent envelope, so the fastest triage is to read the code:
{ "error": { "code": "CONFLICT", "message": "Expected version 3 but current version is 5" }}Write conflicts
Section titled “Write conflicts”Symptom — a write is rejected with a 409 CONFLICT:
{ "error": { "code": "CONFLICT", "message": "Expected version 3 but current version is 5", "details": { "reason": "expected_version_mismatch" } }}Or, when adding a name that already exists:
CONFLICT: Thing "cave" already existsCause — WarmHub uses optimistic concurrency and does not lock things on read. A CONFLICT with details.reason: "expected_version_mismatch" means you passed expectedVersion and another writer advanced the thing’s version before your write landed. A bare ... already exists means you tried to add a name that is already taken.
Fix
- Re-read the current version of the thing:
Terminal window wh thing view cave - Decide whether your change still applies against the new version. Remember that a revise is a full replacement — include every field in
data, not just the ones you changed. - Resubmit with
expectedVersionset to the version you just read:The sameTerminal window wh thing revise cave --shape Location --data '{"x":3,"y":7}' --expected-version 5--expected-versionflag is available onwh commit submit --reviseand on a single--retract. - To create or skip instead of failing on an existing name, set
skipExistingon theadd— it returnsstatus: "noop"rather than aCONFLICT.
Tip — when you need exclusive access across a read-modify-write rather than a single optimistic check, take a read lease instead of retrying conflicts.
Shape mismatches
Section titled “Shape mismatches”Symptom — a write is rejected with a 400:
{ "error": { "code": "SHAPE_MISMATCH", "message": "Field 'status' expected string, got number" }}Related 400 codes on the write path include VALIDATION_ERROR (the operation itself is malformed) and ILLEGAL_OP_SEQUENCE (operations submitted in an order the pipeline can’t apply).
Cause — the data you sent doesn’t match the thing’s shape: a field has the wrong type, a required field is missing, or you’re writing against a graph whose shape you haven’t inspected.
Fix
- Inspect the current shape and compare it field-by-field against your payload:
Terminal window wh shape view Sensor - Correct the payload — fix the mismatched type, add the missing field — and resubmit.
- If the schema change is intentional and you own the shape, revise it in place. A revise increments the shape’s version; things already written under the old schema are unaffected.
Terminal window wh shape revise Sensor --fields '{"status":"string","reading":"number"}'
Note — shapes are not created implicitly on first write; you
adda shape explicitly before writing things against it. See Shapes for field types and constraints.
A reference field was stored as null
Section titled “A reference field was stored as null”Symptom — you wrote a wref into a field, the operation succeeded, but the stored value is null and the result carried a coalescedWrefs warning:
{ "status": "applied", "warnings": { "coalescedWrefs": [ { "fieldPath": "owner", "wref": "nobody", "reason": "thing_absent" } ] }}Cause — the field is an optional wref (wref?) and resolution returned thing_absent: the graph and active shape existed, but the named shaped thing did not. WarmHub coalesces only that outcome to null and reports it as a warning; missing shapes and other failures do not coalesce. The original wref string is discarded — coalescing is lossy. A required wref whose target is missing instead fails that operation with NOT_FOUND and stores no version for it.
Fix
- If the reference should have resolved, create the target first, then revise the field. To reference something created in the same commit, order the target’s
addbefore the reference, or create both entities with explicit deterministic names and reference those names directly. A forward reference to a target added later in the same commit fails withVALIDATION_ERRORrather than coalescing. - If
nullis acceptable, no action is needed. Inspect thecoalescedWrefsentries to see exactly whichfieldPaths coalesced. - If a missing target should be a hard failure, make the field a required wref in its shape — required wrefs never coalesce. See Optional Wref Fields.
Auth failures
Section titled “Auth failures”Symptom — a request returns 401 UNAUTHENTICATED or 403 FORBIDDEN:
{ "error": { "code": "UNAUTHENTICATED", "message": "Token expired" } }{ "error": { "code": "FORBIDDEN", "message": "Missing scope: repo:write" } }Cause — the two codes mean different things:
| Code | Meaning |
|---|---|
401 UNAUTHENTICATED | No token, or the token is invalid, expired, revoked, or belongs to a disabled account. |
403 FORBIDDEN | The token is valid but lacks the scope or graph role the operation requires. |
Fix
For a 401, the right remediation depends on which condition triggered it:
- No token, or the token is invalid, expired, or revoked — if you are not exporting
WH_TOKENin your environment, sign in interactively to refresh the stored credential:Terminal window wh auth login WH_TOKENis set in your environment —wh auth loginupdates only the stored profile and does not replace an exportedWH_TOKEN. The CLI always treats an exportedWH_TOKENas the active credential, sowh auth loginalone will not resolve the failure. Verify the value is a valid, unexpired token — a personal access token or a JWT — and replace it if not:Terminal window export WH_TOKEN=<your-valid-token>- Disabled account — if the token itself is valid but the account behind it has been disabled, ask an administrator to re-enable the account. For a service account, ask an owner or admin of its organization. A new token will not help until the account is enabled.
If you authenticate with a personal access token, wh token list --all shows your active, expired, and revoked PATs, so you can confirm whether the one in use is still valid. It does not cover interactive sign-in sessions or a JWT supplied through WH_TOKEN:
wh token list --allFor a 403, the token authenticated but isn’t authorized. Scopes and graph membership are composed with AND — a token needs both the right scope and a graph role that permits the operation. A repo:read token can’t write even where your role would allow it, and a repo:write token can’t write to a graph where your role is only viewer. (The repo:* scopes keep the spelling from when a graph was called a repo.)
- Create (or recreate) a token with the scope the operation needs:
Terminal window wh token create --name ci-bot --scope myorg/mygraph=repo:read,repo:write - If you’re hitting a graph you don’t own, confirm the owner has granted you a role that permits the operation. See the access reference for the minimum scope per task.
Note — MCP, the SDK, and the HTTP API all authenticate with the same token, so an auth failure is a credential problem, not a transport one: fix the token or its scope rather than switching surfaces. The exact code can vary by surface — some read endpoints return an opaque
404instead of401/403.
Empty or unexpected query results
Section titled “Empty or unexpected query results”Symptom — a query returns fewer results than you expected, or none at all.
Cause — usually a filter that’s narrower than you think, a single page of a paginated result, or the wrong graph scope. There is one genuine timing case: a filtered read — one with a match glob — may lag briefly right after a write while WarmHub updates its read indexes. An unfiltered read reflects a write immediately, so if a thing you just wrote is missing from an unfiltered list, the cause is a filter, pagination, or scope — not propagation.
Fix
- List the graph without filters to confirm the things exist at all:
Terminal window wh thing list --graph myorg/mygraph - Add filters back one at a time to find the one that excludes your results:
Terminal window wh thing query --shape Observation --about cave --graph myorg/mygraph - Check for pagination. A query returns one page at a time; if the response carries a
nextCursor, follow it for the next page. The page size defaults to 50 and caps at 500. Anonymous callers on public graphs are capped at 25 per page and stop after two pages — following the cursor past the second page returns404, so authenticate to page through larger result sets. See Anonymous Pagination Caps. - Confirm the graph. Querying one graph won’t surface things that live in another, and a cross-graph wref lookup needs
repo:readon the target graph. - For a thing you just wrote that’s missing from a filtered read, retry the read after a moment, or drop the filter — the read index catches up shortly. See Filtered Read Freshness.
Tip — authenticated full-text and hybrid search pages can be sparse: a page may hold fewer items than
limit, or zero, whilenextCursoris still present. Paginate untilnextCursoris gone before concluding a result set is empty.
Rate and payload limits
Section titled “Rate and payload limits”Symptom — a request returns 429 RATE_LIMITED or 413 PAYLOAD_TOO_LARGE:
{ "error": { "code": "RATE_LIMITED", "message": "Rate limit exceeded", "retryAfter": 4 }}Cause — you exceeded a request-rate limit (a per-IP cap on unauthenticated traffic, or a per-principal write limit), sent a request body larger than the endpoint accepts, or submitted a write that would store more than 10,000 references in a single thing version (including collection members).
Fix
- For
429, back off until theRetry-Afterresponse header (mirrored aserror.retryAfter, in seconds) has elapsed, then retry. Authenticate to escape the per-IP anonymous cap. See Rate Limiting for the per-tier write limits. - For
413, there are two distinct causes:- Oversized request body — split the work into smaller writes, each independently valid. The SDK’s write helpers stay under the size limit automatically, so you generally only hit this on oversized requests built by hand.
- Per-version reference cap — if the request body itself is small but the write would store more than 10,000 references in one thing version (for example, a collection with a large number of members), split the collection or reduce the number of references per version.
Still stuck?
Section titled “Still stuck?”If none of the above resolves it:
- Re-run the failing command with
--debugto print the full stack trace on failure:Terminal window wh --debug thing view cave - Check status.warmhub.ai for an active incident before reporting an outage.
- Contact support with the error
codeand message, the--debugoutput (redact any tokens), the wrefs or graph involved, and roughly when the failure started.