Troubleshooting
Find the symptom you see and follow its link. Each page it points to explains the cause and the fix; this page only sends you to the right one.
Where the evidence is
Section titled “Where the evidence is”-
Response headers.
gp-cachesays what the cache did, and aPASScarriesgp-cache-reasonsaying why. See headers and cache statuses.Terminal window # GRAPHPILOT_HOST is your service hostname (<subdomain>.<label>.graphpilot.cloud) or your custom domain.curl -sS -D- -o/dev/null "https://$GRAPHPILOT_HOST/graphql" \-H 'content-type: application/json' \-d '{"query":"{ products { id name } }"}' | grep -i '^gp-' -
Insights in the portal, where each stored request shows its cache status, reason, keys and log lines. See Insights.
-
The CLI.
gpilot requestslists recent requests; see the command reference.
Insights and gpilot requests show only requests whose telemetry was stored. Nothing is stored
in lean mode, or for a request your service leaves out with
telemetry.record. For any other
request, the response headers are your evidence.
Nothing is cached
Section titled “Nothing is cached”Every response says gp-cache: PASS. Read its gp-cache-reason and look it up in why is
nothing cached?, which goes from each code to its cause and fix.
The same query never hits
Section titled “The same query never hits”The first response was a MISS, and the same request again is another MISS. The Lookup
field of the second request in Insights says whether no entry was found or
one could not be used. Check, in order:
- The first response was a
PASS, so nothing was stored: see nothing is cached. - The two requests differ. The query text, operation name and variables are all part of the key: see what the entry is keyed by.
- The entry is partitioned per caller, which is correct and looks like a low hit rate: see identifying callers and its vary rules.
- A purge removed it. A mutation purges every entry containing what it returned: see verifying a purge.
- Another edge location answered. Entries warm up per location: see locations.
The hit rate drops to zero, or paid features are paused
Section titled “The hit rate drops to zero, or paid features are paused”Every service of the account goes to its origin at once, and responses carry no gp-cache
header. The billing account runs in lean mode: it used up its
request allowance on Glider (Free), a payment is overdue, or the account was paused. The edge
cache steps aside for every service of the account until lean mode ends, and no request
telemetry is stored meanwhile. Purges keep running. Every organization page of the portal shows a
Paid features are paused notice; see quotas and lean mode and paid features
are paused for what ends it.
Stale data after a write
Section titled “Stale data after a write”A HIT served data your origin has since changed. Find the purge that should have removed it
with verifying a purge, and the keys the entry carries with
surrogate key debugging. A write that did not go
through GraphPilot has to be purged explicitly.
A purge did not clear a response
Section titled “A purge did not clear a response”A response you purged still comes back as a HIT. Check, in order:
- The purge was recorded and named what you expected: see the purge history.
- The entry carries the key you purged. See debugging surrogate keys for the keys a response really has.
- The response you see came from GraphPilot. A purge removes the entry at every edge location
(purging is global), but not a copy held by a
browser or another cache in front of your clients. Confirm at the
edge with the
gp-cacheheader.
Origin rejects requests after going live
Section titled “Origin rejects requests after going live”Your origin answers requests through GraphPilot with 401 or 403. Check the signature
verification against the known test
vector, your
origin’s clock against the replay window, and your
firewall against the current origin allowlist.
The edge refuses a request
Section titled “The edge refuses a request”The response carries a GraphPilot code: for a GraphQL request in extensions.code of a GraphQL
error, for any other request in the title of an application/problem+json body. The request
never reached your origin. Look the code up in the request was
rejected. The usual causes:
| Status | Usual cause | Where the fix is |
|---|---|---|
400 | The operation is deeper, longer or makes more selections than the service allows, uses a field name with the reserved prefix, or could not be read unambiguously. | Hardening, limits |
401 | A credential your auth providers could not accept. | Authentication |
403 | Introspection is off, the operation is not in the service’s persisted query registry, or the client is not declared or below its required version. | Introspection, persisted queries, client profiles |
404 | A GraphQL operation was sent to a path that is not one of the service’s GraphQL paths. | GraphQL on other paths |
405 | A mutation or subscription was sent as a GET. The Allow header names the method to use. | Codes |
413 | The request body is larger than the service, or the client’s profile, allows. | Request size |
415 | A body on a GraphQL path is not a single operation GraphPilot can check, such as a batch or a multipart upload. | request. |
A browser reports a CORS error
Section titled “A browser reports a CORS error”The browser hides the real answer behind a CORS error when a response carries no
Access-Control-Allow-Origin for the calling page. A response from your origin, a cache hit
included, carries the CORS headers your origin sent. A response GraphPilot produces itself, such
as a refusal from the table above, carries none unless you
configure them: see edge rejections. Read the status and
code in the browser’s network panel, not the console message.
A deployment was refused or shows findings
Section titled “A deployment was refused or shows findings”A deployment refused for a value it names: see a deployment can be refused. A deployment that went live with findings: see diagnostics and findings. A deployment that failed with a support reference: contact support and quote the reference (see a deployment can fail with a support reference).
Subscriptions disconnect
Section titled “Subscriptions disconnect”A handshake refused with 401 failed your auth providers; one
that fails with 502 could not reach your origin and is never retried. A close after the
handshake comes from your origin or your client. See subscriptions and
WebSockets.
Requests fail with 502 or 504
Section titled “Requests fail with 502 or 504”A 502 with the code BAD_GATEWAY means your origin could not be reached in time, failed TLS,
or sent a body GraphPilot could not decode. See connection to the
origin: timeouts,
TLS and ports,
compression and
retries. With
shielding, a 504 with the code GATEWAY_TIMEOUT means
no usable answer came back from the shield in time, and a 502 can also mean a mutation could not
be sent through the shield (SHIELD_UNAVAILABLE_NO_FALLBACK in your logs). A 503 or 504
without a GraphPilot error code came from your origin or something in front of it.
A custom domain stays pending
Section titled “A custom domain stays pending”A hostname that stays on Waiting for DNS or Verifying, or ends in Failed: see what the statuses mean and the custom domain troubleshooting, which covers an unverified domain, a record that never appears and a hostname already in use.
The CLI cannot log in
Section titled “The CLI cannot log in”- Check what the CLI holds with
gpilot auth status: see check your status. - In CI or a script,
gpilot auth logincannot ask how to sign in and exits1. Pass--api-key, or setGPILOT_TOKEN: see log in. - Exit code
3is an authentication failure, and exit code6a CLI too old to run; see staying up to date. Every exit code is on codes.
Responses are slow
Section titled “Responses are slow”Read gp-cache first. A MISS or PASS includes your origin’s time, so look at your origin and
its timeouts. A HIT depends on the distance between the
client and the nearest edge location: see latency and how to
measure your own service. If the same
request misses at many locations, shielding lets one location’s cache stand
in front of your origin.
No gp-cache header at all
Section titled “No gp-cache header at all”- The service runs in lean mode, where the cache steps aside.
- The edge refused the request before it reached the cache, for example with a
401or a413. See the edge refuses a request. - The headers are switched off with
response.cache_headers = false; see switching the headers on and off. - The request did not go through GraphPilot. Check that the client calls your service hostname or your custom domain, and that its DNS record points to GraphPilot.
Reproducing a miss deliberately
Section titled “Reproducing a miss deliberately”Configure a bypass expression naming a header only
you send, such as bypass = "header:x-debug-bypass" under [cache.policy]. Such a request is
answered by your origin and reported as PASS with CACHE_PASS_BYPASS_POLICY.
Still stuck: status page and support
Section titled “Still stuck: status page and support”Check the status page. If the problem is on GraphPilot’s side, it is posted at graphpilot.instatus.com.
Contact support from the portal. Open Help and feedback (the button in the bottom right
corner of the portal, or the same entry in your account menu) and choose Contact support.
Open it from the service you are asking about: the request carries the organization and service
you are looking at, so nobody has to ask for them. Include what you already know: the time, the
service hostname, and a gp-request-id or correlation id if you have one. The answer comes by
email; how soon depends on your plan (see contact support).