Skip to content

Partitioning your cache

Most data behind a login is not personal. A project list is the same for everyone in the organization, a billing page is the same for every admin in it. Caching such data per user (PRIVATE) works, but it pays one origin request per person for an answer the whole team shares. This guide keys it on the organization, and on the role where the role changes the answer. The concepts are on who shares an entry; this page is the setup.

The running example is a SaaS product whose identity provider issues a JWT with two claims besides sub: org_id and role. Five people use it:

Userorg_idrole
Anaacmeadmin
Benacmemember
Cemacmemember
Diaglobexmember
Eliglobexmember

Each of them opens the same page once, on a cold cache. How many entries that page’s data ends up in, and so how many requests reach your origin, depends on the audience you give it:

Ana, Ben, Cem, Dia and Eli open the same page once each, on a cold cache.
AudienceScopeEntriesOrigin requests
EverybodyPUBLIC11
The organizationORGANIZATION2 (Acme, Globex)2
The organization and roleORGANIZATION_ROLE3 (Acme admin, Acme member, Globex member)3
One personPRIVATE55

Each row holds on its own because step 2 marks both vary rules scope_only: a named scope, and PUBLIC, then partitions only on the vary names it lists, so configuring x-role for the organization-and-role rung does not split the PUBLIC or ORGANIZATION rungs by role too. Without scope_only, every configured rule partitions every response, and the first three rows would each hold 3 entries.

What you need

Section titled “What you need”
  • A GraphQL origin whose responses carry no Cache-Control of their own, so the schema decides. If your origin sends one, it decides the lifetime and has to say public or s-maxage: a bare max-age on a request carrying a token is filed per caller (see authenticated requests), so the HITs under check that it works do not happen. With public or s-maxage, every configured vary rule keys the entry on that path, so Ana does not share Ben and Cem’s entry; see scopes in the schema. On Apollo Server, see option B. For a plain HTTP API, read the steps and then the plain HTTP section.
  • Your identity provider’s JWKS URL, its issuer, and the names of the claims you partition on.
  • A service you can deploy to, and a token for at least three of the five users.

1. Verify the token at the edge

Section titled “1. Verify the token at the edge”

A claim can only partition the cache once GraphPilot has verified the token it came from. Add a provider to gpilot.toml:

gpilot.toml
[auth.providers.idp]
type               = "jwt"
allowed_algorithms = ["RS256"]
jwks_url           = "https://idp.example.com/.well-known/jwks.json"
allowed_issuers    = ["https://idp.example.com/"]
allowed_audiences  = ["my-api"]

The provider reads the Authorization header and strips a Bearer prefix. Its claims are now readable as auth:idp.<claim>, where idp is the block name. Every key is on [auth.providers.<name>], and what happens to a token that does not verify is on authentication.

2. Derive the organization and the role

Section titled “2. Derive the organization and the role”

Add one vary rule per claim, and a private expression for the data that really is personal:

gpilot.toml
[cache.policy]
private = 'auth:idp.sub'


[[cache.policy.vary]]
name       = "x-org"
value      = 'auth:idp.org_id'
scope_only = true


[[cache.policy.vary]]
name       = "x-role"
value      = 'auth:idp.role'
scope_only = true

auth:idp.org_id reads the verified org_id claim; a nested claim is a dotted path, such as auth:idp.org.id. Each rule writes its value onto the request under the header named in name, and GraphPilot removes anything a client sent under that name first, so the value cannot be forged. Keep both from auth:: an organization and a role are security boundaries, and a header the caller sets is not. See vary rules.

scope_only = true leaves each rule to the scopes that name it, so a response your schema decides pays for x-role only where its scope names it, such as ORGANIZATION_ROLE. Without the flag, every rule keys every response, and the role would split even the PUBLIC price list. scope_only needs pass_to_origin = "never", the default; a rule your origin receives cannot carry it. Configure x-role only if some data really depends on the role.

3. Name the audiences in the schema

Section titled “3. Name the audiences in the schema”

Add the two scopes to your CacheControlScope enum and name the vary rules each one partitions on:

schema.graphql
directive @vary(names: [String!]!) on ENUM_VALUE


enum CacheControlScope {
    PUBLIC
    PRIVATE
    ORGANIZATION      @vary(names: ["x-org"])
    ORGANIZATION_ROLE @vary(names: ["x-org", "x-role"])
}

The names in @vary must match the name of a [[cache.policy.vary]] entry. Declare @cacheControl as well, as shown in its definition.

4. Annotate the types

Section titled “4. Annotate the types”

Put each type on the widest audience that is still correct:

schema.graphql
type Plan @cacheControl(maxAge: 3600) {                       # PUBLIC: the price list
    id: ID!
    name: String!
}


type Project @cacheControl(maxAge: 300, scope: ORGANIZATION) {
    id: ID!
    name: String!
}


type Invoice @cacheControl(maxAge: 300, scope: ORGANIZATION_ROLE) {
    id: ID!
    total: Int!
}


type Query {
    plans: [Plan!]!
    projects: [Project!]!
    invoices: [Invoice!]!                                     # admins see all, members fewer
    me: User @cacheControl(maxAge: 60, scope: PRIVATE)
}

ORGANIZATION asserts that every member may see every row. If some members may see less, the scope has to name what separates them, as ORGANIZATION_ROLE does for invoices. A cache hit never reaches your origin’s authorization; see is your field really organization-uniform?

Deploy both files with gpilot deploy. The build reports a @vary name that no [[cache.policy.vary]] entry defines under Schema → Findings; see findings.

Check that it works

Section titled “Check that it works”

Send the same query as Ben, then as Cem, then as Dia. $BEN, $CEM and $DIA hold their tokens:

Terminal window
for token in "$BEN" "$CEM" "$DIA"; do
  curl -sS -D- -o/dev/null "https://$GRAPHPILOT_HOST/graphql" \
    -H "authorization: Bearer $token" \
    -H 'content-type: application/json' \
    -d '{"query":"{ projects { id name } }"}' | grep -i '^gp-cache:'
done
gp-cache: MISS          # Ben: the first request for Acme members, stored
gp-cache: HIT           # Cem: same organization, same role, same entry
gp-cache: MISS          # Dia: Globex has its own entry

Ana, also at Acme, gets a HIT too: ORGANIZATION partitions only on x-org, and x-role is marked scope_only, so configuring it for Invoice does not split this entry by role. In Insights, the Segmentation field on the request’s Cache tab shows what the entry was split by.

A PASS with gp-cache-reason: CACHE_SKIPPED_PRIVATE_WITHOUT_DISCRIMINATOR means a name the scope lists was missing for that request. The pitfalls below cover why.

Pitfalls

Section titled “Pitfalls”
  • A name with no vary rule behind it never fills. @vary(names: ["x-org"]) without an x-org entry refuses to store every response that selects the type, forever, with CACHE_SKIPPED_PRIVATE_WITHOUT_DISCRIMINATOR. The deployment does not fail; the finding is your only warning.
  • A missing claim refuses the store. A token without org_id resolves x-org to nothing, and a named scope then refuses rather than sharing. The same happens to a claim that is an object, an array or null. A number or a Boolean resolves to its JSON text, so an org_id issued as 42 partitions as "42".
  • An empty claim shares the entry. A verified token whose org_id is "" resolves x-org to the empty string, and the empty string is a value: ORGANIZATION stores the response, and every caller whose claim is empty reads that one entry, whatever organization they belong to. Have your identity provider never issue the claim empty. A rule without scope_only does not have this gap: for a signed-in caller, an empty value there refuses the store with CACHE_SKIPPED_PRIVATE_WITHOUT_DISCRIMINATOR, at the cost of partitioning every response on it. See an empty value is a value.
  • A role changes when the token does. The role comes from the token, so a user promoted or demoted keeps reading the old role’s entries until they get a new token. Keep token lifetimes short where roles change often, and purge what a demotion must hide at once; see revocation.

When PRIVATE is right

Section titled “When PRIVATE is right”

Use PRIVATE for what differs per person: me, a shopping cart, notifications, a user’s settings, rows only they own. It partitions by cache.policy.private (auth:idp.sub above) and misses once per person. A query that selects one PRIVATE field is private as a whole, so keep personal fields out of queries that load shared data.

Per-user data you can still read by organization

Section titled “Per-user data you can still read by organization”

Notifications differ per person, so they belong on the last rung. Scoped PRIVATE, they are keyed on cache.policy.private, and Insights records that value only as a keyed digest: you can see that notification requests miss, but not whether they miss for Acme or for Globex. A named scope keyed on the organization and a pseudonym of the user is just as narrow and keeps the organization readable.

Add a third rule beside the two from step 2, and show vary values in clear:

gpilot.toml
[[cache.policy.vary]]
name       = "x-org-user"
value      = "auth:idp.org_id + ':' + pseudonymize(auth:idp.sub)"
scope_only = true


[telemetry]
vary_values = "clear"

Name it in a scope of its own and put the per-user type on it:

schema.graphql
enum CacheControlScope {
    PUBLIC
    PRIVATE
    ORGANIZATION      @vary(names: ["x-org"])
    ORGANIZATION_ROLE @vary(names: ["x-org", "x-role"])
    ORGANIZATION_USER @vary(names: ["x-org-user"])
}


type Notification @cacheControl(maxAge: 30, scope: ORGANIZATION_USER) {
    id: ID!
    text: String!
}

All five people open their notifications once. That is five entries and five origin requests, as with PRIVATE, and the Segmentation field on each request’s Cache tab in Insights reads:

UserSegment recorded for x-org-user
Ana4:acme:43: and Ana’s pseudonym
Ben4:acme:43: and Ben’s pseudonym
Cem4:acme:43: and Cem’s pseudonym
Dia6:globex:43: and Dia’s pseudonym
Eli6:globex:43: and Eli’s pseudonym

Each part is written as its length in bytes, a colon and the value, because the value combines two sources; the 43 characters after the second colon are the pseudonym. The other rules are recorded in clear too: Ben’s projects request shows acme for x-org, and his invoices request acme and member, which is the point of the setting.

  • Keep scope_only on the rule. Without it the rule partitions every response, and the PUBLIC price list and the ORGANIZATION projects would be stored once per person.
  • vary_values = "clear" covers every rule. Before you set it, check that no rule’s value is a personal identifier in clear: an email address, a user id, a session cookie. Wrap such a value in pseudonymize, as here.
  • Without clear, the segment is recorded as an hmac-sha256: digest like every other vary value, and the organization is no more readable than under PRIVATE.
  • It fails closed. A token without org_id or sub derives no x-org-user, so its notifications are not stored (CACHE_SKIPPED_PRIVATE_WITHOUT_DISCRIMINATOR).

For a plain HTTP API

Section titled “For a plain HTTP API”

Plain HTTP has no schema, so there are no named scopes: steps 1 and 2 are the whole setup, and every vary rule applies to every cached response, scope_only or not.

  • Have your origin mark shared responses public or s-maxage. A request carrying Authorization is otherwise filed per caller or not cached. Cache-Control: s-maxage=300 on GET /projects gives one entry per organization with only x-org configured.
  • Mark personal responses private. Cache-Control: private, max-age=60 on GET /me is partitioned by cache.policy.private.
  • A missing claim refuses a signed-in caller only. Without a named scope, a request that carries a credential and whose rule resolves to nothing, or to the empty string, is not stored (CACHE_SKIPPED_PRIVATE_WITHOUT_DISCRIMINATOR), so its answer never lands in the variant anonymous callers read. A request without a credential is stored as the variant without the header, shared by every other anonymous caller. Make sure every token that reaches a partitioned endpoint carries the claim, or those callers miss every time.

The curl check above works the same way against GET /projects.

Section titled “Related”