Performance and benchmarks

This page explains how the latency figures on the GraphPilot website are measured, what exactly they include, and how you can measure your own service the same way. What the cache does to earn those numbers is on how a request flows; what it can and cannot cache is on when a response is cached. It is published so every figure shown can be checked rather than taken on trust.

What is measured: p95 HTTP response time at the load generator

The latency figures on the GraphPilot website are the p95 HTTP response time of cache hits, measured by a k6 load test from three cloud regions (Europe, US East, Asia). The metric is k6’s built-in http_req_duration.

http_req_duration is measured at the load generator, not at the edge. It is the sum of three k6 timings:

http_req_sending: sending the request to the edge.
http_req_waiting: waiting for the first response byte (time to first byte, TTFB).
http_req_receiving: downloading the response body.

It therefore includes the network round trip between the load generator and the nearest edge location, the edge’s own work, and the full body download. It excludes the DNS lookup and the TCP and TLS connection setup, which k6 reports separately; k6 keeps connections open between requests, so a VU pays that setup once rather than on every request.

k6 load generatorcloud region

GraphPilot edge

Demo origin

Measured: request sent to last byte received

Cache miss only

The measurement starts and ends at the load generator.

Because the clock runs at the load generator, the figure is neither an edge-internal timing nor time to first byte. It is closer to what a client in a well-connected data center sees: a client on home Wi-Fi or a mobile network adds its own last mile on top.

How the benchmark runs

The benchmark runs against a dedicated GraphPilot service on the production edge, in front of a demo origin in a single European region. The demo origin answers a fixed set of deterministic, cacheable queries, so the response bytes are the same on every run.

Each size class (see Payload size classes) gets its own cache-hit run:

Regions. The load is split across three cloud regions: 40% Europe, 30% US East and 30% Asia. Each request is served by the edge location nearest its region, and the percentiles are formed across all three regions combined.
Warm-up. 5 virtual users (VUs) send the query for 15 seconds. The first request at each edge location fills the cache there.
Measurement. 20 VUs send the same query for 60 seconds, in a closed loop with no think time: each VU sends its next request as soon as the previous one has returned.
Hit check. Every request in the measurement phase is checked for status 200 and for the gp-cache: HIT response header, so a cache miss shows up as a failed check instead of passing as a hit.

The whole run lasts 75 seconds, well inside the 300-second max_age of the benchmark queries, so no entry expires mid-run.

http_req_duration is recorded by k6 for every request of the run, the warm-up included. The warm-up’s first request per edge location is a cache miss; a handful of misses among thousands of requests sits above the p95 and barely moves it. Alongside it, each run records time to first byte and full response time for the measurement phase alone, which leave the warm-up out.

The same setup also runs cache-miss tests (10 VUs for 60 seconds after a 10-second warm-up, every request a unique operation so it reaches the origin) and a stress ramp (rising from 20 to 2,000 requests per second over two minutes, then one minute at the peak, 95% cache hits). Those runs exist to compare against and to check that cache-hit latency holds as the request rate rises; the website figures come from the cache-hit runs only. A cache miss reflects the distance to the origin and the origin’s own speed, not the cache, and its numbers would say little about your origin.

Payload size classes

The benchmark uses five queries. Their response sizes are fixed by the demo origin and checked by a test, so a size class means the same bytes on every run. Sizes in KB use 1 KB = 1,024 bytes.

Class	Query selects	Response size	Represents
small	one item: `id name price rating`	101 B	a single-entity or detail query
medium	300 items: `id name price rating`	22,230 B (~22 KB)	a list query
typical	700 items: `id name price rating`	51,831 B (~50 KB)	the size a GraphQL p95 list or collection response commonly lands at
large	1,500 items: `id name price rating description tags`	390,029 B (~381 KB)	a heavy, data-rich response (the tail)
nested	4 stores, 16 departments, 80 products, 240 variants, five object levels deep	26,990 B (~26 KB)	query shape rather than size

The four flat classes measure size. nested measures shape: three of its levels omit id, so on a cache miss the edge adds the key field to the origin request and strips it from the response. On a cache hit the edge serves the stored bytes, so the shape costs nothing there.

On a cache hit, http_req_duration grows with the response size because the body download grows; the edge does no backend work for a larger response. That growth is bandwidth between the edge and the load generator, not cache lookup time.

The figures on the website

The homepage shows the p95 cache-hit response time per size class, from the benchmark run of 2026-10-02. Each measured value is rounded up to the next millisecond:

Class	Response size	Measured p95 (cache hit)	Shown
small	101 B	8.2 ms	9 ms
medium	~22 KB	10.0 ms	10 ms
typical	~50 KB	12.0 ms	12 ms
large	~381 KB	17.8 ms	18 ms

The nested class is measured but not shown on the homepage.

What each run records

Every run is recorded with:

its kind (cache hit, cache miss or stress) and size class, start time and duration,
the regions and their split, and the number of requests sent from each region,
the number of VUs over time,
k6’s built-in HTTP timings for every request (http_req_duration and its parts, including http_req_waiting), and its error rate,
time to first byte and full response time for the measurement phase alone,
the outcome of the checks: status 200, and gp-cache: HIT for a cache-hit run or gp-cache: MISS for a cache-miss run.

The response size is not part of the k6 run. It is fixed per class by the demo origin, as listed in Payload size classes. The query text of each class is fixed as well.

Percentiles, not averages

The figures are percentiles rather than a single average, because averages hide the slow tail that users actually feel.

p50 (median): half of requests are this fast or faster.
p95: 95% of requests are this fast or faster; only the slowest 5% are above it.
p99: 99% of requests are this fast or faster.

A p95 figure of 12 ms means 95% of the requests in that run had an HTTP response time of 12 ms or less. p95 is a common industry choice because it captures real-world consistency without being dominated by rare outliers.

”Reproducible demo benchmark”, not an SLA

The latency figures on the GraphPilot homepage are a demo benchmark run under controlled conditions: a demo origin, fixed queries, and load generators in cloud data centers. They illustrate the effect of edge caching; they are not a guaranteed service level. Actual latency depends on geography, cache hit rate, query shape, response size, and the client’s own network.

GraphPilot does not publish a fixed number of edge locations and does not guarantee a specific latency.

How to measure your own service

You can measure your own service the same way. The key is to measure from stable cloud vantage points, not from a laptop over Wi-Fi, to warm the cache before you measure, and to keep cache hits and cache misses in separate runs.

1. Break a single request into its phases

curl reports each phase, so you can see how much is connection setup, how much is waiting for the edge, and how much is the body:

curl -w "dns:%{time_namelookup} connect:%{time_connect} tls:%{time_appconnect} ttfb:%{time_starttransfer} total:%{time_total}\n" \
  -o /dev/null -s https://<your-service-hostname>/graphql

ttfb - tls (time_starttransfer - time_appconnect) approximates the round trip plus the edge’s time to start answering; total - ttfb is the body transfer. Run it once over Wi-Fi and once over a wired connection to see your own last-mile contribution.

2. Choose the metric that answers your question

k6 reports both timings for every request:

http_req_duration is the full response time, including the body download. It is the metric behind the GraphPilot website figures, and the one closest to what your clients wait for.
http_req_waiting is time to first byte. It leaves out the body download, so it stays roughly flat across response sizes and shows how fast the edge starts answering. Use it when you want to compare cache performance across very different payload sizes without the bandwidth of the link in the number.

Both are measured at the load generator, so both include the network round trip to the nearest edge location. When you publish or compare a figure, name the metric it is.

3. Warm up, measure, then confirm each request was a hit

// k6: warm the cache, then measure cache hits only
import http from "k6/http";
import { check } from "k6";
import { Trend } from "k6/metrics";
import exec from "k6/execution";

const ttfb = new Trend("ttfb_ms", true);
const total = new Trend("total_ms", true);

export const options = {
  scenarios: {
    warmup: { executor: "constant-vus", vus: 5, duration: "15s" },
    measure: { executor: "constant-vus", vus: 20, duration: "60s", startTime: "15s" },
  },
};

export default function () {
  const res = http.post("https://<your-service-hostname>/graphql", body, params);
  if (exec.scenario.name === "measure") {
    ttfb.add(res.timings.waiting);   // time to first byte
    total.add(res.timings.duration); // full response, same as http_req_duration
    check(res, { "cache hit": (r) => r.headers["Gp-Cache"] === "HIT" });
  }
}

Keep the warm-up and measurement windows shorter than your query’s max_age, so entries do not expire mid-run. Run cache misses separately: a miss reflects your origin’s speed, not the cache’s.

4. Measure from multiple regions and combine

A single location is not a global figure. Drive the load from several regions (multiple cloud regions, or a load-testing service with global load zones) and form the p50, p95 and p99 across all of them. Repeat across a few representative response sizes: time to first byte stays roughly flat, while the full response time grows with the payload.

Summary

The website figure is the p95 HTTP response time (k6 http_req_duration) of cache hits, measured at the load generator.
It includes the network round trip to the nearest edge location and the body download; it excludes DNS and connection setup.
It is measured from three cloud regions (Europe, US East, Asia), after a warm-up, with every measured request checked to be a cache hit.
It is reported per payload size class, rounded up to the next millisecond, and as a percentile, not an average.
Homepage figures are a reproducible demo benchmark, not a guaranteed SLA or uptime commitment.