watson: Why Is the AI Slow?

TL;DR

A good ping doesn't mean a fast AI. A 20 ms path can sit in front of an 8-second wait, because a healthy network says nothing about how busy the service is.
Native AI apps have no DevTools. Claude Code, Copilot and desktop chat apps keep their timing to themselves.
watson measures it from the outside: packet timing and sizes of encrypted traffic, per response, per app.
Works in the browser too. A small extension covers web chatbots in any Chromium browser, whether they stream over fetch, SSE, XHR or WebSocket.
On a Mac it watches your side too. Wi-Fi, router pings, CPU, memory and heat tell a bad router from a bad ISP, and a starved app from a busy provider.
In 960 real interactions, the network was almost never the problem: 7 stalls, 23 ms median RTT.

Where it works

Desktop apps and the browser, one timeline

Native clients and web chatbots land in the same log, scored against the same baselines, so you can compare Claude Code with Claude in a browser tab.

Native AI apps

passive packet capture · no install in the app

Recognises each app by the process that owns the connection, and the provider by the TLS server name.

Apps

Claude CodeClaude DesktopChatGPT desktopCodexCursorVS Code + Copilot

AI chatbots in the browser

watson browser extension · timing & size only

Records when each streamed chunk arrived and how big it was. Passive capture adds round-trip and retransmit evidence where it can see the connection.

Any Chromium browser

ChromeEdgeBraveArcOperaVivaldi

Streaming transports

fetchXHRSSEWebSocket

Browser caveat: over HTTP/3 (QUIC), or on a connection opened before watch started, the network is hidden. You still get the exact first byte, duration and pauses, but the score falls back to time-to-first-byte only and is marked ~. For full network evidence, start watch first, then relaunch the browser. More on browsers and QUIC.

The playbook

Triage a slow AI response in three questions

Ask them in this order. Most "slow AI" complaints are settled by question two.

Is the network healthy?

Check round-trip time and retransmits against your normal. On a Mac, watson also tells you which side of your router a stall was on. Only touch Wi-Fi, VPN or ISP if they're off.

pauses marked L → Wi-Fi or routerpauses marked U → past your routerRTT above normal or N pauses → networkall normal → go to 02

Is the service slow?

Long time-to-first-byte or provider wait on a healthy path means the provider is the bottleneck, and nothing you change locally will help.

high TTFB or P pauses → wait or switch modelnormal → go to 03

Is it how you're using it?

Network fine, service fine, still slow? The time is going somewhere off the wire: your machine (pauses marked A, H or C), local tools, model thinking, or giant context.

trim contextstart a fresh conversationfewer tool calls per turn

How it decides

Every pause gets a culprit, with evidence

On an encrypted stream, a thinking model and a stalled network look identical. watson checks what else was happening during every pause over 1 second, in this order:

Local link

A network stall where your first hop was the problem: a spike or lost ping from your router, or weak Wi-Fi.

Check Wi-Fi and the router.

Upstream

A network stall while your router answered normally, so the problem is past your local network.

Check VPN, ISP.

Network stall

A retransmit or an RTT spike big enough to explain the pause, with no Wi-Fi or router samples to say which side it was on.

Check Wi-Fi, VPN, ISP.

Tool activity

The same app was talking to non-AI hosts. A heuristic, labelled as one.

That's the agent working.

Throttled

The Mac was hot or in Low Power Mode while it was also busy.

Plug in, or let it cool down.

App busy

The AI app itself was short of resources: its own CPU was high, memory pressure was up, or the Mac was swapping.

Free up memory, or restart the app.

Machine contention

System CPU above 90% during the pause.

Kill the build or indexer.

Provider wait

Nothing else explains it: the path was healthy and the machine idle, so the service was busy.

Wait. This never lowers the score.

Each response also gets a 0–100 Transport Score against your own rolling baseline per provider and app: connection health 35%, TTFB 30%, network stalls 25%, setup 10%. L, U and N all count as network stalls, and H counts against the score too. Provider wait never does.

If your own CPU was maxed out, watson ignores router spikes, because a busy machine is slow to answer its own pings. Each pause is also marked several apps, same provider or this app only, depending on whether other apps had trouble at the same moment.

Two days, real work

It mostly wasn't the network

Roughly 960 interactions across Claude Code, Claude Desktop, ChatGPT, Codex and VS Code, before browser support landed.

960interactions

23 msmedian round-trip

7network stalls

18%of time was provider wait

one "slow" 32s responsewhere the time went

24.7s on the wire

7.6s nothing on the wire

For 7.6 seconds nothing crossed the network. That could be a tool running, the model thinking, or you, and from the wire watson can't tell which. A network fix won't shorten it.

watson serve live page: Transport Score, TTFB and round-trip tiles, time-per-interaction chart, worst interactions table — `watson serve`: tiles vs your normal, every interaction split into setup, round-trip, server time, provider wait, stall and transfer, plus the worst offenders.

The live page also has a Report tab that counts delivery patterns and device conditions and lists recent interactions the provider didn't slow down, and a Device tab that ranks apps by CPU during spikes, with a CPU-by-app chart for each one. A per-minute strip of Wi-Fi, router ping and CPU is shaded wherever memory was tight, the machine was swapping or the Mac was throttled. The turns table has a Cause column, and opening a turn shows how much of it was on the wire, with one lane per interaction labelled with its duration.

Try it

Two terminals, five minutes

One Go binary plus an optional browser extension. macOS and Linux. Capture needs root; the live page doesn't.

# build
$ go build -o watson ./cmd/watson

# terminal 1: watch every AI response on this machine
$ sudo ./watson watch --min-duration 1

# terminal 2: open the live page it prints
$ ./watson serve

# optional, for web chatbots: in any Chromium browser
# chrome://extensions → Developer mode → Load unpacked → extension/

watson watch terminal table: per-response TTFB, duration, silent periods and score — `watson watch`: one row per response. `SILENT 1N 2P` = one network stall, two provider waits.

A turn that's still running counts up live and is marked "so far". If CPU hit 70% while nothing was on the wire, the turn's header says "machine busy". watson only ever says yes to that; it never claims the machine was idle.

Trust model

What watson never does

Decrypt trafficThere's no proxy and no root certificate. It reads only the TLS server name to know which service you're talking to.

Store contentIt keeps timestamps, sizes and TCP state. It never sees your prompts or answers. App names are recorded only while CPU is above 70%, and --no-process-names turns that off.

Read the pageThe extension keeps each chunk's arrival time and byte count, never the text, headers or URL path, and talks only to watson on this machine.

Phone homeThe log is a local file only you can read. The page refuses anything but loopback.

GuessEvery number says whether it was measured, inferred, or invisible. If watson can't see something, it says so instead of filling the gap.

Known blind spots: no tokens or model names; QUIC/HTTP3 hides retransmits and RTT, though a router spike can still mark a pause as a local-link stall; corporate VPNs and Zscaler tunnels hide destinations (watson detects this and tells you which interface to capture on instead).

"The AI is slow."
Compared to what?