To troubleshoot LLM API latency, separate waiting for the model from waiting for the rest of the Agent. A slow test command, repeated retry, or approval prompt can dominate the task even when generation is fast.
The measurement workflow below is a suggested diagnostic method, not an Agent.Space speed benchmark. Provider guidance was checked September 29, 2026.
Define what “slow” means
Choose the interval you care about:
- Time to first visible output: how long the user sees no useful response.
- Generation time: how long the response continues producing output.
- Tool time: time spent reading files, running commands, or waiting on external services.
- Accepted-task time: the full path to a usable result, including retries and review.
Streaming can improve how quickly progress appears without making the entire task finish faster. OpenAI's latency guidance discusses streaming, output length, and request count as separate levers.
Record one task as a timeline
Keep the model, task, client, and endpoint fixed. Mark these events with a consistent clock:
A response event may be metadata rather than text. Be explicit about your definition instead of comparing two different “first token” measurements.
Keep request IDs, output size, retry count, and whether the prompt was cached. Do not record raw keys or sensitive prompts. The model-routing verification guide explains why a response's self-description is not reliable routing evidence.
Change the layer that is actually slow
Client timing alone cannot separate provider queueing from model computation. Label that interval “provider/transport wait” unless you have stronger evidence. Latency also cannot prove a provider substituted another model.
Compare one controlled change
Run a small, budgeted set of representative tasks. Change one factor: shorter unnecessary output, fewer redundant calls, or a corrected retry policy. Check that the result still satisfies the same acceptance condition.
Keep individual timings and failures, not only an average. A few samples can expose a reproducible stall; they do not establish a reliable production percentile or a universal model ranking.
If tools take most of the time, switching models may add cost without fixing the bottleneck. If shorter answers omit required work, the apparent speedup is not a valid win.
Keep the connection and the task observable
Use the Developer API guide to establish a known connection before diagnosing performance. For model choices, consult the coding Agent model selection guide after the timing evidence is clear.
Open Agent.Space if you need a supported client setup. Start with a reproducible small task, and judge speed by useful completion—not the first animated token.