Appearance
Automatic OpenRouter prompt caching
Dr.Gero automatically checks whether an OpenRouter model can reuse a sufficiently large shared text prefix. This can reduce repeated input processing and cost when many requests use the same instructions or reference document.
This is provider prompt caching, not answer caching. Each entry still gets a new model response. Sending or storing an attachment once does not make its future model input free, and cached text still occupies the model's context window.
Where it applies
| Workflow | Eligible shared content |
|---|---|
| Automatic leaderboard evaluation | Shared text attachments plus static prompt text before the first dynamic field, including embedded reference text. |
Leaderboard inference with input | Static prefix of the leaderboard prompt. |
Leaderboard inference with messages | Leading text-only system/developer messages. |
| Human case generation with OpenRouter candidates | Static instructions and shared text attachments, including extracted DOCX text. |
The Dataset tab now provides shared attachment uploads for automatic and human evaluation. Evaluation attachments do not automatically become attachments on the production inference endpoint.
Prepare a reusable prompt
Place stable instructions and reference material first, followed by the changing entry. Use the supported {input} placeholder:
text
Classify the entry using the reference below.
Return only the category label.
Reference:
[Your compact classification reference]
Entry:
{input}For automatic evaluation, eligibility ends at the first dynamic field, such as {input}, {id}, or {metadata}. If the prompt has no placeholder, its static text can be reused and the input is appended. A long standalone user question is not assumed reusable. Keep timestamps, row IDs and other changing values after the shared reference.
Attachments
Shared text/JSON/Markdown/CSV documents and extracted DOCX text count toward the reusable prefix. In Exact, Judge and Human candidate generation, shared text documents are placed before the prompt. Document content is preserved literally. Case-only documents remain with the variable input.
PDF/image/file bytes are forwarded as before but are not counted as text tokens or given automatic cache markers. Base64 size is not a token estimate. Use an extracted text reference when attachment-only caching is needed. A mixed request can still cache eligible shared text before its binary attachments.
Automatic eligibility
Dr.Gero checks whether the shared text is large enough and whether the selected OpenRouter model supports prompt caching. Minimum lengths and cache availability vary by model and provider. Generation can continue when support cannot be confirmed; an eligible request is not a guarantee of a cache hit.
Caching preserves your instructions and each example's input. No separate paid warm-up generation is required.
Verify reuse and cost
Automatic run artifacts retain provider usage under response_metadata.usage.prompt_tokens_details. When reported, run summaries also include:
| Field | Meaning |
|---|---|
cached_tokens | Total input tokens read from provider caches. |
cache_write_tokens | Total input tokens written to provider caches. |
cache_usage_reported_requests | Requests reporting a valid cached-token count, including zero. |
cache_hit_requests | Requests with a cached-token count greater than zero. |
Missing usage remains unknown rather than being treated as zero. Human response records store cache_usage; inference retains upstream usage. No dedicated cache badge is added to the UI, and existing cost accounting remains unchanged.
Run a small pilot with different inputs and the same prefix. Look for positive cached_tokens after the first request and compare actual provider cost. Cache writes may carry a premium; cache lifetime, routing and minimum lengths affect savings. A single miss does not prove that caching is unsupported.
Custom request settings
Caller-supplied provider routing, session IDs or cache controls take precedence on inference requests: Dr.Gero does not add automatic controls to those requests. Forwarded settings are not a promise that every provider supports them.
If a provider rejects an automatic caching option, Dr.Gero may retry without that option. This does not guarantee retries for unrelated errors or caller-supplied settings.
See also Inference API, Human evaluation and Sampling datasets.
Prompt optimization exception
Prompt optimization reserves spending before each paid call and uses one HTTP attempt without the ordinary ranking cache-fallback path. Unknown interrupted requests retain their reserved cost and are not silently replayed. Provider cache hits may occur, but do not assume ranking cache savings when estimating optimization cost.