Appearance
Inference API
The inference endpoint routes a request through the leaderboard's selected model.
bash
export API_BASE="https://app.drgero.ai"
export DRGERO_TOKEN="drgero_REPLACE_WITH_TOKEN_FROM_SETTINGS"
export LEADERBOARD_ID="b60fe691-06a3-4261-bec3-6080380dc72d"Request
bash
curl -sS -X POST "$API_BASE/v1/leaderboard/$LEADERBOARD_ID/inference" \
-H "Authorization: Bearer $DRGERO_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"input": "What did the user ask?"
}'Requires leaderboards:inference and a completed leaderboard ranking.
Alternate path
The same handler is also available under:
text
POST /api/leaderboards/{leaderboard_id}/inferenceUse the /v1/leaderboard/... path for public runtime integrations.
Body fields
| Field | Type | Description |
|---|---|---|
input | string | Main task input. |
prompt, question, query | string | Aliases used when input is absent. |
messages | array | Chat messages. If present, messages are forwarded instead of rendering the leaderboard prompt around input. |
temperature, max_tokens, etc. | any | Forwarded to compatible providers when safe. |
stream | boolean | Streaming is not supported; stream: true returns an error. |
Reserved fields such as model, input, prompt, question, and query are not forwarded as provider overrides.
Prompt rendering
If you pass input, Dr.Gero renders the leaderboard challenge prompt:
- If the prompt contains
{input},{question}, or{query}, the placeholder is replaced. - If it has no placeholder, the input is appended to the prompt.
If you pass messages, the messages are sent as the chat payload and prompt rendering is skipped.
Automatic OpenRouter prompt caching
For OpenRouter targets, sufficiently large shared prompt prefixes are checked for provider caching support automatically. With input, place stable reference text before the placeholder in the leaderboard prompt. With messages, leading text-only system/developer messages can supply reusable context.
Caller-supplied provider, session_id, prompt_cache_key, prompt_cache_options or cache_control settings (including explicit block-level cache markers) take precedence and suppress Dr.Gero's automatic additions. Provider support still varies. This feature reuses prompt processing, not previously generated answers.
Inspect upstream usage.prompt_tokens_details.cached_tokens when reported. A positive count confirms reuse; missing reporting is unknown. See Automatic prompt caching for attachment limits, fallback behavior and operator configuration.
Selected model behavior
The selected model is determined by the leaderboard:
best_balanced: the default. Use the Gero-0 routing mix when the latest ranking contains an eligible complementary model pair; otherwise fall back to the ranking winner.ranking_winner: use the top model from the latest completed ranking.manual: use the pinned leaderboard model.
A leaderboard without a completed ranking returns a conflict error until a run completes.
Response headers
Important response headers:
http
X-Dr.Gero-Trace-Id: 9fe32c4b-...
X-Dr.Gero-Leaderboard-Model-Id: 3e3c79c0-...
X-Dr.Gero-Model-Source: ranking_winner
X-Dr.Gero-Model-Ref: openai/gpt-5.2
X-Dr.Gero-Balanced-Model: Gero-0
X-Dr.Gero-Balanced-Accurate-Share: 0.35
X-Dr.Gero-Model-Version-Id: 72e18a0c-...
X-Dr.Gero-Token-Budget-Remaining-Usd: 19.42Store X-Dr.Gero-Trace-Id with your application logs so you can join production events to Dr.Gero traces. X-Dr.Gero-Model-Source identifies how the candidate was selected, such as best_balanced, best_balanced_fallback, ranking_winner, manual, or ranking_winner_fallback; it is not the model provider. Balanced-routing headers are present only when Gero-0 selected the request target. The model version header is present when the selected target is a deployed Dr.Gero model version.
OpenAI-style request example
bash
curl -sS -X POST "$API_BASE/v1/leaderboard/$LEADERBOARD_ID/inference" \
-H "Authorization: Bearer $DRGERO_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "system", "content": "Answer as a concise support agent."},
{"role": "user", "content": "Can I reset my password?"}
],
"temperature": 0.2,
"max_tokens": 300
}' | jqCommon errors
| Status | Meaning |
|---|---|
| 401 | Missing/invalid token. |
| 403 | Token lacks leaderboards:inference or workspace lacks entitlement. |
| 404 | Leaderboard not found. |
| 409 | Leaderboard has no completed ranking/selected model yet. |
| 400 | Invalid body, including stream: true. |
Human evaluation routing
Human leaderboards use this same endpoint, token scopes and inference logs. Automatic routing uses candidates with confirmed human verdicts in the current configuration; Gero-0 also requires eligible cost data. Manual pinning may select an active production candidate. Common-case scores remain comparison diagnostics. Requests use the leaderboard default prompt and caller input, not case-specific prompts or attachments. See Human evaluation serving.