Dr.Gero Docs

Appearance

Push datasets API ​

PUSH leaderboards accept examples through a webhook. The runtime path is:

text
/v1/leaderboard/{leaderboard_id}/dataset/push

The same handler also supports /api/leaderboards/{leaderboard_id}/dataset/push.

Push a single row ​

bash
export API_BASE="https://dr-gero-frontend-99142474693.europe-west1.run.app"
export DRGERO_TOKEN="drgero_REPLACE_WITH_TOKEN_FROM_SETTINGS"
export PUSH_LEADERBOARD_ID="241b151a-407b-4dfb-9bad-e95906567647"

curl -sS -X POST "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/push" \
  -H "Authorization: Bearer $DRGERO_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "input": "What did the user ask?",
    "output": "The answer your current LLM returned"
  }'

Requires a Dr.Gero API token with leaderboards:write or a push-only hpd_ token.

Push multiple rows ​

bash
curl -sS -X POST "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/push" \
  -H "Authorization: Bearer $DRGERO_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "rows": [
      {
        "id": "ticket-1001",
        "input": "How do I cancel?",
        "output": "You can cancel from Billing settings.",
        "metadata": {"source": "support", "tier": "free"}
      },
      {
        "id": "ticket-1002",
        "input": "Where are invoices?",
        "output": "Open Settings, then Billing.",
        "tags": ["billing", "invoice"]
      }
    ]
  }' | jq

Response ​

Accepted pushes return 202 Accepted:

json
{
  "accepted_rows": 2,
  "rejected_rows": 0,
  "status": "accepted",
  "batch_id": "bdb2164f-...",
  "pending_events": 102,
  "limits": {
    "daily_event_limit": 5000,
    "monthly_event_limit": 50000,
    "max_samples_to_gather": 1000
  },
  "consolidation": {
    "should_consolidate": false
  }
}

If quotas are exhausted, the endpoint may accept zero rows and return rejection information.

Create a push token ​

bash
curl -sS -X POST "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/token" \
  -H "Authorization: Bearer $DRGERO_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name":"production webhook"}' | jq

The response may include a one-time token value beginning with hpd_, plus webhook and dataset URLs.

Push with hpd_ token ​

bash
export PUSH_TOKEN="hpd_REPLACE_WITH_PUSH_TOKEN"

curl -sS -X POST "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/push" \
  -H "X-Dr.Gero-Push-Token: $PUSH_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"input":"hello","output":"world"}' | jq

Dataset status ​

bash
curl -sS "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/status" \
  -H "Authorization: Bearer $DRGERO_TOKEN" | jq

Download consolidated JSONL ​

bash
curl -sS "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset" \
  -H "Authorization: Bearer $DRGERO_TOKEN" > dataset.jsonl

Consolidate pending rows ​

bash
curl -sS -X POST "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/consolidate" \
  -H "Authorization: Bearer $DRGERO_TOKEN" | jq

Generate auto labels ​

bash
curl -sS -X POST "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/auto-labels" \
  -H "Authorization: Bearer $DRGERO_TOKEN" | jq

Revoke push tokens ​

bash
curl -sS -X DELETE "$API_BASE/v1/leaderboard/$PUSH_LEADERBOARD_ID/dataset/tokens" \
  -H "Authorization: Bearer $DRGERO_TOKEN" | jq

Row normalization ​

The server normalizes rows before storage:

  • input is read from input, prompt, question, query, or converted from messages.
  • expected/output is read from expected, output, response, completion, or answer.
  • Metadata includes metadata plus common top-level fields such as trace_id, session_id, conversation_id, user_id, source, model, provider, latency_ms, cost_usd, tags, and timestamp.
  • A pushed_at timestamp is added.
  • Rows are capped by per-request and per-field limits.