Skip to content

Dataset files and reference documents ​

Dataset uploads and reference attachments use different formats and authentication. A dataset provides test examples; a reference document supplies context to candidate models.

Upload a dataset ​

POST /api/leaderboards/{leaderboard_id}/dataset/upload

Alias: POST /v1/leaderboard/{leaderboard_id}/dataset/upload.

Requires an existing owner/admin session, or a workspace API token with leaderboards:write. Existing dataset billing entitlement applies. Use a new automatic workflow created in the UI, before connecting a source or models. The upload route does not create the leaderboard; an existing configured source cannot be replaced through it.

Send raw file bytes, not multipart or base64:

bash
curl -sS -X POST "$API_BASE/api/leaderboards/$LEADERBOARD_ID/dataset/upload" \
  -H "Authorization: Bearer $DRGERO_TOKEN" \
  -H "Content-Type: application/octet-stream" \
  -H "X-Dataset-Filename: eval.jsonl" \
  --data-binary @eval.jsonl

URI-encode the filename header if it contains spaces or non-ASCII characters. Accepted extensions: .jsonl, .json, .csv. See schema, validation and size limits.

Success returns 201 and { "ok": true, "upload": { ... } }; an identical retry returns 200. Upload metadata includes filename, format, row_count, duplicate_count, sha256, uploaded_at, selected source field names. Use the authenticated dataset endpoint to retrieve the data. Invalid data produces a short error, generally 400; size limits use 413, conflicting sources use 409, and authorization/entitlement failures use existing access errors.

The existing authenticated dataset GET endpoint returns the normalized JSONL. There is no partial-row acceptance for file uploads. If an upload fails, check the current dataset before retrying.

Manage reference documents ​

These routes work in all evaluation modes and require a Supabase session bearer token plus workspace membership. Dr.Gero API tokens and push tokens do not authorize document management.

MethodPathBehavior
GET/api/leaderboards/{id}/documentsList active shared metadata.
POST/api/leaderboards/{id}/documentsUpload an immutable document.
DELETE/api/leaderboards/{id}/documents/{document_id}Archive a shared document.

The original /api/leaderboards/{id}/human/documents routes remain compatible, including for shared documents in automatic modes. Other human-case endpoints still require human evaluation.

POST body:

json
{
  "id": "c18023ef-40f5-4c03-aeab-5943e7f46e13",
  "name": "reference.txt",
  "scope": "shared",
  "data": "UmVmZXJlbmNlIHRleHQu"
}

Use a client-generated UUID and raw base64 content without a data-URL prefix. Automatic modes require scope: shared; Human permits shared or case. Success returns metadata (id, name, scope, media_type, size_bytes, sha256), with 201 for creation or 200 for an identical retry. Reusing an ID for changed content or an archived file returns 409. DELETE returns { "ok": true } and preserves the archived content for saved references.

Shared-file changes during an active automatic ranking return 409 with a wait/cancel explanation. Limits and invalid files return 400 or 413. If a concurrent change causes an error, reload the attachment list before retrying.

There is no content-update endpoint. Archive and replace the document to change it. Versioning rules apply to each meaningful shared-file change.

Human comparison identifiers ​

Human summary responses include attachment_revision, cohort and configuration_cohort. Cases also expose cohort and configuration_cohort:

  • cohort identifies the prompt, candidates and shared references used for ranking comparison.
  • configuration_cohort identifies just the default prompt and candidates, allowing a saved draft to generate with archived references when those remain compatible.
  • attachment_revision is the current shared-file revision, separate from a case's optimistic-locking revision.

Reviews contribute only to comparisons with matching saved configurations.