Skip to content

Reference attachments and versions ​

The Dataset tab supports shared reference documents in Exact, Judge and Human evaluation. Human evaluation additionally supports This case only attachments in its case editor.

Shared documents accompany each selected dataset example's candidate requests in future Exact/Judge ranking runs, including scheduled runs. For Human evaluation, each new case snapshots the shared files and any case-only files. Attachments are reference context, not dataset rows; the Judge mode's configured scoring criteria and reference answers remain unchanged. Evaluation attachments are not automatically inherited by production inference.

Formats and limits ​

Supported: TXT, Markdown, PDF, DOCX, CSV, TSV, JSON, YAML, XML and HTML. Text must use UTF-8. Limits remain 5 MiB per file, 8 attachments, and 10 MiB combined; human cases count shared and case-only files together. Extracted text must contain 1–200,000 characters and is never silently truncated.

DOCX is extracted as text; images and layout are excluded. PDFs require OpenRouter candidates and may incur additional provider processing charges. Other candidate providers explicitly reject PDFs; use a text document for them.

Reference documents are treated as literal content: JSON braces and {input} inside a document are preserved. Shared text precedes the varying example and participates in automatic prompt caching when eligible. PDF file size does not establish caching eligibility. Caching never removes attachments.

When files change ​

ChangeEffect
Add a shared fileUsed by future runs/new cases; advances attachment revision.
Remove a shared fileArchives it; future runs/new cases exclude it. Saved references still resolve.
Update contentArchive the old immutable file and upload a replacement. These are two separate changes.
Repeat an identical upload request or remove an already-archived fileNo additional version change.
Add/remove a case-only file while preparing a human caseChanges that case's input, not the shared leaderboard configuration.

Exact/Judge: after the first successful ranking, shared-file changes advance the working-state and dataset versions even when the evaluation dataset bytes are unchanged. Previous rankings become stale and models can be evaluated again. Setup before the first successful ranking remains pre-v0.

Human: the Dataset tab shows a shared attachment revision. Ranking comparison groups (cohorts) include shared document IDs, hashes, names and order as well as the default prompt and candidates. Reviews made with a different shared reference set do not enter the current comparison. Case-only attachments and per-case prompt overrides remain individual evaluation inputs. Returning to an identical reference-free configuration can reuse matching reference-free evidence.

Existing human cases derive their comparison identity from their saved inputs, so earlier reviews remain associated with the references used to generate their responses. A saved draft can still generate using archived references when its models and default prompt are unchanged; its review contributes only to the matching comparison group.

Active runs and privacy ​

Shared-file edits are blocked during an active automated ranking. Wait until it finishes or cancel it first. If references change while a run is being prepared, retry using the current configuration. Completed runs retain their original reference set.

Documents are private to the workspace. List/upload responses return metadata, not document bytes. Workspace membership is required. Document-management endpoints use a signed-in Supabase session, not a Dr.Gero API token. See the API reference.

References during prompt optimization ​

Optimize prompt protects every reference by default. Explicitly select text guidance that may change; native PDF/image references are not supported by the optimizer. Candidate edits remain private to the experiment until Apply as new version. Apply creates one atomic prompt/reference version, preserving original uploads and historical snapshots; it does not use the ordinary archive-then-upload replacement sequence. Restoration creates another version and refuses to overwrite subsequent input edits. Evaluation references still do not automatically become production inference attachments.