Appearance
Reference attachments and versions
The Dataset tab supports shared reference documents in Exact, Judge and Human evaluation. Human evaluation additionally supports This case only attachments in its case editor.
Shared documents accompany each selected dataset example's candidate requests in future Exact/Judge ranking runs, including scheduled runs. For Human evaluation, each new case snapshots the shared files and any case-only files. Attachments are reference context, not dataset rows; the Judge mode's configured scoring criteria and reference answers remain unchanged. Evaluation attachments are not automatically inherited by production inference.
Formats and limits
Supported: TXT, Markdown, PDF, DOCX, CSV, TSV, JSON, YAML, XML and HTML. Text must use UTF-8. Limits remain 5 MiB per file, 8 attachments, and 10 MiB combined; human cases count shared and case-only files together. Extracted text must contain 1–200,000 characters and is never silently truncated.
DOCX is extracted as text; images and layout are excluded. PDFs require OpenRouter candidates and may incur additional provider processing charges. Other candidate providers explicitly reject PDFs; use a text document for them.
Reference documents are treated as literal content: JSON braces and {input} inside a document are preserved. Shared text precedes the varying example and participates in automatic prompt caching when eligible. PDF file size does not establish caching eligibility. Caching never removes attachments.
When files change
| Change | Effect |
|---|---|
| Add a shared file | Used by future runs/new cases; advances attachment revision. |
| Remove a shared file | Archives it; future runs/new cases exclude it. Saved references still resolve. |
| Update content | Archive the old immutable file and upload a replacement. These are two separate changes. |
| Repeat an identical upload request or remove an already-archived file | No additional version change. |
| Add/remove a case-only file while preparing a human case | Changes that case's input, not the shared leaderboard configuration. |
Exact/Judge: after the first successful ranking, shared-file changes advance the working-state and dataset versions even when the evaluation dataset bytes are unchanged. Previous rankings become stale and models can be evaluated again. Setup before the first successful ranking remains pre-v0.
Human: the Dataset tab shows a shared attachment revision. Ranking comparison groups (cohorts) include shared document IDs, hashes, names and order as well as the default prompt and candidates. Reviews made with a different shared reference set do not enter the current comparison. Case-only attachments and per-case prompt overrides remain individual evaluation inputs. Returning to an identical reference-free configuration can reuse matching reference-free evidence.
Existing human cases derive their comparison identity from their saved inputs, so earlier reviews remain associated with the references used to generate their responses. A saved draft can still generate using archived references when its models and default prompt are unchanged; its review contributes only to the matching comparison group.
Active runs and privacy
Shared-file edits are blocked during an active automated ranking. Wait until it finishes or cancel it first. If references change while a run is being prepared, retry using the current configuration. Completed runs retain their original reference set.
Documents are private to the workspace. List/upload responses return metadata, not document bytes. Workspace membership is required. Document-management endpoints use a signed-in Supabase session, not a Dr.Gero API token. See the API reference.
References during prompt optimization
Optimize prompt protects every reference by default. Explicitly select text guidance that may change; native PDF/image references are not supported by the optimizer. Candidate edits remain private to the experiment until Apply as new version. Apply creates one atomic prompt/reference version, preserving original uploads and historical snapshots; it does not use the ordinary archive-then-upload replacement sequence. Restoration creates another version and refuses to overwrite subsequent input edits. Evaluation references still do not automatically become production inference attachments.