Abstract
Archivists find things in qrate today by matching text in cells. Most of what a collection is about lives in the linked files, not in the cells: what a photograph shows, what a scanned letter says. This proposal extends search to the files themselves, in three capabilities that share one background indexing pipeline:
- Linked-file text search. The existing search bar also matches text inside linked PDFs (and later OCR text from scans).
- Visual search. Describe what is in an image ("men standing in front of a sawmill") and get rows ranked by how well their file matches.
- Similar items. From any item, see the items that look most like it. Used for deduplication, batch matching, and finding unlabelled photos of the same subject.
A later phase reuses the same pipeline for suggested subjects and captions, always staged as drafts the archivist accepts, never written into a cell.
Mockups will be produced separately (Claude Design). This doc states what the UI must let the user do, not how it looks.
Requirements (summary)
Functional
- Text search covers cell values (today) and text inside linked PDFs, with a hit that says which page it is on.
- A visual mode for the same search bar: natural-language query, results ranked by similarity.
- "Find similar" from a gallery card, the Details panel, and the fullscreen viewer.
- Works across every file type qrate can already thumbnail: images, PDFs (cover page), RAW, video (poster frame), audio cover art.
- Indexing runs in the background, survives restarts, and only re-processes files that changed (keyed on path, size, mtime).
- The index records which model produced it. Vectors from different models are never mixed; changing model means re-indexing, and the UI says so.
- A scan with no text layer is reported as "not searchable as text", never as "no matches" (
preview::pdf::has_text_layer already detects this).
Deployment
- On device by default. CPU only must work. GPU is an optional speed-up.
- Organisation-hosted server as an alternative: a URL (and optional key) in settings pointing at an OpenAI-compatible embeddings server (e.g. Infinity serving CLIP or SigLIP). qrate sends the 512px thumbnail and the query text, nothing else.
- Sending collection material off the machine is an explicit, visible choice per project or per install, not a silent default. Relevant to donor restrictions and culturally sensitive material (OCAP/CARE).