Tagging and embeddings
The passes that turn imported card text into concept associations and vectors, and what depends on each.
Imports write what a card says. Two further families of jobs write what it means: tagging, which associates cards with concepts, and embedding, which produces the vectors that make similarity search work.
They run in that order, and the order matters more than it looks. A card's embedding includes its tags, so a card whose tags change needs a new vector before search reflects them.
Tagging
Three passes write to
card_concept_associations, each
recording which pass produced the row:
| Job | Method | Source recorded |
|---|---|---|
pnpm classify:mechanical | Pattern matching from tag_detection_rules | mechanical |
pnpm classify:vector | Cosine similarity between concept and card vectors | vector |
pnpm classify:llm | A model reading the card, for associations the other two cannot see | llm |
The passes are independent and additive. None overwrites another, so a card matched by two passes carries two rows, and agreement between passes is information rather than duplication. See taxonomy and tagging for what the vocabulary is and how to argue with a tag.
classify:vector reads the card vectors, so it needs embeddings to exist
first. That is the one place the ordering doubles back: embed, then vector tag,
then embed again to fold the new tags in. The second embed run picks up exactly
the cards the tagging changed, because writing an association marks the card.
classify:llm costs real money to run across the whole card pool and is run
deliberately rather than routinely.
Embeddings
| Job | Embeds |
|---|---|
pnpm embed:cards | Every card without a vector, and every card whose tags have changed since its last one |
pnpm embed:taxonomy | Every concept in game_concepts |
Both use Voyage voyage-3, truncated to 256 dimensions. The truncation is
deliberate: oracle text is short, and the shorter vector indexes and searches
faster without measurably worse ranking.
A card's embedded text is its oracle text and type line, plus its keywords and
its tags. That is why a card tagged ramp can rank for a search about ramp even
when its rules text never uses the word. For multi-face cards every face
contributes its name, type line and text, since the top-level oracle text on
those cards is empty.
embed:cards covers cards with no vector and cards whose tags have moved since
they were embedded. cards.tags is maintained by a trigger over
card_concept_associations, and
that trigger flags the card when the array changes, so the next run refreshes it
with no flag to pass and no SQL to write.
The card keeps its existing vector until that run, which is deliberate. Clearing it to force a refresh would drop the card out of re-ranking altogether in the meantime, and ranking a card on slightly old text beats not ranking it at all.
Card text is the case this does not cover. Editing oracle text does not mark anything, so a re-embed after an errata is still explicit.
What depends on this
| If this has not run | The symptom |
|---|---|
embed:cards | New cards appear in results but never rank highly, since ranking skips them |
embed:taxonomy | A concept whose definition changed still matches the old wording |
| The tagging passes | tag: searches miss cards that ought to match |
None of these fail loudly. Ranking degrades and searching gets quieter, which is the same shape as the scheduled jobs problem: the absence of a run looks like ordinary output. The data model records the owning job for every column, which is what to re-run when a value looks wrong.