The article argues that choosing an embedding model is a schema decision requiring versioned migrations, not a static configuration value. Vectors from different models or settings—such as OpenAI's text-embedding-3-small (1536 dimensions default) versus text-embedding-3-large (3072 dimensions) or truncated dimensions like 256—occupy incompatible vector spaces; querying with one model against another's index silently degrades results. Model cards dictate critical preprocessing: E5 models require "query: " or "passage: " prefixes, Cohere v3+ mandates input_type (search_query/search_document), and token limits vary (OpenAI 8,192 vs. E5-base-v2 512 tokens). Selection should rely on custom evaluation—recall@10 across 50–100 real queries—rather than MTEB leaderboard averages, as no single model dominates all tasks. Matryoshka Representation Learning enables shorter vectors; OpenAI reports 3-large at 256 dimensions outperforms ada-002 at 1536 dimensions on MTEB (64.6% vs. 61.0%), reducing storage from ~123 GB to ~10 GB for 10M float32 vectors. Re-embedding 10M chunks (5B tokens) costs ~$100 (3-small) or ~$650 (3-large) at list prices, halved via Batch API (50k inputs/batch). Migration strategy: store emb_version (model + dimensions + preprocessing) and content_hash per vector; backfill new version alongside old, dual-write during transition, shadow-test recall, then cut over via single config switch governing both query embedding and index version.
Read original
dev.to