Fine-tuning embedding models and rerankers: contrastive loss, hard negatives and retrieval metrics
When a general embedding model misses your domain's vocabulary, how to fix it: bi-encoders versus cross-encoder rerankers, the contrastive (InfoNCE) loss and in-batch negatives, hard-negative mining, synthetic query generation, and a runnable toy that trains embeddings from scratch and measures recall@1 and MRR.
Retrieval-augmented generation lives or dies on retrieval. If the right passage is not in the top results, even a perfect model cannot answer. General-purpose embedding models are good but they were not trained on your company's jargon, product names or the way your customers describe problems. Fine-tuning the embedding model, or a reranker on top of it, is often the cheapest big improvement to a RAG system, and it needs far less compute than fine-tuning a language model. It follows RAG in depth and embeddings and rerankers at scale.
Pro content
Keep reading with Inference Academy Pro
Unlock the rest of this lesson, including 1 animated diagram, plus every advanced lesson, hands-on lab, the larger playground models and completion certificates.