<div id="readability-page-1" class="page"><div><img src="https://devstarsj.github.io/assets/https://images.unsplash.com/photo-1507003211169-0a1dd7228f2d?w=1200">Retrieval-Augmented Generation (RAG) is the backbone of most production AI applications. Knowledge bases, document Q&amp;A, code search, customer support — if your application needs to answer questions about specific content, RAG is usually how you do it.The gap between “RAG prototype that works on your demo” and “RAG system that works reliably in production” is significant. This post covers the failure modes and how to address them.<img src="https://images.unsplash.com/photo-1481627834876-b7833e8f5570?w=800" alt="Library with rows of books — knowledge retrieval"> Photo by Susan Yin on Unsplash<hr><h2 id="the-basic-rag-pipeline">The Basic RAG Pipeline</h2>First, the fundamentals. A RAG system has two phases:Indexing (offline):<ol><li>Load documents</li><li>Split into chunks</li><li>Embed each chunk (convert text → vector)</li><li>Store in a vector database</li></ol>Retrieval + Generation (online):<ol><li>User submits a query</li><li>Embed the query</li><li>Search the vector database for similar chunks (approximate nearest neighbor)</li><li>Stuff the retrieved chunks into an LLM prompt as context</li><li>LLM generates an answer grounded in the retrieved content</li></ol>This is conceptually simple. The devil is in every implementation detail.<hr><h2 id="failure-mode-1-bad-chunking">Failure Mode 1: Bad Chunking</h2>Chunking is where most RAG prototypes fail silently. The way you split documents determines what can be retrieved — and therefore what the LLM can answer.<h3 id="the-naive-approach-and-why-it-fails">The naive approach and why it fails</h3><div><pre><code># Bad: split by character count, ignore semantics
chunks = [text[i:i+500] for i in range(0, len(text), 500)]
</code></pre></div>This cuts sentences mid-thought, separates context that belongs together, and creates chunks with no meaningful coherent meaning.<h3 id="better-chunking-strategies">Better chunking strategies</h3>Recursive text splitting (the standard baseline):<div><pre><code>from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
 chunk_size=512,
 chunk_overlap=50,
 separators=["\n\n", "\n", ". ", " ", ""],
)
chunks = splitter.split_text(document)
</code></pre></div>The overlap matters: without it, relevant context split across a boundary disappears.Semantic chunking: Instead of splitting on character count or separators, split on semantic boundaries — where the topic actually changes. Embed each sentence, and split when consecutive sentence embeddings diverge beyond a threshold.<div><pre><code>from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai.embeddings import OpenAIEmbeddings

splitter = SemanticChunker(
 OpenAIEmbeddings(),
 breakpoint_threshold_type="percentile",
 breakpoint_threshold_amount=90,
)
chunks = splitter.split_text(document)
</code></pre></div>Document-structure-aware chunking: For structured documents (Markdown, HTML, PDFs with headers), respect the document structure:<div><pre><code># Split Markdown at header boundaries
from langchain.text_splitter import MarkdownHeaderTextSplitter

headers_to_split_on = [("#", "H1"), ("##", "H2"), ("###", "H3")]
splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
chunks = splitter.split_text(markdown_document)
</code></pre></div>Each chunk retains its header context, which dramatically improves retrieval relevance.Parent-child chunking: Store small chunks for retrieval precision, but return the larger parent context to the LLM:<ul><li>Small chunks (128 tokens) are indexed for fine-grained retrieval</li><li>When a small chunk is retrieved, return its parent section (512 tokens) to the LLM</li></ul>This gets you the best of both: precise retrieval, sufficient context.<hr><h2 id="failure-mode-2-embedding-model-mismatch">Failure Mode 2: Embedding Model Mismatch</h2>Your embeddings are only as good as your embedding model. Common mistakes:Using a generic model for specialized content. An embedding model trained on web text performs poorly on code, legal documents, or scientific papers. Match the model to your domain.Mixing embedding models. If you re-index a portion of your documents with a new model, the vectors are not comparable. Old chunks from model A and new chunks from model B will have incorrect similarity scores. Re-embed everything when you change models.Ignoring embedding dimensions. Larger embedding dimensions (1536 vs 384) capture more nuance but cost more. For most production use cases, <code>text-embedding-3-small</code> (1536 dims, reduced to 512 works fine) is a good default.<h3 id="model-recommendations-2026">Model Recommendations (2026)</h3><table><thead><tr><th>Use Case</th><th>Model</th></tr></thead><tbody><tr><td>General text</td><td><code>text-embedding-3-small</code> (OpenAI) or <code>nomic-embed-text</code> (open source)</td></tr><tr><td>Code</td><td><code>voyage-code-3</code> (Voyage AI)</td></tr><tr><td>Multilingual</td><td><code>multilingual-e5-large</code> or <code>paraphrase-multilingual-mpnet-base-v2</code></td></tr><tr><td>Domain-specific</td><td>Fine-tune on your domain data</td></tr></tbody></table><hr><h2 id="failure-mode-3-poor-retrieval-recall">Failure Mode 3: Poor Retrieval Recall</h2>The retriever doesn’t find the relevant chunk. Common causes and fixes:<h3 id="semantic-search-alone-isnt-enough">Semantic search alone isn’t enough</h3>Vector similarity search finds semantically similar content but misses exact term matches. A user asking about “RFC 7519” needs keyword search, not semantic search.Hybrid search (semantic + keyword) is now the production standard:<div><pre><code>from pinecone_text.sparse import BM25Encoder

# Sparse (BM25 keyword) + dense (embedding) search
sparse_encoder = BM25Encoder().default()
sparse_vector = sparse_encoder.encode_queries(query)
dense_vector = embed_query(query)

# Query with both vectors (Pinecone supports this natively)
results = index.query(
 vector=dense_vector,
 sparse_vector=sparse_vector,
 top_k=20,
)
</code></pre></div>Most production vector databases (Pinecone, Weaviate, Elasticsearch with vector support) support hybrid search.<h3 id="retrieve-more-then-rerank">Retrieve more, then rerank</h3>Instead of retrieving top-5 and using them directly, retrieve top-20 and use a reranking model to select the best 5:<div><pre><code>from cohere import Client

co = Client(api_key)
results = vector_db.query(query_vector, top_k=20)

# Rerank using Cohere's reranker
reranked = co.rerank(
 query=user_query,
 documents=[r.text for r in results],
 model="rerank-v3.5",
 top_n=5,
)
</code></pre></div>Reranking models (Cohere Rerank, BGE-reranker, Jina Reranker) are cross-encoders that evaluate (query, document) pairs together — much more accurate than embedding similarity alone. Adding reranking is consistently one of the highest-ROI improvements to RAG quality.<hr><h2 id="failure-mode-4-context-window-stuffing">Failure Mode 4: Context Window Stuffing</h2>You retrieved 20 chunks, you put them all in the prompt. Problems:<ul><li>LLMs lose track of information in long contexts (“lost in the middle” effect)</li><li>Cost grows linearly with retrieved context</li><li>Irrelevant chunks actively harm answer quality</li></ul>Best practices:<ul><li>Retrieve top-20, rerank, use top-3 to 5</li><li>Measure context relevance — if retrieved chunks have similarity below 0.7, don’t include them (no context is better than bad context)</li><li>Order matters — put the most relevant chunk first or last, not buried in the middle</li></ul><hr><h2 id="failure-mode-5-no-grounding-verification">Failure Mode 5: No Grounding Verification</h2>The LLM generates a confident answer that’s not actually supported by the retrieved context. This is silent hallucination.Add a grounding check:<div><pre><code>GROUNDING_PROMPT = """
Given the following retrieved context and the AI's answer:

Context: {context}

Answer: {answer}

Question: Is every factual claim in the answer supported by the context?
Respond: grounded
"""

def check_grounding(context, answer) -&gt; dict:
 result = llm.complete(GROUNDING_PROMPT.format(context=context, answer=answer))
 return json.loads(result)
</code></pre></div>For high-stakes applications, fail or flag answers that aren’t fully grounded. For others, include a confidence score.<hr><h2 id="production-architecture">Production Architecture</h2>A mature RAG system in 2026 looks like this:<div><pre><code>User Query
 ↓
[Query Transformation] ← rewrite query for retrieval, expand synonyms
 ↓
[Hybrid Retrieval] ← dense + sparse, top-20
 ↓
[Reranking] ← cross-encoder, select top-5
 ↓
[Context Assembly] ← format, truncate, order
 ↓
[LLM Generation] ← with grounded context
 ↓
[Grounding Check] ← verify factual support
 ↓
[Response] → User
</code></pre></div>Each step is instrumented with traces (LangSmith, Langfuse, or Phoenix) so you can debug when answers go wrong.<img src="https://images.unsplash.com/photo-1558494949-ef010cbdcc31?w=800" alt="Abstract data flow and architecture diagram"> Photo by Taylor Vick on Unsplash<hr><h2 id="evaluation-framework-for-rag">Evaluation Framework for RAG</h2>Use RAGAS metrics to measure quality:<table><thead><tr><th>Metric</th><th>Measures</th><th>Target</th></tr></thead><tbody><tr><td>Faithfulness</td><td>Is the answer grounded in context?</td><td>&gt; 0.85</td></tr><tr><td>Answer Relevancy</td><td>Does the answer address the question?</td><td>&gt; 0.80</td></tr><tr><td>Context Precision</td><td>Are the retrieved chunks relevant?</td><td>&gt; 0.75</td></tr><tr><td>Context Recall</td><td>Were relevant chunks found?</td><td>&gt; 0.70</td></tr></tbody></table><div><pre><code>from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision

result = evaluate(
 dataset=eval_dataset,
 metrics=[faithfulness, answer_relevancy, context_precision],
)
print(result)
</code></pre></div>Run this against a golden dataset on every RAG pipeline change.<hr><h2 id="vector-database-selection">Vector Database Selection</h2>For 2026 production use:Pinecone — managed, scalable, hybrid search, minimal operational burden. Good default for teams that don’t want to self-manage.Weaviate — open source, self-hostable, excellent hybrid search, object storage. Good for teams that want control.pgvector (PostgreSQL) — if you’re already on Postgres and scale requirements are moderate (&lt;10M vectors), staying in-database simplifies architecture enormously. <a href="https://github.com/pgvector/pgvector">pgvector 0.8+</a> with HNSW indexing is competitive.Qdrant — high-performance, written in Rust, great for self-hosting. Strong payload filtering support.Don’t choose a vector database based on benchmarks alone. Operational maturity, filtering capabilities, and hybrid search support matter more for most production use cases.<hr><h2 id="quick-wins">Quick Wins</h2>If you have an existing RAG system and want to improve it fast:<ol><li>Add reranking — Cohere Rerank or BGE-reranker. Biggest bang for effort.</li><li>Switch to hybrid search — add BM25 alongside embeddings.</li><li>Add chunk overlap — if your current overlap is 0, add 50-100 tokens.</li><li>Add a grounding check — detect and flag hallucinations.</li><li>Instrument with traces — you can’t debug what you can’t see.</li></ol><hr><h2 id="resources">Resources</h2><ul><li><a href="https://docs.ragas.io/">RAGAS — RAG Evaluation Framework</a></li><li><a href="https://python.langchain.com/docs/modules/data_connection/document_transformers/">LangChain Text Splitters</a></li><li><a href="https://docs.pinecone.io/guides/data/understanding-hybrid-search">Pinecone Hybrid Search Guide</a></li><li><a href="https://docs.cohere.com/docs/reranking">Cohere Rerank Documentation</a></li><li><a href="https://github.com/pgvector/pgvector">pgvector — Vector Search for PostgreSQL</a></li><li><a href="https://langfuse.com/">Langfuse — Open Source LLM Engineering Platform</a></li></ul> <div>이 글이 도움이 되셨다면 공감 및 광고 클릭을 부탁드립니다 :) </div></div></div>

Dev Note

Retrieval-Augmented Generation (RAG) is the backbone of most production AI applications. Knowledge bases, document Q&A, code search, customer support — if your application needs to answer questions about specific content, RAG is usually how you do it.
The gap between “RAG prototype that works on your demo” and “RAG system that works reliably in production” is significant. This post covers the failure modes and how to address them.
Photo by Susan Yin on Unsplash
The Basic RAG Pipeline
First, the fundamentals. A RAG system has two phases:
Indexing (offline):
Load documents
Split into chunks
Embed each chunk (convert text → vector)
Store in a vector database
Retrieval + Generation (online):
User submits a query
Embed the query
Search the vector database for similar chunks (approximate nearest neighbor)
Stuff the retrieved chunks into an LLM prompt as context
LLM generates an answer grounded in the retrieved content
This is conceptually simple. The devil is in every implementation detail.
Failure Mode 1: Bad Chunking
Chunking is where most RAG prototypes fail silently. The way you split documents determines what can be retrieved — and therefore what the LLM can answer.
The naive approach and why it fails
# Bad: split by character count, ignore semantics
chunks = [text[i:i+500] for i in range(0, len(text), 500)]
This cuts sentences mid-thought, separates context that belongs together, and creates chunks with no meaningful coherent meaning.
Better chunking strategies
Recursive text splitting (the standard baseline):
from langchain.text_splitter import RecursiveCharacterTextSplitter
splitter = RecursiveCharacterTextSplitter(
 chunk_size=512,
 chunk_overlap=50,
 separators=["\n\n", "\n", ". ", " ", ""],
)
chunks = splitter.split_text(document)
The overlap matters: without it, relevant context split across a boundary disappears.
Semantic chunking:
Instead of splitting on character count or separators, split on semantic boundaries — where the topic actually changes. Embed each sentence, and split when consecutive sentence embeddings diverge beyond a threshold.
from langchain_experimental.text_splitter import SemanticChunker
from langchain_openai.embeddings import OpenAIEmbeddings
splitter = SemanticChunker(
 OpenAIEmbeddings(),
 breakpoint_threshold_type="percentile",
 breakpoint_threshold_amount=90,
)
chunks = splitter.split_text(document)
Document-structure-aware chunking:
For structured documents (Markdown, HTML, PDFs with headers), respect the document structure:
# Split Markdown at header boundaries
from langchain.text_splitter import MarkdownHeaderTextSplitter
headers_to_split_on = [("#", "H1"), ("##", "H2"), ("###", "H3")]
splitter = MarkdownHeaderTextSplitter(headers_to_split_on=headers_to_split_on)
chunks = splitter.split_text(markdown_document)
Each chunk retains its header context, which dramatically improves retrieval relevance.
Parent-child chunking:
Store small chunks for retrieval precision, but return the larger parent context to the LLM:
Small chunks (128 tokens) are indexed for fine-grained retrieval
When a small chunk is retrieved, return its parent section (512 tokens) to the LLM
This gets you the best of both: precise retrieval, sufficient context.
Failure Mode 2: Embedding Model Mismatch
Your embeddings are only as good as your embedding model. Common mistakes:
Using a generic model for specialized content. An embedding model trained on web text performs poorly on code, legal documents, or scientific papers. Match the model to your domain.
Mixing embedding models. If you re-index a portion of your documents with a new model, the vectors are not comparable. Old chunks from model A and new chunks from model B will have incorrect similarity scores. Re-embed everything when you change models.
Ignoring embedding dimensions. Larger embedding dimensions (1536 vs 384) capture more nuance but cost more. For most production use cases, text-embedding-3-small (1536 dims, reduced to 512 works fine) is a good default.
Model Recommendations (2026)
Use Case
 Model
 
General text
 text-embedding-3-small (OpenAI) or nomic-embed-text (open source)
 
Code
 voyage-code-3 (Voyage AI)
 
Multilingual
 multilingual-e5-large or paraphrase-multilingual-mpnet-base-v2
 
Domain-specific
 Fine-tune on your domain data
 
Failure Mode 3: Poor Retrieval Recall
The retriever doesn’t find the relevant chunk. Common causes and fixes:
Semantic search alone isn’t enough
Vector similarity search finds semantically similar content but misses exact term matches. A user asking about “RFC 7519” needs keyword search, not semantic search.
Hybrid search (semantic + keyword) is now the production standard:
from pinecone_text.sparse import BM25Encoder
# Sparse (BM25 keyword) + dense (embedding) search
sparse_encoder = BM25Encoder().default()
sparse_vector = sparse_encoder.encode_queries(query)
dense_vector = embed_query(query)
# Query with both vectors (Pinecone supports this natively)
results = index.query(
 vector=dense_vector,
 sparse_vector=sparse_vector,
 top_k=20,
)
Most production vector databases (Pinecone, Weaviate, Elasticsearch with vector support) support hybrid search.
Retrieve more, then rerank
Instead of retrieving top-5 and using them directly, retrieve top-20 and use a reranking model to select the best 5:
from cohere import Client
co = Client(api_key)
results = vector_db.query(query_vector, top_k=20)
# Rerank using Cohere's reranker
reranked = co.rerank(
 query=user_query,
 documents=[r.text for r in results],
 model="rerank-v3.5",
 top_n=5,
)
Reranking models (Cohere Rerank, BGE-reranker, Jina Reranker) are cross-encoders that evaluate (query, document) pairs together — much more accurate than embedding similarity alone. Adding reranking is consistently one of the highest-ROI improvements to RAG quality.
Failure Mode 4: Context Window Stuffing
You retrieved 20 chunks, you put them all in the prompt. Problems:
LLMs lose track of information in long contexts (“lost in the middle” effect)
Cost grows linearly with retrieved context
Irrelevant chunks actively harm answer quality
Best practices:
Retrieve top-20, rerank, use top-3 to 5
Measure context relevance — if retrieved chunks have similarity below 0.7, don’t include them (no context is better than bad context)
Order matters — put the most relevant chunk first or last, not buried in the middle
Failure Mode 5: No Grounding Verification
The LLM generates a confident answer that’s not actually supported by the retrieved context. This is silent hallucination.
Add a grounding check:
GROUNDING_PROMPT = """
Given the following retrieved context and the AI's answer:
Context: {context}
Answer: {answer}
Question: Is every factual claim in the answer supported by the context?
Respond: grounded
"""
def check_grounding(context, answer) -> dict:
 result = llm.complete(GROUNDING_PROMPT.format(context=context, answer=answer))
 return json.loads(result)
For high-stakes applications, fail or flag answers that aren’t fully grounded. For others, include a confidence score.
Production Architecture
A mature RAG system in 2026 looks like this:
User Query
 ↓
[Query Transformation] ← rewrite query for retrieval, expand synonyms
 ↓
[Hybrid Retrieval] ← dense + sparse, top-20
 ↓
[Reranking] ← cross-encoder, select top-5
 ↓
[Context Assembly] ← format, truncate, order
 ↓
[LLM Generation] ← with grounded context
 ↓
[Grounding Check] ← verify factual support
 ↓
[Response] → User
Each step is instrumented with traces (LangSmith, Langfuse, or Phoenix) so you can debug when answers go wrong.
Photo by Taylor Vick on Unsplash
Evaluation Framework for RAG
Use RAGAS metrics to measure quality:
Metric
 Measures
 Target
 
Faithfulness
 Is the answer grounded in context?
 > 0.85
 
Answer Relevancy
 Does the answer address the question?
 > 0.80
 
Context Precision
 Are the retrieved chunks relevant?
 > 0.75
 
Context Recall
 Were relevant chunks found?
 > 0.70
 
from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision
result = evaluate(
 dataset=eval_dataset,
 metrics=[faithfulness, answer_relevancy, context_precision],
)
print(result)
Run this against a golden dataset on every RAG pipeline change.
Vector Database Selection
For 2026 production use:
Pinecone — managed, scalable, hybrid search, minimal operational burden. Good default for teams that don’t want to self-manage.
Weaviate — open source, self-hostable, excellent hybrid search, object storage. Good for teams that want control.
pgvector (PostgreSQL) — if you’re already on Postgres and scale requirements are moderate (<10M vectors), staying in-database simplifies architecture enormously. pgvector 0.8+ with HNSW indexing is competitive.
Qdrant — high-performance, written in Rust, great for self-hosting. Strong payload filtering support.
Don’t choose a vector database based on benchmarks alone. Operational maturity, filtering capabilities, and hybrid search support matter more for most production use cases.
Quick Wins
If you have an existing RAG system and want to improve it fast:
Add reranking — Cohere Rerank or BGE-reranker. Biggest bang for effort.
Switch to hybrid search — add BM25 alongside embeddings.
Add chunk overlap — if your current overlap is 0, add 50-100 tokens.
Add a grounding check — detect and flag hallucinations.
Instrument with traces — you can’t debug what you can’t see.
Resources
RAGAS — RAG Evaluation Framework
LangChain Text Splitters
Pinecone Hybrid Search Guide
Cohere Rerank Documentation
pgvector — Vector Search for PostgreSQL
Langfuse — Open Source LLM Engineering Platform

Building Reliable RAG Systems: From Prototype to Production

댓글