Dev Note

The AI application framework landscape has matured rapidly. Three years ago, developers were stitching together raw API calls. Today, frameworks like LangChain, LlamaIndex, and Haystack handle the heavy lifting — document loading, chunking, embedding, retrieval, memory, and agent loops.
But they’ve also evolved in different directions, making the choice less obvious. This guide cuts through the hype to help you pick the right tool for 2026.
Photo by Steve Johnson on Unsplash
The TL;DR
LangChain: Best for complex agents, tool use, and flexible chain composition
LlamaIndex: Best for advanced RAG, data indexing, and document-heavy applications
Haystack: Best for production pipelines with enterprise requirements and modularity
Now let’s go deeper.
LangChain: The Swiss Army Knife
LangChain started as a simple chain abstraction but has grown into a comprehensive AI application platform. LangChain v0.3 (2025+) cleaned up the API significantly after earlier complexity criticisms.
Core Concepts
LCEL (LangChain Expression Language) makes pipelines composable with the pipe operator:
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_community.vectorstores import Chroma
from langchain_core.runnables import RunnablePassthrough
# Simple RAG chain
def build_rag_chain(docs_path: str):
    # 1. Load and split documents
    from langchain_community.document_loaders import DirectoryLoader
    from langchain_text_splitters import RecursiveCharacterTextSplitter
    
    loader = DirectoryLoader(docs_path, glob="**/*.md")
    docs = loader.load()
    
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200
    )
    splits = splitter.split_documents(docs)
    
    # 2. Create vector store
    embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
    vectorstore = Chroma.from_documents(splits, embeddings)
    retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
    
    # 3. Build chain
    prompt = ChatPromptTemplate.from_template("""
    Answer the question using only the provided context.
    
    Context:
    {context}
    
    Question: {question}
    """)
    
    llm = ChatOpenAI(model="gpt-4o", temperature=0)
    
    chain = (
        {"context": retriever, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
    )
    
    return chain
chain = build_rag_chain("./docs")
response = chain.invoke("What is the deployment process?")
LangGraph: Stateful Agent Loops
LangGraph has become LangChain’s crown jewel for building agents:
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from typing import TypedDict, Annotated
import operator
class AgentState(TypedDict):
    messages: Annotated[list, operator.add]
@tool
def search_web(query: str) -> str:
    """Search the web for current information."""
    # Integration with Tavily, Brave, etc.
    return f"Search results for: {query}"
@tool  
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression."""
    return str(eval(expression))
def build_agent():
    tools = [search_web, calculate]
    llm = ChatOpenAI(model="gpt-4o").bind_tools(tools)
    
    def call_model(state: AgentState):
        response = llm.invoke(state["messages"])
        return {"messages": [response]}
    
    def should_continue(state: AgentState):
        last_message = state["messages"][-1]
        return "continue" if last_message.tool_calls else END
    
    graph = StateGraph(AgentState)
    graph.add_node("agent", call_model)
    graph.add_node("tools", ToolNode(tools))
    
    graph.set_entry_point("agent")
    graph.add_conditional_edges("agent", should_continue, {"continue": "tools", END: END})
    graph.add_edge("tools", "agent")
    
    return graph.compile()
agent = build_agent()
result = agent.invoke({"messages": [("human", "What's 15% of 847, and who invented calculus?")]})
LangSmith: Observability
LangChain’s observability platform is now production-grade:
from langsmith import traceable
@traceable(name="RAG Pipeline")
def answer_question(question: str) -> str:
    # Every step automatically traced in LangSmith
    docs = retriever.invoke(question)
    response = chain.invoke({"context": docs, "question": question})
    return response
LlamaIndex: The Data Framework
LlamaIndex (formerly GPT Index) took a different path. Where LangChain generalized, LlamaIndex specialized in data connectivity and retrieval. If your app is primarily about querying documents, databases, or APIs with LLMs, LlamaIndex’s abstractions are more natural.
Multi-Document RAG
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.core.node_parser import SentenceWindowNodeParser
from llama_index.core.postprocessor import MetadataReplacementPostProcessor
# Configure globally
Settings.llm = OpenAI(model="gpt-4o", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
# Advanced parsing: sentence windows preserve context
node_parser = SentenceWindowNodeParser.from_defaults(
    window_size=3,
    window_metadata_key="window",
    original_text_metadata_key="original_text"
)
# Load documents
documents = SimpleDirectoryReader("./knowledge_base").load_data()
# Build index with sentence window parsing
index = VectorStoreIndex.from_documents(
    documents,
    transformations=[node_parser]
)
# Query with metadata replacement (retrieves surrounding context)
query_engine = index.as_query_engine(
    similarity_top_k=5,
    node_postprocessors=[
        MetadataReplacementPostProcessor(target_metadata_key="window")
    ]
)
response = query_engine.query(
    "What are the main performance optimization strategies?"
)
print(response.source_nodes[0].score)  # Relevance scores included
Agentic RAG with RouterQueryEngine
from llama_index.core.query_engine import RouterQueryEngine
from llama_index.core.selectors import LLMSingleSelector
from llama_index.core.tools import QueryEngineTool
# Multiple specialized indices
code_index = VectorStoreIndex.from_documents(code_docs)
design_index = VectorStoreIndex.from_documents(design_docs)
api_index = VectorStoreIndex.from_documents(api_docs)
# Define tools
code_tool = QueryEngineTool.from_defaults(
    query_engine=code_index.as_query_engine(),
    description="Code examples, programming tutorials, and implementation guides"
)
design_tool = QueryEngineTool.from_defaults(
    query_engine=design_index.as_query_engine(),
    description="Architecture decisions, design patterns, and system design"
)
api_tool = QueryEngineTool.from_defaults(
    query_engine=api_index.as_query_engine(),
    description="API reference, endpoint documentation, and integration guides"
)
# Router automatically selects the right index
router_engine = RouterQueryEngine(
    selector=LLMSingleSelector.from_defaults(),
    query_engine_tools=[code_tool, design_tool, api_tool]
)
response = router_engine.query("How do I authenticate with the REST API?")
# Automatically routes to api_tool
Strengths of LlamaIndex
Best-in-class retrieval quality (sentence windows, HyDE, reranking)
Excellent data connectors (Google Drive, Notion, Confluence, 150+ sources)
Property Graph Index for complex relationship queries
llama_cloud for production-grade parsing (PDFs, tables, images)
Haystack: The Production-Grade Pipeline
Haystack (by deepset) is the most enterprise-focused of the three. It’s designed around composable, type-safe pipelines that are easy to test, deploy, and monitor.
from haystack import Pipeline, Document
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
# Build document store
document_store = InMemoryDocumentStore()
document_store.write_documents([
    Document(content="Python is a high-level programming language.", meta={"source": "wiki"}),
    Document(content="FastAPI is a modern web framework for building APIs with Python.", meta={"source": "docs"}),
])
# Define prompt template
template = """
Given the following context, answer the question.
Context:
Question: 
"""
# Build pipeline
pipeline = Pipeline()
pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipeline.add_component("prompt_builder", PromptBuilder(template=template))
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))
# Connect components
pipeline.connect("retriever", "prompt_builder.documents")
pipeline.connect("prompt_builder", "llm")
# Run
result = pipeline.run({
    "retriever": {"query": "What is FastAPI?"},
    "prompt_builder": {"question": "What is FastAPI?"}
})
print(result["llm"]["replies"][0])
Pipeline Serialization and Versioning
Haystack pipelines can be serialized to YAML — critical for reproducibility:
import yaml
# Save pipeline
with open("rag_pipeline.yaml", "w") as f:
    yaml.dump(pipeline.to_dict(), f, default_flow_style=False)
# Load and run later
loaded_pipeline = Pipeline.from_dict(yaml.safe_load(open("rag_pipeline.yaml")))
This makes A/B testing pipelines, version control, and deployment trivial.
Feature Matrix
Feature
      LangChain
      LlamaIndex
      Haystack
    
RAG quality
      Good
      Excellent
      Good
    
Agent support
      Excellent (LangGraph)
      Good
      Good
    
Learning curve
      Medium
      Medium
      Low
    
Observability
      LangSmith (paid)
      LlamaCloud
      Built-in tracing
    
Pipeline serialization
      Partial
      Partial
      YAML native
    
Data connectors
      Many
      Most
      Good
    
Enterprise support
      Yes
      Yes
      Yes (deepset)
    
Community size
      Largest
      Large
      Medium
    
Choosing in 2026
Start with LangChain if:
Building agents with complex tool use (LangGraph is superb)
You want the largest ecosystem and community
Prototyping quickly and exploring patterns
You need multi-modal workflows
Start with LlamaIndex if:
Your app is primarily document Q&A or knowledge base search
Retrieval quality is the top priority
You’re indexing complex data (PDFs with tables, code, structured data)
You need advanced techniques (HyDE, reranking, query planning)
Start with Haystack if:
Enterprise requirements: testability, versioning, reproducibility
Team of engineers (not just ML/AI folks)
Production-first approach from day one
You want clean, serializable pipeline definitions
For most new AI applications in 2026, the practical answer is: try LangChain first. Its community, documentation, and LangGraph for agents are hard to beat. Switch to LlamaIndex if you find retrieval quality isn’t meeting expectations — its advanced RAG features are genuinely superior.
Which AI framework are you building with? Share your experience below!

LangChain vs LlamaIndex vs Haystack: Choosing Your AI Framework in 2026

댓글