LLM Frameworks

LangChain

LangChain is a general-purpose orchestration framework for building LLM applications: chaining prompts, models, tools, memory, and output parsers into a pipeline. It popularized the "chain" abstraction and now also ships LCEL (LangChain Expression Language) for composing components declaratively.

What you can actually build

• A retrieval-augmented question-answering chain: retriever → prompt template → LLM → output parser
• Tool-using agents that call APIs, run code, or query a database based on the model's decisions
• Standardized integrations across model providers, vector stores, and document loaders without rewriting glue code per provider

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser

prompt = ChatPromptTemplate.from_template(
    "Answer using only this context:\n{context}\n\nQuestion: {question}"
)
model = ChatOpenAI(model="gpt-4o-mini")
chain = prompt | model | StrOutputParser()

answer = chain.invoke({"context": retrieved_docs, "question": user_query})

LangGraph

LangGraph extends the LangChain ecosystem with a graph-based execution model purpose-built for agents: nodes are steps (often LLM calls or tool calls), edges define transitions, and the graph can branch, loop, and persist state across steps — including pausing for human approval mid-execution.

What you can actually build

• A multi-step agent that retrieves, evaluates whether the answer is sufficient, and loops back to retrieve again if not — a cycle a plain linear chain can't express cleanly
• Human-in-the-loop workflows where the graph pauses and waits for approval before a sensitive action (e.g. sending an email, executing a transaction)
• Multi-agent systems where different nodes represent different specialized agents collaborating on a task
• Durable execution — state checkpoints so a long-running workflow can resume after a failure

from langgraph.graph import StateGraph, END

def retrieve(state): ...
def grade_docs(state): ...
def generate(state): ...

graph = StateGraph(AgentState)
graph.add_node("retrieve", retrieve)
graph.add_node("grade", grade_docs)
graph.add_node("generate", generate)

graph.add_edge("retrieve", "grade")
graph.add_conditional_edges(
    "grade",
    lambda s: "generate" if s["relevant"] else "retrieve"
)
graph.add_edge("generate", END)

app = graph.compile()

LlamaIndex

LlamaIndex is a data framework focused on the ingestion, indexing, and retrieval side of RAG. Where LangChain is broad orchestration, LlamaIndex goes deep on how to structure and query your own data: chunking strategies, multiple index types, and a large library of data connectors.

What you can actually build

• A document ingestion pipeline pulling from PDFs, databases, Slack, Notion, etc. via built-in connectors (LlamaHub)
• Multiple index strategies beyond flat vector search — tree indexes for hierarchical summarization, keyword indexes, knowledge graph indexes
• Query engines with automatic query routing — deciding which index or retrieval strategy fits a given question
• Sub-question decomposition: breaking one complex query into several smaller retrievals and synthesizing the results

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)

query_engine = index.as_query_engine(similarity_top_k=5)
response = query_engine.query("What are the EDI 277CA status codes?")

Ollama

Ollama packages open-source LLMs (Llama, Mistral, Gemma, and others) for easy local execution. It exposes a local REST API compatible-ish with common conventions, handles model downloading/quantization, and removes the need to send data to a third-party API for inference.

What you can actually build

• A fully local RAG pipeline with no data leaving your machine or network — relevant for HIPAA-sensitive prototyping
• Cost-free development and testing loops without burning API credits
• On-prem or air-gapped deployments where cloud LLM APIs aren't an option
• Easy local model swapping to compare outputs across open-source models before committing to a cloud provider

# pull and run a model locally
ollama pull llama3.1
ollama run llama3.1

# call it like an API from your application
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Summarize this EDI 277CA response."
}'

Ollama and local-model details change quickly; verify current model list, performance, and licensing against Ollama's own docs before relying on specifics in an interview.

Tool Calling & Function Calling

Tool calling lets an LLM decide at runtime to invoke an external function rather than generating a text answer directly. The model outputs a structured request — call lookup_claim_status with claim_id=CLM123 — your code executes it, feeds the result back, and the model uses it to generate a final answer. This is the bridge between an LLM and the real world.

LangChain tool definition

from langchain_core.tools import tool
from langchain_openai import ChatOpenAI

@tool
def lookup_claim_status(claim_id: str) -> dict:
    """Look up the current status of an insurance claim."""
    return db.query("SELECT status FROM claims WHERE id=%s", claim_id)

llm = ChatOpenAI(model="gpt-4o-mini")
llm_with_tools = llm.bind_tools([lookup_claim_status])

# Model decides which tool to call
response = llm_with_tools.invoke("What is the status of claim CLM-9821?")

Structured outputs with Pydantic

from pydantic import BaseModel

class ClaimExtraction(BaseModel):
    claim_id: str
    patient_name: str
    diagnosis_code: str
    amount: float

structured_llm = ChatOpenAI(model="gpt-4o-mini").with_structured_output(ClaimExtraction)
result = structured_llm.invoke(
    "Extract: Patient John Doe, claim CLM-991, $2400, ICD-10 Z00.00"
)
# result.claim_id == "CLM-991", result.amount == 2400.0

Tool call vs structured output — when to use which

Memory & State Management

LLMs are stateless by default — each API call has no knowledge of previous ones. Memory in LLM frameworks is the engineering work of deciding what context to maintain across turns, how to store it, and how much to inject into each prompt. The wrong strategy either wastes tokens on irrelevant history or loses critical context mid-conversation.

The four memory types

from langchain.memory import ConversationBufferMemory
memory = ConversationBufferMemory(return_messages=True)

from langchain.memory import ConversationBufferWindowMemory
memory = ConversationBufferWindowMemory(k=5, return_messages=True)

from langchain.memory import ConversationSummaryMemory
memory = ConversationSummaryMemory(llm=ChatOpenAI())

from langchain.memory import VectorStoreRetrieverMemory
memory = VectorStoreRetrieverMemory(
    retriever=vectorstore.as_retriever(k=4))

Memory comparison

LangGraph persistent state (checkpointing)

from langgraph.graph import StateGraph
from langgraph.checkpoint.sqlite import SqliteSaver
from typing import TypedDict, List

class AgentState(TypedDict):
    messages: List
    claim_id: str
    retrieval_count: int
    approved: bool

checkpointer = SqliteSaver.from_conn_string(":memory:")
graph = StateGraph(AgentState)
app = graph.compile(checkpointer=checkpointer)

# Resume a previous run using its thread_id
config = {"configurable": {"thread_id": "session-42"}}
result = app.invoke({"messages": [...]}, config=config)

Debugging & Evaluation

LLM framework bugs are different from regular software bugs — the failure is usually not an exception but a subtly wrong answer. Debugging requires visibility into every step: what prompt was sent, what tool was called, what context was injected, and where reasoning went off-track.

LangSmith tracing — one-line setup

import os
os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"]     = "lsv2_your_key"
os.environ["LANGCHAIN_PROJECT"]     = "eaize-rag-prod"

# All LangChain and LangGraph calls now trace automatically.
# Each trace shows: prompt sent, LLM response, tool calls,
# retrieved context, latency and token cost per step.

Trace symptom → diagnosis table

DeepEval CI integration

from deepeval import evaluate
from deepeval.metrics import AnswerRelevancyMetric, FaithfulnessMetric
from deepeval.test_case import LLMTestCase

test_cases = [
    LLMTestCase(
        input="What is the 277CA transaction?",
        actual_output=chain.invoke("What is the 277CA transaction?"),
        retrieval_context=retrieved_docs
    )
]
# Run in CI — fails build if scores drop below threshold
evaluate(test_cases, [
    AnswerRelevancyMetric(threshold=0.75),
    FaithfulnessMetric(threshold=0.80)
])

Production-Level Interview Q&A

Click a question to expand the answer.