Site Logo

LangSmith Intro: Your Toolkit for Debugging and Optimizing LangChain Apps

Building AI apps with LangChain is exciting—whether it’s a chatbot answering questions, a system summarizing documents, or an agent fetching live data. But when things go wrong, or you want to make your app faster and smarter, figuring out what’s happening under the hood can feel like a puzzle. That’s where LangSmith comes in, acting like a super-powered magnifying glass for your LangChain workflows. It’s a platform that helps you trace, debug, evaluate, and optimize your apps with ease, giving you insights into every step of the process.

In this guide, part of the LangChain Fundamentals series, I’ll walk you through what LangSmith is, how it works, and why it’s a must-have for your AI projects. We’ll dive into a hands-on example to show it in action, keeping things clear and practical for beginners and developers alike. By the end, you’ll be ready to use LangSmith to level up your chatbots, document search engines, or customer support bots. Let’s get started!

What’s LangSmith All About?

LangSmith is a platform designed to help you understand, debug, and improve your LangChain applications. It’s like a control room for your AI workflows, offering tools to trace every step, log data, evaluate performance, and fine-tune your app. Built by the LangChain team, it integrates seamlessly with LangChain’s core components—prompts, chains, agents, memory, tools, and document loaders—and works with LLMs from providers like OpenAI or HuggingFace.

Think of LangSmith as your app’s personal detective, helping you:

  • Trace Workflows: See exactly what’s happening in your RetrievalQA chain or agent—from input to output.
  • Debug Issues: Spot errors, like a misfired prompt or slow retrieval, without digging through logs.
  • Evaluate Performance: Measure token usage, latency, or response quality to make your app faster and better.
  • Optimize Prompts: Test and refine prompts to get the best LLM outputs.

Whether you’re building a chatbot that needs to stay sharp or a RAG app that’s too slow, LangSmith gives you the tools to make it shine. It’s a key part of creating enterprise-ready applications and workflow design patterns. Want to see how it fits into LangChain? Check the architecture overview or Getting Started.

How LangSmith Powers Up Your Workflow

LangSmith works by connecting to your LangChain app and collecting detailed data about its execution, which you can view and analyze through a user-friendly dashboard. It integrates with LangChain’s LCEL (LangChain Expression Language), capturing events from chains, agents, and other components, supporting both synchronous and asynchronous execution, as covered in performance tuning. Here’s the flow:

  • Set Up Tracing: Enable LangSmith in your app with a few lines of code, linking it to your LangChain workflow.
  • Capture Data: LangSmith logs every step—prompt inputs, LLM calls, tool usage, retrievals, and errors—along with metrics like token counts and latency.
  • View Traces: Use the LangSmith dashboard to see a visual timeline of your workflow, showing what happened and where.
  • Evaluate and Test: Run tests to compare outputs, measure response quality, or check performance metrics.
  • Optimize: Use insights to tweak prompts, adjust retrieval, or streamline workflows.

For example, in a RetrievalQA Chain, LangSmith can show you how long it takes to fetch documents from a vector store, how many tokens the LLM used, and whether the output matches expectations. This makes LangSmith invaluable for:

Key features include:

  • Detailed Traces: See every step of your workflow, from input to output.
  • Performance Metrics: Track token usage, latency, and costs for optimization.
  • Evaluation Tools: Compare outputs, score responses, or run automated tests.
  • Collaboration: Share traces and insights with your team for faster debugging.

LangSmith is your go-to for making sense of complex workflows and ensuring your app runs like a well-oiled machine.

Exploring LangSmith’s Core Features

LangSmith offers a suite of tools to help you debug, evaluate, and optimize your LangChain apps. Below, we’ll break down the main features, how they work, and when to use them, with examples to make it practical.

Tracing: See Every Step of Your Workflow

Tracing is LangSmith’s core feature, letting you follow the exact path of your app’s execution, from user input to final output. It captures every event—prompts, LLM calls, tool usage, retrievals, and errors—showing you a visual timeline in the dashboard.

  • What It Does: Logs detailed data about each step, including inputs, outputs, and metadata like token counts and latency.
  • Best For: Debugging errors in chatbots, identifying slow steps in RAG apps, or understanding agent decisions.
  • Mechanics: Enable tracing with a callback handler, and LangSmith uploads data to the dashboard for visualization.
  • Setup: Add the LangSmith callback to your chain. Example:
from langchain.callbacks import LangSmithCallbackHandler
from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate

# Simple chain
prompt = PromptTemplate(input_variables=["query"], template="Answer: {query}")
llm = ChatOpenAI(model="gpt-4o-mini")
chain = prompt | llm

# Run with LangSmith tracing
handler = LangSmithCallbackHandler()
result = chain.invoke({"query": "What is AI?"}, config={"callbacks": [handler]})
print(result.content)

LangSmith Dashboard Output: You’d see a trace showing the chain’s execution, with details like “Prompt input: Answer: What is AI?”, “LLM took 0.5s, used 100 prompt tokens, 20 completion tokens”, and the final output.

  • Example: Your chatbot is giving odd answers. LangSmith’s trace shows the prompt is missing context, so you tweak it with few-shot prompting.

Tracing is like having x-ray vision for your app, pinpointing exactly where things go right or wrong.

Logging: Capture Data for Analysis

LangSmith logs every interaction—inputs, outputs, errors, and metrics—for later analysis, making it easy to review past runs or audit your app.

  • What It Does: Saves detailed event data, including timestamps, token usage, and error stacks, to the LangSmith platform.
  • Best For: Auditing customer support bots, reviewing SQL query generation, or analyzing web research for compliance.
  • Mechanics: Automatically logs data when tracing is enabled, storing it for dashboard access or export.
  • Setup: Same as tracing—use the LangSmith callback. Logs are stored in the dashboard alongside traces.
  • Example: A CRM bot logs all user interactions, letting you review a specific session where a customer reported an issue, pinpointing an error in the prompt.

Logging ensures you have a record to fall back on, especially in production.

Evaluation: Measure and Improve Quality

LangSmith’s evaluation tools let you test and score your app’s outputs, helping you measure response quality, accuracy, or performance. You can run automated tests or manually review runs.

  • What It Does: Compares outputs against expected results, scores responses, or measures metrics like token usage and latency.
  • Best For: Testing prompt quality in chatbots, evaluating RAG app accuracy, or benchmarking multi-PDF QA performance.
  • Mechanics: Define test cases, run them through LangSmith, and view results in the dashboard, with scores or pass/fail metrics.
  • Setup: Use LangSmith’s evaluation API or dashboard. Example:
from langchain import hub
from langchain_openai import ChatOpenAI
from langsmith import Client

# Pull a prompt from the Hub
prompt = hub.pull("prompts/conversational-agent")
llm = ChatOpenAI(model="gpt-4o-mini")
chain = prompt | llm

# Set up LangSmith client
client = Client()

# Define test case
test_case = {"input": "What is AI?", "expected_output": "AI is the development of systems..."}
result = chain.invoke(test_case["input"], config={"callbacks": [LangSmithCallbackHandler()]})

# Log test result
client.log_evaluation(test_case["input"], result.content, expected_output=test_case["expected_output"])

LangSmith Dashboard Output: The dashboard shows the test result, scoring how closely the output matches the expected response, with metrics like token usage and latency.

  • Example: You’re testing a chatbot and find that 10% of responses are off. LangSmith’s evaluation highlights a vague prompt, so you refine it with prompt validation.

Evaluation helps you ensure your app is delivering top-notch results.

Hands-On: Debugging a Document QA System with LangSmith

Let’s build a question-answering system that loads a PDF, uses a RetrievalQA Chain with a prompt from the LangChain Hub, and uses LangSmith to trace and evaluate the workflow, returning structured JSON.

Get Your Environment Ready

Follow Environment Setup to prepare your system. Install the required packages:

pip install langchain langchain-openai langchain-community faiss-cpu pypdf langsmith

Set your OpenAI API key and LangSmith API key securely, as outlined in security and API key management. Assume you have a PDF named “policy.pdf” (e.g., a company handbook).

Load the PDF Document

Use PyPDFLoader to load the PDF:

from langchain_community.document_loaders import PyPDFLoader

loader = PyPDFLoader("policy.pdf")
documents = loader.load()

This creates Document objects with page_content (text) and metadata (e.g., {"source": "policy.pdf", "page": 1}).

Set Up a Vector Store

Store the documents in a FAISS vector store:

from langchain_openai import OpenAIEmbeddings
from langchain.vectorstores import FAISS

embeddings = OpenAIEmbeddings()
vector_store = FAISS.from_documents(documents, embeddings)

Pull a Prompt from the LangChain Hub

Grab a RetrievalQA prompt from the Hub:

from langchain import hub

prompt = hub.pull("prompts/retrieval-qa")

This pulls a pre-built prompt optimized for question-answering with retrieved context.

Set Up an Output Parser

Use an Output Parser for structured JSON:

from langchain_core.output_parsers import StructuredOutputParser, ResponseSchema

schemas = [
    ResponseSchema(name="answer", description="The response to the question", type="string")
]
parser = StructuredOutputParser.from_response_schemas(schemas)

Build the RetrievalQA Chain with LangSmith Tracing

Combine components into a RetrievalQA Chain with LangSmith tracing:

from langchain_openai import ChatOpenAI
from langchain.chains import RetrievalQA
from langchain.callbacks import LangSmithCallbackHandler

# Customize the prompt with parser instructions
prompt = PromptTemplate(
    template=prompt.template + "\n{format_instructions}",
    input_variables=prompt.input_variables,
    partial_variables={"format_instructions": parser.get_format_instructions()}
)

# Build the chain
chain = RetrievalQA.from_chain_type(
    llm=ChatOpenAI(model="gpt-4o-mini"),
    chain_type="stuff",
    retriever=vector_store.as_retriever(),
    chain_type_kwargs={"prompt": prompt},
    output_parser=parser,
    callbacks=[LangSmithCallbackHandler()]
)

Test the System

Run a question to test the chain and trace with LangSmith:

result = chain.invoke({"query": "What is the company’s vacation policy?"})
print(result)

Sample Output:

{'answer': 'Employees receive 15 vacation days annually.'}

In the LangSmith dashboard, you’ll see a trace showing:

  • The input query: “What is the company’s vacation policy?”
  • Retrieval step: Number of documents fetched, time taken (e.g., “3 documents, 0.2s”).
  • LLM call: Prompt input, token usage (e.g., “150 prompt tokens, 20 completion tokens”), and latency.
  • Final output: The structured JSON response.

Evaluate the Output

Use LangSmith to evaluate the response quality:

from langsmith import Client

client = Client()
test_case = {
    "input": {"query": "What is the company’s vacation policy?"},
    "expected_output": "{'answer': 'Employees receive 15 vacation days annually.'}"
}
client.log_evaluation(
    input=test_case["input"],
    output=str(result),
    expected_output=test_case["expected_output"]
)

LangSmith Dashboard Output: The dashboard shows the evaluation result, scoring how closely the output matches the expected response, with metrics like token usage and latency.

Debug and Improve

If the trace shows issues—say, slow retrieval or vague answers—use LangSmith’s dashboard to pinpoint the problem. For example, if retrieval takes too long, optimize the vector store with metadata filtering. If the answer is off, refine the prompt with few-shot prompting:

prompt = PromptTemplate(
    template=prompt.template + "\nExamples:\nQuestion: What is the dress code? -> {'answer': 'Business casual'}\n{format_instructions}",
    input_variables=prompt.input_variables,
    partial_variables={"format_instructions": parser.get_format_instructions()}
)

For persistent issues, consult troubleshooting. Enhance with memory for conversational flows or deploy as a Flask API.

Tips to Get the Most Out of LangSmith

Here’s how to make LangSmith work for you:

  • Start with Tracing: Enable tracing early to catch issues during development, saving time on manual debugging.
  • Focus on Key Metrics: Track token usage and latency to optimize costs and speed, especially for RAG apps.
  • Run Regular Evaluations: Use LangSmith’s evaluation tools to test prompt quality and response accuracy, refining with prompt validation.
  • Collaborate: Share traces with your team via the LangSmith dashboard to speed up debugging and optimization.
  • Secure Your Data: Protect sensitive data in traces, following security and API key management.

These tips align with enterprise-ready applications and workflow design patterns.

Keep Building with LangSmith

Ready to take your LangChain skills further? Here are some next steps:

Wrapping It Up: LangSmith Makes Your AI Smarter

LangSmith is your secret weapon for building better LangChain apps, offering tracing, logging, and evaluation tools to debug and optimize with ease. Whether you’re fine-tuning a chatbot, speeding up a RAG app, or ensuring a customer support bot delivers spot-on answers, LangSmith gives you the insights to make it happen. Start with the document QA example, explore tutorials like Build a Chatbot or Create RAG App, and share your creations with the AI Developer Community or on X with #LangChainTutorial. For more, visit the LangChain Documentation and keep building awesome AI!