Site Logo

Building a Multi-PDF Question-Answering System with LangChain and OpenAI: A Comprehensive Guide

Question-answering (QA) systems that can process multiple PDF documents are invaluable for researchers, businesses, and developers seeking to extract insights from large document sets. By leveraging Retrieval-Augmented Generation (RAG), such systems combine document retrieval with powerful language model generation.

Introduction to Multi-PDF QA and LangChain

A multi-PDF QA system retrieves relevant information from multiple PDF documents and generates accurate, context-aware answers. Unlike traditional chatbots, it relies on a knowledge base of documents, making it ideal for applications like academic research, legal analysis, or technical documentation search. LangChain simplifies this by providing tools for document loading, vector storage, and retrieval-augmented chains.

OpenAI’s API, powering models like gpt-3.5-turbo, drives the generation, while LangChain handles document processing and retrieval. This tutorial assumes basic Python knowledge, with references to LangChain’s getting started guide and OpenAI’s API documentation.

Prerequisites for Building the Multi-PDF QA System

Ensure you have:

pip install langchain openai faiss-cpu pypdf
  • Development Environment: Use a virtual environment, as detailed in LangChain’s environment setup guide.
  • Sample PDFs: Prepare multiple PDF files (e.g., research papers, manuals) in a directory.
  • Basic Python Knowledge: Familiarity with syntax and package installation, with resources in Python’s documentation.

Step 1: Setting Up the Development Environment

Configure your environment by importing libraries and setting the OpenAI API key.

import os
from glob import glob
from langchain.llms import OpenAI
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.vectorstores import FAISS
from langchain.document_loaders import PyPDFLoader
from langchain.chains import RetrievalQA
from langchain.text_splitter import RecursiveCharacterTextSplitter

# Set your OpenAI API key
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"

Replace "your-openai-api-key" with your actual key. Environment variables enhance security, as explained in LangChain’s security and API keys guide. The imported modules are core to the QA system, detailed in LangChain’s core components overview.

Step 2: Loading and Processing Multiple PDFs

Load multiple PDFs from a directory using PyPDFLoader and process them into chunks.

# Load all PDFs from a directory
pdf_files = glob("path/to/pdfs/*.pdf")
documents = []
for pdf_file in pdf_files:
    loader = PyPDFLoader(
        file_path=pdf_file,
        extract_images=False
    )
    documents.extend(loader.load())

# Split documents into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
    length_function=len
)
docs = text_splitter.split_documents(documents)

Key Parameters for PyPDFLoader

  • file_path: Path to a PDF file (e.g., "path/to/pdfs/doc.pdf"). Iterated over multiple files in the loop.
  • extract_images: If True, extracts images as text (requires dependencies). Set to False for text-only extraction.

Key Parameters for RecursiveCharacterTextSplitter

  • chunk_size: Maximum characters per chunk (e.g., 1000). Ensures manageable retrieval units.
  • chunk_overlap: Overlapping characters between chunks (e.g., 200). Preserves context across splits.
  • length_function: Measures text length (default: len). Customizable for specific requirements.

For other document types, see LangChain’s document loaders.

Step 3: Creating Embeddings and Vector Store

Convert document chunks into embeddings and store them in a FAISS vector database.

embeddings = OpenAIEmbeddings(
    model="text-embedding-ada-002",
    chunk_size=1000
)
vectorstore = FAISS.from_documents(
    documents=docs,
    embedding=embeddings,
    distance_strategy="COSINE"
)

Key Parameters for OpenAIEmbeddings

  • model: Embedding model (e.g., text-embedding-ada-002). Determines vector quality and cost.
  • chunk_size: Texts processed per API call (e.g., 1000). Balances speed and API limits.

Key Parameters for FAISS.from_documents

  • documents: List of document chunks to embed.
  • embedding: Embedding model instance (e.g., OpenAIEmbeddings).
  • distance_strategy: Similarity metric (e.g., "COSINE", "L2", "IP"). "COSINE" suits text similarity.

For alternatives, explore Pinecone or Weaviate.

Step 4: Initializing the Language Model

Initialize the OpenAI LLM for response generation.

llm = OpenAI(
    model_name="gpt-3.5-turbo",
    temperature=0.5,
    max_tokens=512,
    top_p=0.9,
    frequency_penalty=0.2,
    presence_penalty=0.1
)

Key Parameters for LLM Initialization

  • model_name: OpenAI model (e.g., gpt-3.5-turbo, gpt-4). gpt-3.5-turbo is fast; gpt-4 excels in reasoning. See OpenAI’s model documentation.
  • temperature (0.0–2.0): Controls randomness. At 0.5, responses are focused yet creative. Lower (e.g., 0.2) for precision; higher (e.g., 1.0) for diversity.
  • max_tokens: Maximum response length (e.g., 512). Adjust for detail; higher values increase costs. See LangChain’s token limit handling.
  • top_p (0.0–1.0): Nucleus sampling. At 0.9, focuses on high-probability tokens.
  • frequency_penalty (–2.0–2.0): Discourages repetition. At 0.2, promotes variety.
  • presence_penalty (–2.0–2.0): Encourages new topics. At 0.1, mildly promotes novelty.

See LangChain’s OpenAI integration guide.

Step 5: Building the RetrievalQA Chain

Create a RetrievalQA chain to combine retrieval and generation.

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(
        search_type="similarity",
        search_kwargs={"k": 5}
    ),
    return_source_documents=True,
    verbose=True
)

Key Parameters for RetrievalQA.from_chain_type

  • llm: The initialized LLM.
  • chain_type: Document processing method (e.g., "stuff", "map_reduce", "refine"). "stuff" combines documents into one prompt.
  • retriever: Retrieval mechanism from the vector store.
  • return_source_documents: If True, includes retrieved documents in output.
  • verbose: If True, logs execution for debugging.

Key Parameters for as_retriever

  • search_type: Retrieval method (e.g., "similarity", "mmr"). "similarity" prioritizes closest matches.
  • search_kwargs: Settings, e.g., {"k": 5} retrieves 5 documents.

See LangChain’s RetrievalQA chain guide.

Step 6: Querying the Multi-PDF QA System

Test the system by querying across multiple PDFs.

query = "What are the common themes across the documents?"
response = qa_chain({"query": query})
print(response["result"])
print("Source Documents:", [doc.metadata for doc in response["source_documents"]])

Example Output:

Common themes include sustainability, technological innovation, and collaborative frameworks, as discussed across the documents.
Source Documents: [
    {'page': 3, 'source': 'doc1.pdf'},
    {'page': 10, 'source': 'doc2.pdf'},
    {'page': 7, 'source': 'doc3.pdf'}
]

The chain retrieves relevant chunks from multiple PDFs and generates a cohesive answer. See LangChain’s document QA chain or conversational flows.

Step 7: Customizing the QA System

Enhance the system with custom prompts, additional data sources, or tool integration.

7.1 Custom Prompt Engineering

Modify the prompt for specific response styles.

from langchain.prompts import PromptTemplate

custom_prompt = PromptTemplate(
    input_variables=["context", "question"],
    template="You are a research assistant. Provide a detailed, academic-style answer based on the following context:\n\n{context}\n\nQuestion: {question}\n\nAnswer: ",
    validate_template=True
)

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=vectorstore.as_retriever(search_kwargs={"k": 5}),
    return_source_documents=True,
    verbose=True,
    prompt=custom_prompt
)

PromptTemplate Parameters:

  • input_variables: Variables in the template (e.g., ["context", "question"]).
  • template: Defines prompt structure and tone.
  • validate_template: If True, validates variable usage.

See LangChain’s prompt templates guide.

7.2 Adding Diverse Data Sources

Incorporate text or web content using LangChain’s document loaders.

from langchain.document_loaders import WebBaseLoader

web_loader = WebBaseLoader(
    web_path="https://example.com/research",
    verify_ssl=True
)
web_docs = web_loader.load()
all_docs = docs + text_splitter.split_documents(web_docs)
vectorstore = FAISS.from_documents(all_docs, embeddings)

WebBaseLoader Parameters:

  • web_path: URL to load.
  • verify_ssl: If True, enforces SSL verification.

7.3 Tool Integration with Agents

Add tools like SerpAPI for real-time data.

from langchain.agents import initialize_agent, Tool
from langchain.tools import SerpAPIWrapper

search = SerpAPIWrapper()
tools = [
    Tool(
        name="Search",
        func=search.run,
        description="Fetch current research trends."
    )
]

agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent="zero-shot-react-description",
    verbose=True,
    max_iterations=3,
    early_stopping_method="force"
)

response = agent.run("What are recent trends related to the documents’ themes?")
print(response)

initialize_agent Parameters:

  • tools: List of tools.
  • llm: The LLM for the agent.
  • agent: Agent type (e.g., "zero-shot-react-description").
  • verbose: If True, logs decisions.
  • max_iterations: Limits reasoning steps (e.g., 3).
  • early_stopping_method: Stops execution (e.g., "force") at limit.

See LangChain’s agents guide.

Step 8: Deploying the QA System

Deploy as a web application using Flask.

from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route("/query", methods=["POST"])
def query():
    try:
        user_query = request.json.get("query")
        if not user_query:
            return jsonify({"error": "No query provided"}), 400
        response = qa_chain({"query": user_query})
        return jsonify({
            "answer": response["result"],
            "sources": [doc.metadata for doc in response["source_documents"]]
        })
    except Exception as e:
        return jsonify({"error": str(e)}), 500

if __name__ == "__main__":
    app.run(debug=True, host="0.0.0.0", port=5000)

Save as app.py, install Flask (pip install flask), and run. Send POST requests to http://localhost:5000/query with JSON like {"query": "What are the common themes?"}. See LangChain’s Flask API tutorial. For production, use FastAPI or AWS.

Step 9: Evaluating and Testing the QA System

Evaluate responses using LangChain’s evaluation metrics.

from langchain.evaluation import load_evaluator

evaluator = load_evaluator(
    "qa",
    criteria=["correctness", "relevance"]
)
result = evaluator.evaluate_strings(
    prediction="The documents discuss sustainability and innovation.",
    input="What are the main themes?",
    reference="The documents focus on sustainability, innovation, and collaboration."
)
print(result)

load_evaluator Parameters:

  • evaluator_type: Metric type (e.g., "qa"). Others include "string_distance".
  • criteria: Evaluation criteria (e.g., ["correctness", "relevance"]).

For human-in-the-loop testing, see LangChain’s evaluation guide. Debug with LangSmith per LangChain’s LangSmith intro.

Advanced Features and Next Steps

Enhance the system with:

See LangChain’s startup examples or GitHub repos.

Conclusion

Building a multi-PDF QA system with LangChain and OpenAI enables precise, context-aware answers from diverse documents. This guide covered setup, document processing, vector storage, LLM integration, deployment, evaluation, and key parameters. Leverage LangChain’s chains, vector stores, and integrations to create robust QA systems.

Explore agents, tools, or evaluation metrics. Debug with LangSmith. Happy coding!