Building a Claude API Summarization Pipeline for PDFs

Summarizing PDFs with the Claude API requires more than a single prompt. You need a pipeline that extracts text reliably, handles documents larger than a single context window, produces consistent summary structure, and scales across hundreds of files without manual review. This article walks through a working pipeline you can adapt directly.

The short answer: extract text from the PDF (don't send raw PDF bytes unless you're using a model with native document support), chunk long documents by section or token count, summarize each chunk with a consistent prompt template, then run a final "summary of summaries" pass to produce one coherent output. Below is each stage with code.

Step 1: Extract text from the PDF

Claude works on text, so the first job is getting clean text out of the PDF. For most pipelines, a library like pdf-parse (Node) or pdfplumber (Python) is sufficient. Scanned PDFs need OCR first (Tesseract or a cloud OCR service) before any of this applies.

import fs from "fs";
import pdf from "pdf-parse";

async function extractText(path) {
  const buffer = fs.readFileSync(path);
  const data = await pdf(buffer);
  return data.text;
}

Strip repeated headers/footers and page numbers at this stage — they add noise to every chunk and waste tokens.

Step 2: Chunk the document

Most real-world PDFs — contracts, reports, research papers — exceed what you want to stuff into a single prompt, even with a large context window. Chunking by logical sections (headings, page breaks) gives better summaries than fixed character splits, but a token-based fallback is necessary for unstructured text.

function chunkText(text, maxChars = 12000) {
  const chunks = [];
  let current = "";
  for (const paragraph of text.split("\n\n")) {
    if ((current + paragraph).length > maxChars) {
      chunks.push(current);
      current = "";
    }
    current += paragraph + "\n\n";
  }
  if (current) chunks.push(current);
  return chunks;
}

Adjust maxChars based on your model's context window and how much of it you want to reserve for the response and any system instructions.

Step 3: Summarize each chunk

Use a consistent prompt template so chunk summaries are structurally compatible when you merge them. Ask for a fixed format — bullet points, key entities, or a short paragraph — rather than letting the model choose freely per chunk.

async function summarizeChunk(chunk, index, total) {
  const response = await fetch("https://api.subtoapi.app/v1/messages", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "claude-sonnet-4-5",
      max_tokens: 500,
      messages: [{
        role: "user",
        content: `This is part ${index + 1} of ${total} from a document. Summarize the key points in 4-6 bullet points. Preserve names, dates, and figures exactly.\n\n${chunk}`,
      }],
    }),
  });
  const data = await response.json();
  return data.content[0].text;
}

Running chunk summarization through SubToAPI means each call goes through a standard sub_live_... application key rather than juggling raw Anthropic credentials across your extraction workers — useful once this pipeline runs on a schedule or in a queue.

Step 4: Merge into a final summary

Once every chunk has a bullet-point summary, concatenate them and run one more pass to produce the final output. This step is where you enforce tone, length, and format — executive summary, structured report, or Q&A style.

async function finalSummary(chunkSummaries) {
  const combined = chunkSummaries.join("\n\n");
  const response = await fetch("https://api.subtoapi.app/v1/messages", {
    method: "POST",
    headers: {
      "Authorization": `Bearer ${process.env.SUBTOAPI_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: "claude-sonnet-4-5",
      max_tokens: 800,
      messages: [{
        role: "user",
        content: `Combine these section summaries into one coherent document summary. Remove redundancy, keep it under 300 words, and preserve all specific figures and names.\n\n${combined}`,
      }],
    }),
  });
  const data = await response.json();
  return data.content[0].text;
}

Running it at scale

For a handful of PDFs, sequential processing is fine. For hundreds, two things matter: concurrency control and retry logic. Fire off chunk summarization requests with a concurrency cap (5-10 at once is usually safe), and retry on transient failures with exponential backoff.

async function processPdf(path) {
  const text = await extractText(path);
  const chunks = chunkText(text);
  const summaries = [];

  for (let i = 0; i < chunks.length; i++) {
    const summary = await summarizeChunk(chunks[i], i, chunks.length);
    summaries.push(summary);
  }

  return finalSummary(summaries);
}

Batch multiple PDFs through a worker pool rather than a simple for loop in production — this keeps throughput predictable and lets you cap concurrent requests to stay well under any rate limit.

Where to plug in streaming or tool use

If you're surfacing summaries in a UI as they're generated, use streaming instead of waiting for the full response — this matters most on the final merge step, which tends to be the longest-running call. If your pipeline needs to look up related documents or fetch metadata mid-summary, tool use lets Claude call out to your own functions rather than hardcoding that logic into the prompt.

Why route this through an API layer

A PDF summarization pipeline usually ends up as a background job, a cron task, or a queue consumer — not a user-facing chat interface. That means you want clean separation between "which service called the API" and "how much it cost," especially once multiple teams or products share the same underlying model access. SubToAPI turns your Claude access into a standard HTTPS API with scoped sub_live_... keys per project, so your summarization worker, your extraction service, and any other integration each get their own key and their own usage breakdown instead of one shared credential. Check the quickstart to get a key running in a few minutes, or see pricing if you're scaling this across a team.

questions

Do I need to send the raw PDF file to Claude, or extract text first? Extract text first in most pipelines — it's more reliable, cheaper on tokens, and gives you control over chunking. Native document upload support varies by integration, so text extraction remains the more portable approach for pipelines you control end to end.

How do I handle PDFs with tables and figures? Preserve table structure as plain text (rows separated by newlines, columns by consistent delimiters) before summarization — Claude handles tabular text well if it's not mangled by the extraction step. For charts or scanned figures, OCR or image captioning needs to run separately since summarization only sees text.

What's the best chunk size for summarization accuracy? There's no universal number — it depends on document structure. Chunking by logical section (headings, page breaks) usually outperforms fixed character counts because each chunk stays topically coherent, which produces better individual summaries before the final merge pass.