Skip to content
PageSaver

How We Built PDF to Markdown: Tables, Images and Equations

PixelPDF's PDF to Markdown tool converts a PDF entirely in your browser. The file is never uploaded. Getting the text out turned out to be the easy part. The hard part was everything a PDF lays out in two dimensions: tables, charts, equations, even a contents page. Markdown is a stream of lines, and every one of those structures arrives scrambled if you only read the text.

This post walks through what broke when we ran real papers through it, and the passes we added to fix tables, images and equations without making anything else worse.

The starting point: pdf-inspector in WebAssembly

We didn't write a PDF parser. The first version, shipped in August 2026, runs Firecrawl's open-source pdf-inspector compiled to WebAssembly (MIT licensed, about 5 MB). It runs in a worker, reads the document, tells us which pages carry real text and which are scans, and writes the text out as Markdown. Headings, paragraphs and ordinary ruled tables came through well.

One small trap worth passing on: its detectPdf() and processPdf() calls report 1-indexed page numbers, while classifyPdf() reports 0-indexed ones. That is very easy to carry into a page loop by accident, so our adapter normalises everything to 0-indexed before any other code sees it.

Then we fed it academic papers, and the problems showed up in the same places every time.

The parser only gives you text

The runtime returns Markdown and nothing else. No coordinates, no shapes. So anything that has to reason about layout reads the page a second time through pdf.js, walking its operator list to collect three things: text items with their positions, the paths drawn on the page, and the thin horizontal rules among those paths.

That walk had its own surprises. Paths are drawn in local coordinates, so a transform stack has to be carried along to place them on the page. And the path data can't be read as plain x/y pairs: a rectangle is stored as a corner plus a width and height, and reading it as pairs turns the width and height into a phantom point. The step codes have to be walked one by one.

Reading the operator list is also slow. For one page it can cost about as much as the runtime spends on the whole document. So all the passes use one shared geometry reader, and each pass gets a time budget of 8 seconds, so a long PDF gets a bounded amount of extra effort instead of an open-ended stall.

The conversion ended up as a pipeline:

  1. pdf-inspector writes the base Markdown.
  2. OCR fills in scanned pages (optional).
  3. A table pass rebuilds flattened tables.
  4. A figure pass keeps charts as images and turns diagrams into lists.
  5. A contents pass rebuilds the table of contents.
  6. An equation pass rebuilds display formulas as LaTeX.

Every pass after the first only ever improves the Markdown. If one throws, the Markdown it was handed goes through unchanged.

Tables without vertical lines

Tables drawn as full grids were fine. The ones that broke were academic tables in the booktabs style: horizontal rules only, no vertical ones, and prose cells that wrap over two or three lines. The runtime read those as ordinary multi-column text, so the cells interleaved into a single paragraph. In our testing, pdfplumber, PyMuPDF and a newer pdf-inspector release missed the same shape, so swapping libraries wasn't going to fix it.

The table is still there in the geometry, though. The pass rebuilds it in a few steps.

Find the table by its rules. Booktabs draws a top rule, a rule under the header, sometimes group rules, and a bottom rule, all at exactly the same width. Rules that share an x-extent belong to one table; a lone rule bounds nothing and is dropped. With three or more rules, the second one marks the header.

Find the columns by the gaps. A column boundary is a vertical strip that no text ever crosses. One detail cost us a phantom empty first column: the rules overhang the outer columns, so the scan has to be bounded by the text, not by the rules.

Tell a wrapped cell from a new row. Both are just a line break. The difference is the leading: inside a wrapped cell, the step from one baseline to the next is the font's normal line spacing, and between rows it's larger. If every step is the same, nothing wraps and each line is its own row. Wrapped lines are then joined back together, closing up the hyphen the line break introduced (an em dash keeps its space, since it ends a clause, not a word).

There was also an ordering issue. pdf.js often hands a table over column by column, while the runtime flattens it line by line, so the items are sorted into reading order before anything else happens. Cells that contain formulas reuse the equation renderer described below.

The pass is deliberately narrow. It only looks at pages where the runtime reported no table, so a page that already converted correctly is never touched. Body text never has rules sharing an x-extent, so ordinary paragraphs can't match. And a real table fills most of its grid; when the cells don't line up well enough, the original text stays and gets an HTML comment marking it, instead of a guessed table. Our test fixture uses coordinates measured from a real TMLR paper, so the heuristics are checked against an actual layout rather than an idealised one.

Putting the fix back is the hard part

Every pass ends up holding a correct rendering of something, plus a new problem: where did the scrambled version end up in the Markdown, so it can be replaced?

For tables it's manageable. The runtime usually emits the whole region as one paragraph, so the paragraph whose characters match the region is the one to swap. Matching ignores spacing and punctuation, because the flattened text runs cells together and keeps the hyphens from line wrapping. Some output turned every table row into its own block, often a heading, so the splicer also looks for that run by the header and the first cell of each row, and replaces only those blocks so the caption underneath survives.

Diagrams are worse. One paragraph can hold several boxes in an order they were never in, and a short label may also match a heading elsewhere in the document. Two tests handle it together: short blocks must appear as substrings of the rebuilt text, and long blocks match by vocabulary, since a paragraph built entirely from the region's own words came from that region. The region is then located by its densest run of matching blocks, not by the first and last block that happen to match. If none of that resolves, the original text is kept word for word, because collapsing it would delete the only copy.

Images and charts

A chart can't be turned into text. Its axis labels, legend and data points only make sense in two dimensions, and the runtime emits them as a scatter of numbers. So for charts we keep a picture instead.

The signal we use is a numbered caption ("Figure 3") with a substantial drawn path directly above it. That works no matter which tool produced the PDF. The page is rendered locally with pdf.js, cropped to that region, and saved next to the Markdown. You can download a ZIP with the .md file plus the images, or a single .md with the images embedded as data URIs. The caption itself stays as text, so it's still searchable.

The first crops had a bug that took a while to spot: they included the top half of the caption. PDF text coordinates give the baseline, not the top of the letters, and our fixed five-point gap above the caption wasn't enough for a typical 10-point caption. The crop now steps above the caption's full font height, plus a small margin for anti-aliasing, while keeping the chart's axis labels.

Box-and-arrow diagrams, such as the taxonomy figures common in survey papers, get different treatment. The runtime emits their boxes in drawing order, so the hierarchy is lost and labels get shuffled among the leaves. But each box is a drawn shape enclosing its label, and each connector is a line that starts at one box and ends at another. Following the connectors gives the real parent and child links, which turns the figure into a nested Markdown list.

Working out which way the tree grows was trickier than expected. Comparing how far the boxes spread on each axis gets it wrong, because a left-to-right tree still fills the page top to bottom. What works is looking at the step across each single edge: large along the direction of growth, near zero across it. This pass is gated hard too. It needs several labelled boxes over at least two columns, connectors joining nearly all of them, and a single root. A shaded table also looks like a grid of boxes, but its few lines touch only a handful of them, so it isn't even flagged.

Tables of contents had the same flattening problem, with the dot leaders surviving as long runs of periods. An entry is a line ending in a small number pinned to the right margin with a gap or leader run in front of it, and its depth is how far it's indented. That's enough to rebuild a contents page as a nested list.

Equations

A typeset formula is two-dimensional, and reading glyphs in one dimension collapses every level onto the baseline. Equation 1.4 of the scaling laws paper by Kaplan et al. is a good example. It reads:

B_crit(L) = B_* / L^(1/α_B)

Plain text extraction gave us this:

B8 Bcrit(L) = 1*=* ; B 2 10

Every piece is in there, just in the order the typesetter happened to draw it. The levels can still be read from the page, though. A superscript or subscript is set smaller than its base and sits above or below the baseline. A fraction is a thin horizontal rule with content above and below it. That's enough to rebuild the common shapes of display equations as ^{...}, _{...} and \frac{...}{...}, with no machine learning model involved. Multi-letter runs such as "crit" or "tokens" are set upright as names; single letters stay as variables.

A few cases needed care. Numbers printed in a row under a formula are usually its uncertainties, not a subscript, and reading them as one would invent a term, so script glyphs have to sit close together to count. A small fraction inside an exponent, as in (N_c/N)^(α_N/α_D), is fine as long as the two fraction bars sit side by side. If they overlap, they're genuinely nested, and that needs a full math layout tree.

That's where we stopped on purpose. The pass doesn't attempt matrices, multi-column alignment, integrals and sums with limits, radicals, or stretched brackets. Meeting any of those abandons the whole equation and keeps the original text. A formula that looks right but is wrong is worse than an obviously scrambled one, because people copy it into their notes without checking.

Scanned pages

After the first release we read through Reddit threads where people described PDF-to-Markdown conversions going wrong. Tables, equations and figures came up a lot, which was reassuring. The other big one was scanned PDFs: either the converter returns nothing, or it OCRs the whole document, which is slow and can make good native text worse.

So OCR only runs on pages pdf-inspector flagged as scanned, and only after checking that the page's own text layer really is sparse. It uses Tesseract, running locally in WebAssembly, with English, Simplified Chinese, Traditional Chinese and mixed options. Pages are rendered one at a time and each canvas is released straight away, so memory is bounded by one page rather than the whole document. Two-column scans are read column by column, but only when both sides hold a real body of text, so a stray page number can't reorder the page.

Merging OCR pages back in needed page boundaries, so the conversion always adds page markers internally and strips them at the end unless you asked to keep them. When it's done, the tool reports what it did: how many tables and equations it rebuilt, how many figures it kept, which pages went through OCR, and which ones OCR couldn't read.

What we'd do again

Looking back, the decisions that held up were the conservative ones. We kept a good general parser and added narrow passes, one per structure, rather than trying to replace it. Each pass only runs when there's clear evidence the structure is on the page, and silently hands back the original text when it isn't sure or runs out of time. In every case, a confidently wrong table or formula would have been worse than leaving the original text alone.

There are still limits. Merged cells and tables that span two pages aren't rebuilt, charts without a numbered caption may stay as text, and OCR on handwriting or poor scans is hit and miss. But because the tool keeps the original text wherever it isn't sure, you still get readable text there, just without the rebuilt structure.

If you want to try it, the PDF to Markdown tool is free and runs in your browser. For a non-technical walkthrough, including when you can skip conversion and upload the PDF straight to ChatGPT or Claude, see How to Convert PDF to Markdown for ChatGPT and Claude.