Granite-Docling VLM Pipeline in docling-serve (Tested)

Granite-Docling is IBM’s 258M-parameter vision language model that reads a whole PDF page image and writes it out as structured DocTags, which Docling then turns into Markdown, HTML or JSON. In docling-serve 1.32.0 you turn it on per request with pipeline=vlm and vlm_pipeline_preset=granite_docling, after downloading its weights. In our tests on four single pages it wrote equations as LaTeX and kept code line breaks, which the default pipeline does not. It also misread a dense numeric table that the standard pipeline got almost entirely right, and it took 3 to 15 seconds per page on the Apple GPU and 74 to 110 seconds per page on CPU, against 0.2 to 10 seconds for the standard pipeline. SmolDocling, its predecessor, gave no usable output on any of the four pages on the Apple GPU.

Everything below was run locally against docling-serve 1.32.0, the release that pins Docling 2.124. Disclosure: Parsebridge, which publishes this blog, sells a hosted Docling API that runs the standard pipeline.

Granite-Docling and SmolDocling at a glance

Both models output DocTags, a markup that records each element’s type (heading, paragraph, table, formula, code, picture) and its position on the page. The Granite-Docling card says it is meant to be used inside Docling and is “not intended for general image understanding”.

How the VLM pipeline differs from the standard pipeline

The standard pipeline reads the PDF’s text layer, runs a layout model to find headings, paragraphs, tables and pictures, rebuilds tables with TableFormer, and runs OCR on image regions. Layout, tables and OCR each use their own model. Our Docling OCR guide covers how its OCR step works.

The VLM pipeline replaces all of that with one model. Docling renders each page at 2x scale, the model generates DocTags for the whole page (up to 8,192 tokens), and Docling parses the DocTags into the same document object the standard pipeline produces. With the granite_docling preset all text comes from the model’s output and the PDF’s text layer is ignored, so a born-digital PDF is read the same way as a scan.

Enable the VLM pipeline in docling-serve 1.32.0

1. Download the model weights

The docling-serve container images ship only the standard models (layout, TableFormer, picture classifier, RapidOCR and EasyOCR). The server does not download missing models at runtime when DOCLING_SERVE_ARTIFACTS_PATH is set, which it is in the images. Download Granite-Docling with docling-tools, which is installed alongside docling-serve:

docling-tools models download -o "$DOCLING_SERVE_ARTIFACTS_PATH" granitedocling
# Apple Silicon only, for the MLX engine:
docling-tools models download -o "$DOCLING_SERVE_ARTIFACTS_PATH" granitedocling_mlx

The download name is granitedocling, without the underscore that the request preset granite_docling has. SmolDocling is smoldocling and smoldocling_mlx. The Transformers weights were 505 MB on disk and the MLX weights 624 MB.

For a container, bake the model into a derived image. We did not build this image; it follows the project’s models guide and the Containerfile, which sets DOCLING_SERVE_ARTIFACTS_PATH:

FROM quay.io/docling-project/docling-serve-cpu:v1.32.0
RUN docling-tools models download -o "$DOCLING_SERVE_ARTIFACTS_PATH" granitedocling

Our Docling Docker guide covers the other options, such as mounting a model directory.

If the weights are missing, the synchronous endpoint answers HTTP 404 with {"detail":"Task result not found. Please wait for a completion status."}. The actual cause only appears in the server log:

Worker 0 failed to process job ...: Model 'ibm-granite/granite-docling-258M' not found in artifacts_path.

2. Send a request with the VLM pipeline

With a file upload:

curl -X POST http://localhost:5001/v1/convert/file \
  -F files=@report.pdf \
  -F pipeline=vlm \
  -F vlm_pipeline_preset=granite_docling \
  -F to_formats=md

With a URL, the same fields go inside options:

curl -X POST http://localhost:5001/v1/convert/source \
  -H "Content-Type: application/json" \
  -d '{
    "sources": [{"kind": "http", "url": "https://arxiv.org/pdf/1706.03762v7"}],
    "options": {
      "pipeline": "vlm",
      "vlm_pipeline_preset": "granite_docling",
      "page_range": [4, 4],
      "to_formats": ["md"]
    }
  }'

pipeline accepts standard (the default), vlm, legacy and asr. vlm_pipeline_preset accepts default, which resolves to the server’s default preset, or a preset ID. Docling 2.124 registers 19 of them, including granite_docling, smoldocling, granite_vision, qwen and several OCR models. The older vlm_pipeline_model=granite_docling field still worked in our run, but the docs mark it deprecated in favor of vlm_pipeline_preset. Add doctags to to_formats if you want the raw model output.

A VLM page takes longer than the standard pipeline, and on CPU a single page came close to the synchronous endpoint’s 120-second limit (DOCLING_SERVE_MAX_SYNC_WAIT). Past that limit the server answers 504 and keeps converting. For anything longer than a page or two on CPU, use the async endpoints. This Python version ran against our local server:

import time

import httpx

BASE = "http://localhost:5001"

with open("table.pdf", "rb") as f:
    task = httpx.post(
        f"{BASE}/v1/convert/file/async",
        files={"files": ("table.pdf", f, "application/pdf")},
        data={"pipeline": "vlm", "vlm_pipeline_preset": "granite_docling", "to_formats": "md"},
    ).json()

while task["task_status"] not in ("success", "failure"):
    time.sleep(2)
    task = httpx.get(f"{BASE}/v1/status/poll/{task['task_id']}").json()

result = httpx.get(f"{BASE}/v1/result/{task['task_id']}").json()
print(result["document"]["md_content"])

3. Server settings that matter

There is no setting that preloads the VLM. DOCLING_SERVE_LOAD_MODELS_AT_BOOT loads the models for the default options, which use the standard pipeline, so Granite-Docling loads on the first VLM request. In our runs that cost almost nothing: with the weights already on local disk, the first VLM request after a fresh server start took 6.5 seconds on the code page, the same as later requests. The full list of variables is in our docling-serve configuration reference.

4. Which engine runs the model

The presets use Docling’s auto_inline engine, which picks the backend when the model loads. We read the selection logic in Docling 2.124 and saw the first two cases in the server log:

• macOS on Apple Silicon: MLX, if the mlx-vlm package is installed and the model has an MLX build. Granite-Docling and SmolDocling both do. docling-serve’s own dependencies install mlx-vlm on Apple Silicon; our install got version 0.6.4.
• CPU, or any machine without a supported GPU: Hugging Face Transformers. By the same logic, this is the path the docling-serve-cpu image takes. Setting DOCLING_DEVICE=cpu on a Mac forces this path too.
• NVIDIA GPU: Transformers on CUDA. vLLM is used only when an engine config sets prefer_vllm, which is off by default.

How we tested

The machine was a Mac mini with an Apple M4 (4 performance and 6 efficiency cores) and 16 GB of RAM, on macOS 26.6 and Python 3.12.15. We installed docling-serve from the v1.32.0 tag with uv sync --frozen --no-dev --all-extras --no-extra flash-attn, the same lockfile and extras the container build uses. That gave Docling 2.124.0, docling-jobkit 3.5.0, PyTorch 2.13.0, Transformers 5.8.1 and mlx-vlm 0.6.4. Models were downloaded into one directory with docling-tools and served with DOCLING_SERVE_ARTIFACTS_PATH pointing at it, one worker, and Hugging Face downloads switched off (HF_HUB_OFFLINE=1).

The four test pages, each a one-page PDF:

• Table: Table 1 from page 31 of the EIA Short-Term Energy Outlook, October 2026, a born-digital page with 15 numeric columns and 22 data rows.
• Scan: our scanned order fixture, an image-only page with a small table.
• Equation: page 4 of Attention Is All You Need, with one display equation, inline math and two figures.
• Code: page 2 of the Docling technical report, with a seven-line Python listing.

We ran each configuration as a separate request to /v1/convert/file: one warm-up request, then two timed ones. Granite-Docling on CPU got one timed request per page, three on the table page. We report the server’s own processing_time field. Client-side times came in 2-second steps because the synchronous endpoint polls for the result. Repeated runs produced byte-identical output.

The default run used the Apple GPU (MPS for the standard models, MLX for the VLMs). The CPU run set DOCLING_DEVICE=cpu and OMP_NUM_THREADS=4, the thread count the container image sets.

The standard pipeline ran at the same speed in the CPU run (9.6, 2.2, 2.1 and 0.3 seconds). Granite-Docling on CPU was 7 to 25 times slower than on the Apple GPU, and 11 to 310 times slower than the standard pipeline. The Docling maintainers’ GPU guide reports 2.0 to 3.8 pages per second for Granite-Docling behind an inference server on NVIDIA cards. We did not test an NVIDIA GPU.

Results on the same pages

Snippets below are verbatim. Long table rows are cut and padding spaces collapsed.

Dense table

The standard pipeline returned a 15-column table with 20 of the 22 data rows correct. Its one error, seen in our earlier tests too, merged the heating and cooling degree-day rows.

Granite-Docling returned the whole page as a single 21-column table, with the title missing and the footnotes as table rows. It got 9 of the 22 rows right once empty cells were ignored, and none with the column layout intact. The two-line row labels confused it. Values moved up a row and some rows mixed numbers from two different lines of the PDF. Here the first four “Electricity” values are the coal consumption figures from the row above:

| Electricity | 118 | 99 | 127 | 108 | 11.41 | 11.17 | 13.42 | 10.98 | ...

It got the two degree-day rows right, the ones the standard pipeline merged:

| U.S. Heating Degree-Days | 2,104 | 436 | 55 | 1,429 | 1,932 | ...
| U.S. Cooling Degree-Days | 53 | 464 | 902 | 120 | 83 | ...

This was the one page where the two engines disagreed. With Transformers on CPU, Granite-Docling kept the title and footnotes as text and got 8 of 22 rows right, with similar row shifts. On the other three pages MLX and CPU output were byte-identical.

Scanned page

Both pipelines read the scan’s text correctly and built the same three-column table. Granite-Docling kept the address lines and the total closer to the page:

## Ship to

Example Research Lab

42 Sample Street

Zurich, Switzerland

...

Total: CHF 42.00

The standard pipeline joined the address into one line (Example Research Lab 42 Sample Street Zurich, Switzerland) and split Total: and CHF 42.00 into two paragraphs. Granite-Docling left out the page’s last line, This PDF contains one raster image and no embedded text., which the standard pipeline kept.

Equation

With default options the standard pipeline leaves a placeholder where the equation was, and writes inline math as plain text (dimension d k):

<!-- formula-not-decoded -->

Granite-Docling wrote the display equation as LaTeX, with each letter of the word “Attention” spaced out:

$$\ A t t e n t i o n ( Q , K , V ) = \text {softmax} ( \frac { Q K ^ { T } } { \sqrt { d _ { k } } } ) V$$

It also turned inline subscripts into math, but the Markdown export escaped the underscore, so they read d$\_{k}$ in the text. It labeled both diagrams Flow chart.

The standard pipeline can decode formulas with do_formula_enrichment=true. In our install its default formula model, CodeFormulaV2, failed on every request with Cannot use apply_chat_template because this processor does not have a chat template. in the server log, and the request still returned success with the placeholder. Pointing the enrichment step at Granite-Docling worked:

curl -X POST http://localhost:5001/v1/convert/file \
  -F files=@paper.pdf \
  -F do_formula_enrichment=true \
  -F do_code_enrichment=true \
  -F code_formula_preset=granite_docling \
  -F to_formats=md

That produced the same LaTeX with the equation number kept (... ) V \quad ( 1 )$$) in 2.8 seconds, while leaving the rest of the page to the standard pipeline. Inline math stayed plain text.

Code listing

The standard pipeline found the code block but put the whole listing on one line:

from docling.document_converter import DocumentConverter source = "https://arxiv.org/pdf/2206.01062" # PDF path or URL converter = DocumentConverter() ...

Granite-Docling kept the line breaks, blank line and indentation:

from docling.document_converter import DocumentConverter

source = "https://arxiv.org/pdf/2206.01062"  # PDF path or URL
converter = DocumentConverter()
result = converter.convert_single(source)

The standard pipeline with code_formula_preset=granite_docling gave the same multi-line block. Granite-Docling’s own output also doubled the list bullets on this page (- · Converts PDF documents ...).

SmolDocling

SmolDocling on MLX was not usable on these pages. On the equation page it wrote Siehe Auswertung von 2020-01-01 87 times. On the code page it wrote ## Here is what you can do with the code. 24 times. On the table page it produced 245 lines of dot leaders. On the scan the request failed: the server log showed bbox[0]<=bbox[2] and the client got the same 404 as for a missing model.

To check whether this was the model or the MLX build, we ran SmolDocling once more with Transformers on CPU. The code page came out mostly right, with the code as plain paragraphs and a typo (PDFFs). The equation page stopped at the equation, losing the rest of the page. Either way, Granite-Docling was the better of the two on every page, which matches the SmolDocling card pointing users to it.

When to use which pipeline

Use the standard pipeline for born-digital PDFs with numeric tables, for long documents, and on CPU-only servers. It was the most accurate on the dense table, and 11 to 310 times faster than Granite-Docling on CPU.

Use the standard pipeline with code_formula_preset=granite_docling and formula and code enrichment on when you need LaTeX equations or multi-line code blocks. It kept the standard table results and added 0.7 to 0.9 seconds per page on our test pages.

Use the VLM pipeline with granite_docling when you have a GPU or Apple Silicon, when inline math matters, or when you want one model for scans and born-digital pages alike. Check its tables against the source before trusting the numbers, and expect it to drop occasional lines.

Skip SmolDocling. Its preset is still in docling-serve, but its successor did better on every page we tried.

On a CPU-only server the VLM pipeline is too slow for anything beyond small batches. At 74 to 110 seconds per page, a 100-page document takes two to three hours on one worker.

Parsebridge

Parsebridge is a hosted Docling API. It runs Docling 2.124 inside docling-serve 1.32.0 with the standard pipeline, TableFormer in accurate mode, and OCR turned on when it detects scanned pages. It does not offer the VLM pipeline. It accepts PDFs up to 50 MB, and new accounts get 50 free pages once. For self-hosting the server instead, see Docling Serve: self-hosting the Docling REST API.

To see the standard pipeline’s output on your own PDF first, the free PDF to Markdown converter converts the first page of a PDF up to 10 MB without an account, five times per hour.

Option names, defaults, preset IDs and engine selection were read from the docling-serve v1.32.0 source and the Docling 2.124.0 package it pins, and every request shown here was run against a local server on October 29, 2026. Model facts are from the Granite-Docling and SmolDocling model cards. The Dockerfile above was not built.