A dbt Core 1.8.0 + sqlparse dependency drift story

It’s easy to assume that if a Dockerfile hasn’t changed, rebuilding the image should produce identical results.

In practice, that assumption can be dangerous.

This article walks through a realistic dbt Core scenario that demonstrates how rebuilding the same Docker image can introduce breaking changes — even when application code, SQL models, and dbt versions remain untouched.

A Common Setup

Consider a typical production-style setup:

• dbt Core 1.8.0
• dbt executed from a custom Docker image
• SQL models that include:
  • Multiple CTEs
  • Heavy macro usage
  • Large Jinja expansions

Everything runs reliably for months.

Then, one day, the Docker image is rebuilt using the same Dockerfile.

Suddenly, dbt jobs start failing during compilation.

The Failure Mode

The error message looks like this:

Maximum number of tokens exceeded (10000)

The failure occurs during SQL parsing — before the query ever reaches the database.

At first glance, this seems puzzling:

• No SQL changes
• No dbt Core upgrade
• No infrastructure changes

So what caused the behavior to change?

Understanding dbt’s Dependency on sqlparse

dbt Core relies on an external SQL parsing library: sqlparse.

In dbt Core 1.8.0, this dependency is defined using a version range:

sqlparse >= 0.5.0, < 0.6.0

This allows any compatible 0.5.x patch release to be installed when dependencies are resolved.

While this provides flexibility, it also means behavior can change between image builds.

What Changed in sqlparse

In December 2025, a new patch version of sqlparse was released.

This version introduced stricter internal limits as part of defensive parsing improvements, including:

• A hard limit on the number of tokens processed
• The limit set to 10,000 tokens
• An explicit exception raised when the limit is exceeded

Queries that previously parsed successfully could now fail if their compiled size crossed this threshold.

Why Rebuilding the Image Triggers the Issue

Before the rebuild:

• The existing image contained an earlier sqlparse patch version
• Large compiled SQL statements parsed without errors

After the rebuild:

• pip install resolved dependencies again
• A newer sqlparse patch version was pulled
• The same SQL exceeded the new internal token limit

The Dockerfile did not change — but the dependency graph did.

Why dbt Models Are Particularly Sensitive

dbt SQL is rarely static.

Compiled SQL often grows due to:

• Jinja templating
• Macro expansion
• Repeated logic across models

A model that appears reasonable in source form can expand into tens of thousands of tokens after compilation, making it susceptible to parser-level limits.

How This Is Typically Resolved

Short-term mitigation

Pin the SQL parser dependency explicitly:

sqlparse==0.5.4

This avoids pulling in patch releases that introduce stricter parsing behavior.

Long-term best practices

To prevent similar surprises:

1. Pin all dependencies, including transitive ones
2. Commit dependency lock files to version control
3. Treat Docker image rebuilds as upgrades, not routine actions
4. Test rebuilt images before production deployment