Open source · Apache-2.0 · v0.1
Know what a change does to your data before it ships.
bricksentry is a change-safety check for Databricks pipelines and the AI agents that edit them. It diffs the data, maps the blast radius from Unity Catalog lineage, applies your rules, and returns a verdict with the rollback ready.
$ pip install "bricksentry[databricks,mcp]"- PySpark, Jobs, Lakeflow
- MCP server for agents
- Works in CI
$ bricksentry assess sales.orders --pr 42- risk
- high
- safe_to_ship
- false
- base
- 20,000
- candidate
- 20,000
- added
- 0
- removed
- 0
- changed
- 6,666
- dup keys
- 0
6,666 rows changed (33.33%). Most changed: amount. Totals: sum(amount) −2.67%. 100% of affected rows are region = APAC.
- [needs_human]certified-downstream: 1 downstream asset tagged certified: sales.daily_revenue
- [warn]many-rows-changed: changed 33.33% of rows (> 10%)
- [warn]dashboard-downstream: 1 downstream dashboard: Exec Revenue
blast radius
sales.orders ├─ table sales.daily_revenue finance-analytics [certified] │ ├─ table finance.month_close finance-analytics │ └─ genie Revenue Q&A ├─ job orders_hourly └─ dash Exec Revenue
notify: ["finance-analytics", "sales-data-eng"]
next_step Do not merge. Open the pull request with the report and request review from: finance-analytics, sales-data-eng.
01 · The gap
AI writes the pipeline. Nobody checks what it did to the data.
Code review reads code. It can't see a silent 3% revenue shift, a join that duplicates rows, or the certified table three hops downstream.
72% vs 24%
of data teams prioritise AI-assisted coding; only 24% prioritise AI pipeline management.
Source: dbt Labs, State of Analytics Engineering 2026 ↗53%
of data engineering capacity at large enterprises goes to pipeline maintenance.
Source: Fivetran 2026 Enterprise Data Infrastructure Benchmark ↗2024
Datafold archived its open-source data-diff. The pull-request review tools that remain mostly target dbt projects or need a paid platform.
Source: Datafold open-source docs ↗02 · How it works
Four checks, in order, on every change.
Everything runs as SQL on your SQL warehouse. No Spark session, no data leaves your workspace, and it's cheap enough for an agent to call in a loop.
1 / Diff
What changed in the data?
Matches production and your change on key columns; counts added, removed and changed rows, schema changes, column totals and duplicate keys, then finds where the change is concentrated.
2 rows changed · amount / total −6.7% · 100% APAC2 / Blast radius
Who depends on it?
Walks Unity Catalog's runtime lineage, which sees PySpark notebooks, jobs, pipelines, dashboards and queries, and attaches owners and tags.
→ daily_revenue [certified] → Exec Revenue dashboard3 / Policy
Is it allowed?
Your rules in policy.yml decide the verdict. The same change always gets the same answer, because no LLM is involved in the decision.
certified downstream → needs_human / total moved > 5% → warn4 / Rollback
How do I undo it?
Reads Delta history and hands you the exact restore command before anything is applied.
RESTORE TABLE sales.orders TO VERSION AS OF 41
| order_id | region | amount |
|---|---|---|
| 1 | APAC | 100 |
| 2 | EMEA | 200 |
| 3 | APAC | 300 |
| order_id | region | amount |
|---|---|---|
| 1 | APAC | 90 |
| 2 | EMEA | 200 |
| 3 | APAC | 270 |
03 · Verdicts
One answer, four levels.
The worst matching rule wins. CI fails at the level you choose with --fail-on, and agents get a next_step they must follow.
pass
Ship it
no rule fired. Merge or apply.
warn
Ship, with notes
something moved within tolerance. The warnings go into the PR description.
needs_human
Ask an owner
a certified table, PII or a big row loss is involved. Owners from Unity Catalog are tagged for review.
block
Stop and fix
duplicate keys or another hard rule. Rebuild the candidate and check again.
04 · For AI agents
A safety loop your coding agent can call.
Databricks' managed MCP servers let agents read data and run queries. bricksentry adds the step that tells an agent whether its change is safe.
| Tool | Access | What it returns |
|---|---|---|
| whats_downstream | read | Downstream assets with owners and tags. Called before editing. |
| rollback_plan | read | The restore SQL, captured before anything changes. |
| preview_write | sandbox | Runs one write on a disposable shallow clone and returns the diff and verdict. Production is never touched. |
| assess_change | read | Production vs. the candidate your change built, plus lineage and policy. |
| explain_diff | read | A plain-English summary of what moved and where. |
| safe_to_ship | read | The final gate: a boolean, the reasons, and the next step. |
- Verdicts come from rules, never from an LLM.
- preview_write accepts one INSERT, MERGE, UPDATE or DELETE aimed at the clone, and refuses DROP, TRUNCATE, ALTER and GRANT.
- Errors return as JSON, so the agent can recover instead of crashing.
- Hash sampling and depth limits keep every call's cost bounded.
claude mcp add bricksentry -- \
bricksentry mcp --config ./bricksentry.ymlfrom bricksentry import agent_tools as bs
plan = bs.rollback_plan("main.sales.orders")
a = bs.assess_change(["main.sales.orders"], pr="42")
gate = bs.safe_to_ship(a["id"])
if not gate["safe_to_ship"]:
open_pr_with_report(gate["next_step"])05 · For data engineers
A data review on every pull request.
Build your branch's tables into a per-PR schema with your Asset Bundle, then add one step that posts a comment with the diff, blast radius, verdict and rollback.
- run: pip install "bricksentry[databricks]"
- run: >
bricksentry assess --from-git origin/${{ github.base_ref }}
--pr ${{ github.event.pull_request.number }}
--comment --fail-on needs_humanbricksentry impact main.sales.orders
bricksentry preview main.sales.orders --sql "MERGE INTO {{target}} t USING updates u ON t.order_id = u.order_id WHEN MATCHED THEN UPDATE SET *"
bricksentry assess --from-git origin/main --pr 42 --format markdown --fail-on needs_human
bricksentry rollback main.sales.orders06 · Why it's different
Built for where Databricks work actually lives.
- Not only dbt
- covers PySpark notebooks, Jobs and Lakeflow pipelines, where most Databricks work lives.
- Runtime lineage
- uses what Unity Catalog recorded at runtime, instead of guessing from parsed SQL, so notebooks and dashboards show up.
- Free and local to you
- Apache-2.0 and pip-installable. No platform subscription, and queries run in your own workspace.
07 · FAQ
Questions data engineers ask first.
Does it work without Databricks?
Yes for trying it: a DuckDB backend runs the full demo locally (pip install "bricksentry[duckdb]" then run examples/demo/agent_loop_demo.py from the source distribution). Production use targets Databricks SQL warehouses.
Does my data leave my workspace?
No. All diffs run as SQL on your own warehouse; bricksentry only receives aggregate counts and a few sample rows.
Does an LLM decide the verdict?
No. Rules in policy.yml decide. An agent's LLM can read the explanation, but never changes the verdict.
Can an agent damage production with preview_write?
It only runs a single write statement against a disposable shallow clone and refuses DROP, TRUNCATE, ALTER, GRANT and multi-statement SQL.
Does it support dbt?
It works on any table, including dbt-built ones; its focus is the PySpark, Jobs and Lakeflow work that dbt-only tools don't cover.
AWS, Snowflake, BigQuery?
Databricks first (on any cloud, including AWS). Athena/Iceberg and an OpenLineage provider are next on the roadmap.
08 · Roadmap
What ships next.
- v0.1 · nowData diff, Unity Catalog blast radius, policy verdicts, preview_write, rollback, CLI, CI step, MCP server
- v0.2AWS: Athena and Iceberg on the Glue Data Catalog, SageMaker / DataZone lineage
- v0.2OpenLineage provider for Airflow, dbt and Spark
- v0.3Column-level blast radius from system.access.column_lineage
- v0.3Aggregate diff for tables without a stable key
- LaterAsset Bundle parsing, Redshift, scheduled drift checks, a Databricks App UI