Ekbasis · client 0.1.19 · October 2026

Migration guard

Before a pull request is merged, it says whether its PostgreSQL migrations will fail or lose data.

How it works

The action builds the database schema by applying the base branch's migrations to a scratch PostgreSQL in the runner. If you give it a read-only replica, it also reads planner statistics — row estimates and the share of NULLs per column — never data values. Ekbasis then forecasts, for each new migration file, whether it fails and whether it loses data; the verdict and its reason are decided in code from those two answers. It leaves one comment on the pull request, updated on each push, and can fail the check.

The SQL of the pull request runs as a throwaway role without superuser rights, in a container with no network. A migration that needs more than that is not run: the guard says it cannot foresee it and asks for a review by hand.

# .github/workflows/migration-guard.yml
name: migration guard
on:
  pull_request:                 # never pull_request_target
    paths: ["**/migrations/**", "**/*.sql"]
permissions:
  contents: read
  pull-requests: write          # the comment
jobs:
  guard:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0        # the base branch's migrations build the schema
      - uses: OpenInterpretability/migration-guard@v1
        with:
          api-key: ${{ secrets.EKBASIS_API_KEY }}
          # db-stats-url: ${{ secrets.READ_REPLICA_URL }}   # optional: planner statistics only

Supports PostgreSQL SQL migrations from Prisma, Diesel, golang-migrate, goose and plain *.sql folders. Documentation. An API key is free to create at /console.

What we measured

On 184 PostgreSQL migrations it had never seen, each run on a database seeded with synthetic rows, its verdict was right 87.5% of the time, against 69.6% for linter-style rules. It flagged 94.6% of the migrations that fail or lose data, and raised a false alarm on 15.6% of the safe ones, against 43% for the rules. When it flags a migration, the reason it gives — it fails, or it loses data — was right in 92.5% of cases. The plan was pre-registered and hashed before any call.

Not everything worked. An earlier version that only told the model about transactions did not cut false loss alarms on migrations that fail, and its loss probabilities were not calibrated; the verdict is now decided in code. On the migrations that look destructive, accuracy was 78.6% against 67.9% for the rules. The tables held synthetic rows, so results on real databases may differ. Every result, the plans and the scripts.

The model sees the part of the schema the migration reaches: the tables it touches in full, each table linked to them by a foreign key in one line, the views that read them, their triggers with the functions they call, and the sequences, enums and domains involved. In a pre-registered test on 184 migrations this was right more often than the earlier name-matching state with replica statistics (89.7% against 88.0%), and as often without them, with fewer false alarms and about a quarter fewer input tokens. The study. States up to about 64,000 tokens are read; a separate pre-registered test found no loss in accuracy up to that length. Long-state test.

Alembic and Django

With framework: alembic or framework: django, the framework itself renders the SQL. The pull request's code runs only inside a container with no network and no secrets; the API key is used in a later step. A migration with code that is not SQL, such as a Django RunPython, cannot be foreseen: it is listed in the comment for review by hand and does not fail the check, unless you set on-code: block.

In a second pre-registered evaluation, on 571 migrations from five projects it had never seen, it answered 383; the rest held code it cannot foresee. On those 383, its verdict was right 98.6% of the time for Alembic and 95.5% for Django, against 77.3% and 82.6% for linter-style rules, with 1.6% and 2.4% false alarms. Alembic passed every criterion, but with only 13 risky migrations among those it answered, so its numbers are less certain. Django missed 6 of 34 risky migrations, below the recall we required, so Django support is marked experimental. Code it cannot foresee is common: 45% of the Alembic migrations and 24% of the Django ones, and a good share of the real risk sits there; counting it as missed, it caught 31% and 48% of all risky migrations. A first evaluation, which blocked on that code, failed. Rails is experimental too: in its own pre-registered test its verdict was right 88.6% of the time against 60.7% for the rules, but its reason was right on 33 of 40 risky migrations, below our 85% target. Rails study. The plans, both results and the scripts.

On real incidents

We also ran it on 25 migrations that caused real incidents in open-source projects, each checked as its pull request stood. It flagged 9 for the right reason and missed 8; the most common miss is a new constraint that existing rows violate. This was not a random sample, and an attempt to fix that miss with more facts did not help. Every case.

Cost

A migration takes about 4,000 input tokens and a second or two of model time. A migration already checked in the same pull request, unchanged, reuses its answer from the Actions cache at no cost. The guard is a warning layer that can be wrong, not a replacement for review or backups.