kura_migrator (kura v2.20.7)

View Source

Runs, rolls back, and reports status of migrations.

Discovers migration modules automatically from the application that owns the repo module. Any module named m<YYYYMMDDHHMMSS>_<name> in the application's module list is treated as a migration. Tracks applied versions in a schema_migrations table.

A repo may pull in migrations from further applications - an extension shipping its own schema, for instance - by exporting the optional kura_repo callback migration_apps/0. The applications are ordered by the OTP applications dependency graph, so a dependency's migrations always run before its dependents'. See migration_apps/1.

schema_migrations records versions only. A version is therefore global across every application feeding a repo: two applications shipping the same version is a hard error (duplicate_migration_version) rather than a silent last-one-wins.

All migrations run within a single transaction protected by a PostgreSQL advisory lock (pg_advisory_xact_lock) to prevent concurrent execution across multiple nodes. The applied-version set is read inside that lock, so two nodes booting together cannot both decide the same migration is pending, and schema_migrations is created under the same lock in its own transaction, so they cannot race to create it either.

migrate/1 and status/1 span every application. The two operations that record versions without running the DDL they stand for - rollback/1,2 and fake/1 - refuse a multi-application set rather than guess which application an operator meant; rollback/3 and fake/2 name one.

Summary

Functions

Check migration operations for actions that are unsafe during rolling deployments.

Compile a single DDL operation to SQL using RepoMod's dialect.

Create schema_migrations if it is not already there.

Stamp all pending migrations as applied WITHOUT running their DDL.

Stamp App's pending migrations as applied WITHOUT running their DDL.

Run all pending migrations in order.

The applications whose migrations run for RepoMod, in dependency order.

Roll back the last migration.

Roll back the last Steps migrations.

Roll back the last Steps migrations of App.

Return the status of all discovered migrations (up or pending).

Functions

check_unsafe_operations/2

-spec check_unsafe_operations([kura_migration:operation()], [kura_migration:safe_entry()]) -> [map()].

Check migration operations for actions that are unsafe during rolling deployments.

compile_operation/2

-spec compile_operation(module(), kura_migration:operation()) -> binary().

Compile a single DDL operation to SQL using RepoMod's dialect.

ensure_database(RepoMod)

-spec ensure_database(module()) -> ok.

ensure_schema_migrations(RepoMod)

-spec ensure_schema_migrations(module()) -> ok | {error, term()}.

Create schema_migrations if it is not already there.

Runs under the migration advisory lock in a transaction of its own. CREATE TABLE IF NOT EXISTS is not atomic against a concurrent creator: both sessions can pass the existence check and the loser fails on pg_type's unique index. Taking the lock first is what makes two nodes booting together safe, and the transaction is separate from the migration one so the lock is released before migrate/1 re-takes it - no lock is held across the two.

Returns {error, {schema_migrations_failed, Reason}} when the statement does not succeed, rather than leaving every later query to fail on a table that was never created.

fake(RepoMod)

-spec fake(module()) -> {ok, [integer()]} | {error, term()}.

Stamp all pending migrations as applied WITHOUT running their DDL.

Baseline for brownfield adoption: after rebar3 kura gen_schemas bootstraps schema modules from an existing database, the first compile emits create_table migrations for tables that already exist. fake/1 records those in schema_migrations so real migrations proceed from there. It never executes migration DDL.

Refuses a repo drawing migrations from more than one application with {error, {ambiguous_fake, Apps}}, and does not stamp anything. An operator baselining a brownfield database has the host's tables in front of them; an extension added at the same time has none, and stamping its migrations too means its tables are never created and migrate/1 thereafter believes there is nothing to do. Name the one application being baselined with fake/2.

Precondition: fake/1 stamps EVERY pending migration of the single application it covers. Only run it when every pending migration corresponds to schema that already exists - a genuinely-new migration in the pending set would be stamped without its table ever being created, and a later migrate/1 would then treat it as done. Check status/1 first; the versions about to be stamped are also logged at warning level. For the mixed new/existing case use a version-scoped baseline (kura#156).

fake(RepoMod, App)

-spec fake(module(), atom()) -> {ok, [integer()]} | {error, term()}.

Stamp App's pending migrations as applied WITHOUT running their DDL.

The per-application form of fake/1, for a repo drawing migrations from more than one application. Only App's pending migrations are stamped; every other application's stay pending, so a host can be baselined while a freshly installed extension still gets its tables created by migrate/1.

App must be one of migration_apps/1, otherwise {error, {unknown_migration_app, App, Apps}} and nothing is stamped.

fake/1's precondition applies unchanged, scoped to App: every one of its pending migrations must correspond to schema that already exists.

migrate(RepoMod)

-spec migrate(module()) -> {ok, [integer()]} | {error, term()}.

Run all pending migrations in order.

migration_apps(RepoMod)

-spec migration_apps(module()) -> {ok, [atom()]} | {error, term()}.

The applications whose migrations run for RepoMod, in dependency order.

The application owning the repo module is always included. A repo pulls in further applications by exporting the optional kura_repo callback migration_apps/0:

-module(my_repo).
-behaviour(kura_repo).
-export([otp_app/0, migration_apps/0]).

otp_app() -> my_app.

migration_apps() -> [my_gdpr_extension].

The result is a topological sort of the applications' OTP applications lists, so a dependency's migrations always run before its dependents'. There is no separate ordering configuration - the dependency graph OTP already declares is the one that is used. Applications are compared by their transitively reachable dependencies, so an intermediate application that ships no migrations still orders the two that do.

Every declared application must be loaded; one that is not yields {error, {migration_apps_not_loaded, Apps}} rather than contributing zero migrations silently.

rollback(RepoMod)

-spec rollback(module()) -> {ok, [integer()]} | {error, term()}.

Roll back the last migration.

rollback(RepoMod, Steps)

-spec rollback(module(), non_neg_integer()) -> {ok, [integer()]} | {error, term()}.

Roll back the last Steps migrations.

The window is the Steps highest applied versions. Every version in it must resolve to a discovered migration module: if one does not, the rollback aborts with {error, {unknown_applied_versions, Versions}} rather than quietly rolling back fewer migrations than asked for. The usual cause is a migration module deleted from source while its schema_migrations row remains.

Within the window, migrations run in reverse apply order, so a dependency's down/0 never runs before its dependents'.

Refuses a repo drawing migrations from more than one application with {error, {ambiguous_rollback, Apps}}, and rolls nothing back. schema_migrations records versions and nothing else, so the only window this function can compute is one ordered by version - and "the last three" across unrelated applications can take one migration each from three of them and leave every one half-migrated, in an order their dependency graph never sanctioned. Name the application with rollback/3.

rollback(RepoMod, App, Steps)

-spec rollback(module(), atom(), non_neg_integer()) -> {ok, [integer()]} | {error, term()}.

Roll back the last Steps migrations of App.

The per-application form of rollback/2, for a repo drawing migrations from more than one application. The window is the Steps highest applied versions among those App's discovered migrations claim, so no other application's migrations can enter it whatever their versions are. Within the window, execution reverses App's apply order.

App must be one of migration_apps/1, otherwise {error, {unknown_migration_app, App, Apps}} and nothing is rolled back.

Rolling an application back below a version a dependent application builds on is not detected: schema_migrations has no record of which application applied a version, so kura cannot tell an orphan row of App's from any other application's. Roll dependents back first.

status(RepoMod)

-spec status(module()) -> [{integer(), module(), up | pending}] | {error, term()}.

Return the status of all discovered migrations (up or pending).

Listed in apply order: applications in dependency order, versions ascending within each. Spans every application, so it is the thing to check before fake/2 or rollback/3.

Returns [] when the pool is unavailable, and {error, Reason} when schema_migrations cannot be created or discovery itself fails (a duplicate version across applications, for instance).

wait_for_pool(RepoMod)

-spec wait_for_pool(module()) -> ok | {error, pool_unavailable}.

wait_for_pool(RepoMod, Timeout)

-spec wait_for_pool(module(), non_neg_integer()) -> ok | {error, pool_unavailable}.