kura_migrator (kura v2.20.7)
View SourceRuns, rolls back, and reports status of migrations.
Discovers migration modules automatically from the application that owns the
repo module. Any module named m<YYYYMMDDHHMMSS>_<name> in the application's
module list is treated as a migration. Tracks applied versions in a
schema_migrations table.
A repo may pull in migrations from further applications - an extension
shipping its own schema, for instance - by exporting the optional
kura_repo callback migration_apps/0. The applications are ordered by
the OTP applications dependency graph, so a dependency's migrations
always run before its dependents'. See migration_apps/1.
schema_migrations records versions only. A version is therefore global
across every application feeding a repo: two applications shipping the
same version is a hard error (duplicate_migration_version) rather than
a silent last-one-wins.
All migrations run within a single transaction protected by a PostgreSQL
advisory lock (pg_advisory_xact_lock) to prevent concurrent execution
across multiple nodes. The applied-version set is read inside that lock,
so two nodes booting together cannot both decide the same migration is
pending, and schema_migrations is created under the same lock in its
own transaction, so they cannot race to create it either.
migrate/1 and status/1 span every application. The two operations
that record versions without running the DDL they stand for -
rollback/1,2 and fake/1 - refuse a multi-application set rather than
guess which application an operator meant; rollback/3 and fake/2 name
one.
Summary
Functions
Check migration operations for actions that are unsafe during rolling deployments.
Compile a single DDL operation to SQL using RepoMod's dialect.
Create schema_migrations if it is not already there.
Stamp all pending migrations as applied WITHOUT running their DDL.
Stamp App's pending migrations as applied WITHOUT running their DDL.
Run all pending migrations in order.
The applications whose migrations run for RepoMod, in dependency order.
Roll back the last migration.
Roll back the last Steps migrations.
Roll back the last Steps migrations of App.
Return the status of all discovered migrations (up or pending).
Functions
-spec check_unsafe_operations([kura_migration:operation()], [kura_migration:safe_entry()]) -> [map()].
Check migration operations for actions that are unsafe during rolling deployments.
-spec compile_operation(module(), kura_migration:operation()) -> binary().
Compile a single DDL operation to SQL using RepoMod's dialect.
-spec ensure_database(module()) -> ok.
Create schema_migrations if it is not already there.
Runs under the migration advisory lock in a transaction of its own.
CREATE TABLE IF NOT EXISTS is not atomic against a concurrent
creator: both sessions can pass the existence check and the loser
fails on pg_type's unique index. Taking the lock first is what makes
two nodes booting together safe, and the transaction is separate from
the migration one so the lock is released before migrate/1 re-takes
it - no lock is held across the two.
Returns {error, {schema_migrations_failed, Reason}} when the
statement does not succeed, rather than leaving every later query to
fail on a table that was never created.
Stamp all pending migrations as applied WITHOUT running their DDL.
Baseline for brownfield adoption: after rebar3 kura gen_schemas
bootstraps schema modules from an existing database, the first
compile emits create_table migrations for tables that already
exist. fake/1 records those in schema_migrations so real
migrations proceed from there. It never executes migration DDL.
Refuses a repo drawing migrations from more than one application with
{error, {ambiguous_fake, Apps}}, and does not stamp anything. An
operator baselining a brownfield database has the host's tables in
front of them; an extension added at the same time has none, and
stamping its migrations too means its tables are never created and
migrate/1 thereafter believes there is nothing to do. Name the one
application being baselined with fake/2.
Precondition: fake/1 stamps EVERY pending migration of the single
application it covers. Only run it when every pending migration
corresponds to schema that already exists - a genuinely-new migration
in the pending set would be stamped without its table ever being
created, and a later migrate/1 would then treat it as done. Check
status/1 first; the versions about to be stamped are also logged at
warning level. For the mixed new/existing case use a version-scoped
baseline (kura#156).
Stamp App's pending migrations as applied WITHOUT running their DDL.
The per-application form of fake/1, for a repo drawing migrations
from more than one application. Only App's pending migrations are
stamped; every other application's stay pending, so a host can be
baselined while a freshly installed extension still gets its tables
created by migrate/1.
App must be one of migration_apps/1, otherwise
{error, {unknown_migration_app, App, Apps}} and nothing is stamped.
fake/1's precondition applies unchanged, scoped to App: every one
of its pending migrations must correspond to schema that already
exists.
Run all pending migrations in order.
The applications whose migrations run for RepoMod, in dependency order.
The application owning the repo module is always included. A repo pulls
in further applications by exporting the optional kura_repo callback
migration_apps/0:
-module(my_repo).
-behaviour(kura_repo).
-export([otp_app/0, migration_apps/0]).
otp_app() -> my_app.
migration_apps() -> [my_gdpr_extension].The result is a topological sort of the applications' OTP applications
lists, so a dependency's migrations always run before its dependents'.
There is no separate ordering configuration - the dependency graph OTP
already declares is the one that is used. Applications are compared by
their transitively reachable dependencies, so an intermediate
application that ships no migrations still orders the two that do.
Every declared application must be loaded; one that is not yields
{error, {migration_apps_not_loaded, Apps}} rather than contributing
zero migrations silently.
Roll back the last migration.
-spec rollback(module(), non_neg_integer()) -> {ok, [integer()]} | {error, term()}.
Roll back the last Steps migrations.
The window is the Steps highest applied versions. Every version in it
must resolve to a discovered migration module: if one does not, the
rollback aborts with {error, {unknown_applied_versions, Versions}}
rather than quietly rolling back fewer migrations than asked for. The
usual cause is a migration module deleted from source while its
schema_migrations row remains.
Within the window, migrations run in reverse apply order, so a
dependency's down/0 never runs before its dependents'.
Refuses a repo drawing migrations from more than one application with
{error, {ambiguous_rollback, Apps}}, and rolls nothing back.
schema_migrations records versions and nothing else, so the only
window this function can compute is one ordered by version - and
"the last three" across unrelated applications can take one migration
each from three of them and leave every one half-migrated, in an order
their dependency graph never sanctioned. Name the application with
rollback/3.
-spec rollback(module(), atom(), non_neg_integer()) -> {ok, [integer()]} | {error, term()}.
Roll back the last Steps migrations of App.
The per-application form of rollback/2, for a repo drawing migrations
from more than one application. The window is the Steps highest
applied versions among those App's discovered migrations claim, so
no other application's migrations can enter it whatever their versions
are. Within the window, execution reverses App's apply order.
App must be one of migration_apps/1, otherwise
{error, {unknown_migration_app, App, Apps}} and nothing is rolled back.
Rolling an application back below a version a dependent application
builds on is not detected: schema_migrations has no record of which
application applied a version, so kura cannot tell an orphan row of
App's from any other application's. Roll dependents back first.
Return the status of all discovered migrations (up or pending).
Listed in apply order: applications in dependency order, versions
ascending within each. Spans every application, so it is the thing to
check before fake/2 or rollback/3.
Returns [] when the pool is unavailable, and {error, Reason} when
schema_migrations cannot be created or discovery itself fails (a
duplicate version across applications, for instance).
-spec wait_for_pool(module()) -> ok | {error, pool_unavailable}.
-spec wait_for_pool(module(), non_neg_integer()) -> ok | {error, pool_unavailable}.