Plan a release#
- Read the CHANGELOG upgrade notes for every release you skip. A historical migration may require a coordinated stop rather than a rolling deployment.
- Run the paging upgrade check with the new image against the production database, and fix its findings.
- Take and verify a recoverable backup, including encryption keys and state-service recovery information. Do not start an upgrade while a database dump is running: migrations wait behind it, and paging queries then queue behind the migrations.
- Build or preload approved images and record immutable digests. Rebuilding and restarting are separate actions.
- Run migrations with the schema-owner identity. Keep the runtime credential separate in production.
- Roll out matching API, worker and Web images, then verify readiness and the end-to-end incident workflow.
- Observe queue age, errors, provider outcomes, ACK latency and audit continuity before declaring success.
The upgrade check gate#
Release 1.0.0-rc.1 changes who may be paged: rotation and override participants are paged only while they are an acting member of the schedule's team, and organization viewers are never paged. Anyone else is skipped afterwards, which can leave a schedule with nobody on call. The read-only upgradecheck command reports this before you migrate.
docker run --rm --entrypoint upgradecheck IMAGE_OF_THE_NEW_RELEASE \
paging --database-url "$DATABASE_URL" --format text| Exit code | Meaning |
|---|---|
| 0 | Nothing changes who gets paged. |
| 3 | Findings: people who will stop being paged, or schedules that lose coverage. Fix them and rerun. |
| 1 | Error, such as a connection failure or a database too old for a direct upgrade. |
Helm runs the check as a pre-install/pre-upgrade hook Job before the migration Job, and findings fail the release. Compose runs an upgradecheck service that must succeed before migrate, API and worker start. On a fresh database, and once that migration has been applied, the gate passes without checking. After reviewing findings, continue with upgradeCheck.allowFindings=true (Helm) or ACKNEXA_ALLOW_UPGRADE_FINDINGS=true (Compose).
Why restart did not update the UI#
Restarting a container starts the same image again. Rebuild or pull the intended approved image, then recreate the relevant service. Make sure the browser is using the correct installation and port; a local test stack and a default development stack can coexist with different images and databases.
Helm and migration hooks#
The production chart runs the upgrade check and then a pre-install/pre-upgrade migration hook. Index builds in this release's migrations block writes to the affected tables while they run, the largest being on audit_events; rehearse on a restored copy to know how long they take. Required database Secrets, CA ConfigMaps and pull credentials must exist before these hooks run. Do not assume an External Secrets resource created in the same release has synchronized in time for first install.
Rollback boundaries#
An image rollback does not automatically reverse a database migration. Only use a previous image if it is compatible with the current schema. Do not run destructive down migrations as an automatic rollback step. Follow an explicitly tested recovery plan if schema/data restoration is required.
Production acceptance#
- Complete the real monitoring-source to responder ACK workflow for every enabled channel.
- Prove operation with internet egress denied for a private-only deployment.
- Test backup restore, private-CA rotation, failover and a representative load profile.
- Verify tenant isolation, least-privilege access, secret handling and audit preservation.
- Record PASS, FAIL or NOT TESTED rather than treating manifests or local mocks as production evidence.