Docs · Get started
Updates and maintenance
Upgrades verify signatures, back up first, check health and roll back on their own. This page covers updates, backups, secrets, status and removal, and which command works on which provider.
Which command works where
| Task | gcp | gcp-firebase | Self-hosted |
|---|---|---|---|
upgrade | yes | yes | yes |
| Install update from the admin | yes, via the upgrader job | yes, via the upgrader job | yes, via the update agent (Linux with systemd) |
backup | on-demand Cloud SQL backup | on-demand Cloud SQL backup | one .tar.gz archive |
| Restore | printed gcloud sql backups restore command | printed gcloud sql backups restore command | restore --from … --yes |
rotate-secret | yes | yes | not available |
status | yes | install job progress only | status-self-hosted (basic info) |
destroy | deletes the project | not available | not available |
All commands run through the same package: bunx @caelo-cms/provisioning@<version> <command>. Name the version: without it, bunx may keep running an old copy it downloaded earlier. For Google Cloud installs, run the commands on a machine that knows the install. That is the machine you installed from, or any machine after bunx @caelo-cms/provisioning attach --project <project-id>. Self-hosted commands run from the directory that holds .caelo/.
Settings, Caelo updates
The admin page Settings, Caelo updates (/settings/updates) shows your version, whether a newer release is out, and its release notes. For an Owner it also shows:
- Install update, which you confirm with your Caelo password. The admin never installs anything itself. On Google Cloud it starts the upgrader, a Cloud Run job in your project. On a self-hosted install it writes a request that the update agent on your server picks up. A timeline shows each step, and Update history keeps every attempt.
- The exact terminal command for your install, as an alternative.
- Automatic updates. By default Caelo tells you about a release and you install it ("notify"). You can let it install security fixes, patch releases and minor releases on its own, each with its own switch. An automatic install waits 24 hours after a release comes out (2 hours for security fixes), and you can limit it to a maintenance window. If an automatic install fails, automatic updates pause until an Owner resumes them.
- Some releases are never installed automatically: releases with destructive database changes, releases that change your cloud setup, and major releases. The signed release information says which kind a release is, and the updates page tells you what to do.
The AI can tell you which version runs and whether an update is out. It cannot install one or change the policy.
Upgrade from the terminal
Google Cloud:
bunx @caelo-cms/provisioning@<version> upgrade --version <version> --install <domain>
Self-hosted, from the directory that holds .caelo/ (without --install, which names Google Cloud installs):
bunx @caelo-cms/provisioning@<version> upgrade --version <version>
The version appears twice on purpose. @<version> runs the installer of the release you are moving to, and --version rolls exactly that release even if a newer one ships in the meantime. Without --version, upgrade takes the newest stable release. --channel rc or --channel beta opts into pre-releases.
What every upgrade does, in this order:
- Verify. Resolves the release images to digests and checks their cosign signatures. A failed check changes nothing. You need cosign installed.
- Back up. Google Cloud: an on-demand Cloud SQL backup, whose id is printed. Self-hosted:
pg_dumpof both databases into.caelo/backups/. If the backup fails, the upgrade stops. - Migrate. Runs the database migrations with the new release's image while the current release keeps serving.
- Switch and check. Rolls the admin, then the gateway, and checks the health of each.
- Roll back if needed. If a health check fails, the previous images are started again. The database is not restored automatically: the backup id (Google Cloud) or path (self-hosted) is printed for the case that the old release no longer works with the migrated schema. Restoring overwrites everything written since the backup.
On Google Cloud an upgrade also brings the infrastructure up to what the new release expects, such as permissions and secrets. Run it signed in with gcloud auth login as an account that may change the project and can open the admin, because the health check goes through Google's Identity-Aware Proxy. --skip-health-check skips the deep check, for example when moving to a release that predates it. Re-running upgrade is safe on every provider.
Releases that change your Google Cloud setup cannot be installed by the in-admin upgrader. The updates page then gives you an Open Cloud Shell link and the commands to run there.
The self-hosted update agent
On Linux with systemd, init and upgrade set up a systemd timer that runs as your user once a minute. It picks up update requests from the admin, verifies the release's signed manifest and the new installer's npm provenance, and then runs the same upgrade as above. It reaches Postgres through its own restricted login on the loopback interface only. Manage it with:
bunx @caelo-cms/provisioning@<version> agent install
bunx @caelo-cms/provisioning@<version> agent status
bunx @caelo-cms/provisioning@<version> agent logs
bunx @caelo-cms/provisioning@<version> agent uninstall
The agent needs cosign and npm on its PATH, and your user in the docker group. The agent and a hand-run upgrade never roll at the same time.
Backups and restore
Google Cloud (gcp and gcp-firebase)
| What | How it is protected |
|---|---|
| Database (both databases on one Cloud SQL instance) | Automated backup every night at 03:00. The last 7 are kept. |
| Point-in-time recovery | gcp-firebase: always on, for the last 7 days. gcp: only with high availability, which is off by default. |
| Before every upgrade | An extra Cloud SQL backup. If it fails, the upgrade changes nothing. |
| Uploaded media | The media bucket keeps older versions of every file, with no expiry. A deleted or overwritten file stays recoverable in the bucket. Older versions cost storage. Caelo has no restore command for media. |
| Published site | gcp-firebase: every published build is kept as a Firebase Hosting version, and the live site can be rolled back to an earlier build. gcp: the site bucket has no versions. The site can be rebuilt from the database, but rolling the live site back to an earlier build does not work yet. |
| Install state (Pulumi state and install files) | A versioned bucket in your project. Older versions are kept for 30 days. |
| Secrets | Not backed up by Caelo. They stay in Secret Manager in your project. |
backup takes an extra backup now and prints the matching restore command:
bunx @caelo-cms/provisioning@<version> backup --install <domain>
# prints e.g.: gcloud sql backups restore <id> --restore-instance=<instance> --project=<project>
There is no restore command on Google Cloud. You run the printed gcloud command yourself. It overwrites the whole database instance, both databases, and everything written since the backup is lost. Caelo does not wrap point-in-time recovery; use Cloud SQL's own tools for that.
Self-hosted
bunx @caelo-cms/provisioning@<version> backup --to caelo-backup.tar.gz
bunx @caelo-cms/provisioning@<version> restore --from caelo-backup.tar.gz --yes
The archive holds a pg_dump of both databases, the published site and the uploaded media. The admin and the gateway stop while it runs, so the parts match: a short outage. The archive does not contain .caelo/config.json, on purpose, so you can copy the archive off the server. Keep a copy of that file yourself, because without it the stored AI provider keys in a backup cannot be decrypted. restore checks the archive first, replaces both databases and the files, migrates, and starts everything with health checks. If it stops half-way, run it again.
There is no schedule and no point-in-time recovery. Schedule backup yourself, for example as a daily cron job, and copy the archives off the server.
Rotate secrets (Google Cloud)
bunx @caelo-cms/provisioning@<version> rotate-secret <name> --install <domain>
Names: postgres-password, public-role-password, internal-secret, tool-approval-secret, staging-gate-secret. The command stores a new value in Secret Manager, updates the database role for the password secrets, and rolls the services that read it. Rotating staging-gate-secret signs every staging session out. Your AI provider key is changed in the admin under Settings, AI.
The old name gateway-role-password is still accepted but has no effect since 0.10.51, because the gateway no longer logs into the content database. It will be removed.
Status
bunx @caelo-cms/provisioning@<version> status
On gcp this shows the admin's Cloud Run URL, the Cloud SQL state, your URLs, the installer version and whether a newer release exists. On gcp-firebase it currently shows only the install job's progress. On a self-hosted install use status-self-hosted for the basic facts and docker compose -f .caelo/docker-compose.yml ps for container health.
Remove an install
gcp:destroyasks twice (the second time you type the domain) and then deletes the whole Google Cloud project. Google keeps a deleted project restorable for 30 days.gcp-firebase:destroydoes not delete anything yet. Delete the project in the Google Cloud console or withgcloud projects delete <project-id>.- Self-hosted: there is no command. Stop the stack with
docker compose -f .caelo/docker-compose.yml down; adding-valso deletes the database volume.
In every case the local install directory (~/.caelo-<install-id>/ or .caelo/) stays until you delete it.
Something wrong or missing on this page? Open an issue on GitHub