Prove your WAL archive can actually restore
PostgreSQL only checks the exit code of archive_command. A command that exits 0 and copies nothing drove archived_count from 23 to 27 with failed_count 0, while the archive directory gained no files at all.
Problem
What you're actually looking at
The symptom as it shows up on a real server.
WAL archiving is the foundation of point-in-time recovery, and its health is usually monitored through pg_stat_archiver. That view reports whether archive_command exited zero, not whether a file arrived anywhere. Any command that swallows its own failure — a script with no error check, a full destination, a permission problem — produces a green dashboard over an archive that cannot restore.
Meridian's monitoring shows archiving healthy with zero failures. Nobody has compared the archive's contents against what PostgreSQL believes it archived.
In plain English
PostgreSQL hands each finished WAL file to a command you supply and asks one question: did it work? It decides the answer purely from the command's exit code. If your command reports success without actually copying anything — because a script ignored an error, or the destination filled up — PostgreSQL takes it at its word, marks the file archived, and moves on. The counters climb, the dashboard is green, and the archive is empty. The only real check is to look at the archive itself.
Full runbook for this incident
- The full identify checklist — the exact signals that tell you it's this incident
- Every diagnostic query; PostgreSQL 18 output is attached only to the steps we captured
- The resolution path and the pitfalls that make it worse
- Mitigation steps to stop it recurring, plus a verify-you're-done query
More in this category
Other Backup & recovery runbooks
Neighbouring incidents that share the same diagnostic surface.
Connected
How this connects to the rest of the library
A live view of this page's real cross-references — what explains it, what fixes it, what to tune, and where to go next. Every link is an authored relationship, not a guess.
Fixes these errors