staffPostgreSQL HA / PatroniPro answerTested in a controlled lab, with the output captured

Patroni leadership loss during a DCS partition: prove one writer.

The question is short because the trap is large. PostgreSQL has no opinion about which node should be primary; that decision belongs to Patroni and the distributed store it uses to hold the leader lock, and candidates who blur that boundary tend to propose promoting something. The second trap is treating an unreachable store as a dead leader, which is exactly the reasoning that manufactures two writers. What a staff panel wants is a proof, not a status page: a check that would fail loudly if the invariant were broken. They also listen for what you refuse to claim, particularly around recovery objectives and fencing hardware. This page covers the proof, the three distinct scenarios, and the honest limits.

What the interviewer is scoring

“Switch the VIP and promote.” Or “DCS is down so fail over.” Or quoting a lag threshold as a production RPO. Or claiming watchdog saved the cluster when the device was never present.

In short: HA JDs score whether you treat DCS unreachability as different from confirmed leader death, whether you prove one writer with writes on every member, and whether you refuse raw pg_promote() and invented RPO.

ProFrom a real interview loop

The spoken answer, backed by captured output

The question, the intro, what the interviewer is scoring, the reasoning outline, what you would verify and the follow-up probes are all open above. Pro unlocks the spoken answer to “Patroni leadership loss during a DCS partition: prove one writer.”, the reasoning behind it, and the verification that backs it.
  • The full 90-second answer, written first person, the way you would actually say it
  • How I reason through it: the mechanism, the decision points, and where the claim stops
  • Verification steps with output captured verbatim from a controlled PostgreSQL 18 lab run.
  • Worked responses to the 3 follow-up probes listed above, plus the traps that lose the point

Card required. Cancel before day 7 and you are not charged.

Compare plans

How to reason through it

  • patronictl list (or REST /cluster): one Leader, others Replica.
  • Prove it: INSERT on every member — exactly one succeeds, others 25006 read-only.
  • Planned switchover is patronictl switchover, not pg_promote().
  • Unplanned: stop the leader; require a later timeline and an eligible streaming candidate; do not hard-code the winner.
  • DCS unreachable with every member acknowledging the failsafe check → the current leader may keep writing.
  • DCS unreachable with a required failsafe member missing → expect zero writers, not a second primary.
  • Treat the lag threshold, the leader-lock TTL, and run-local WAL volume as inputs, not an SLA.
  • If no watchdog device is present, say so — do not simulate fencing.

What I would verify

  • List the cluster first, then prove the listing with a write attempt on every member.
  • Require a later timeline on the new leader before accepting that a failover completed.
  • Read the current leader's own logs when the store is unreachable, rather than inferring from the store's absence.
  • Check whether a watchdog device actually exists before describing fencing behaviour.
  • Treat any observed lag or WAL volume as a run-local input rather than a recovery objective.

Follow-ups they push on

  • The coordination store is unreachable. Do you fail over?
  • Why not call the promote function directly?
  • What is your RPO on this cluster?

The probes are open. Pro carries the spoken answer, the reasoning behind it, and the verification steps, including a worked response to each of these.

Concepts tested

Patroni DCS leader lockplanned switchoverunplanned failovercandidate eligibilitytimelinefailsafe ACKfailsafe demotionone-writer invariantwatchdog capabilityasynchronous lag bound as input

Learn it, run it, then say it

Three steps, in order. Nothing here is a detour.

Questions that go with this one