Pgbouncer Pool Exhaustion
Clients see connection acquisition or query latency spikes while the database CPU and lock graphs stay calm. The number of waiting application threads...
In 10 seconds
The application reports that database connections are slow or hanging, but PostgreSQL itself looks healthy -- low CPU, no long locks, plenty of idle backends. Everything runs through PgBouncer, and the queue is forming there, not in Postgres. More clients want a pooled server connection than the pool has (DEFAULT_POOL_SIZE), so PgBouncer accepts the clients but parks the excess ones until a server frees up. You need to recognise that the bottleneck is the pool, read PgBouncer's own admin console to prove it, and resize the pool safely instead of blindly raising max_connections on the database.
Why This Happens
PgBouncer multiplexes many client connections onto a small number of server connections. Each (database, user) pair gets a pool of at most DEFAULT_POOL_SIZE server connections to PostgreSQL. MAX_CLIENT_CONN governs how many CLIENT connections PgBouncer will accept overall -- it is deliberately much larger than the pool. So when N clients each need a server at the same time and N exceeds DEFAULT_POOL_SIZE, PgBouncer hands a server to the first DEFAULT_POOL_SIZE of them and puts the rest in cl_waiting until a server is returned to the pool. WHEN a server is returned depends on pool_mode: in transaction mode it is released at COMMIT (so an idle-between-transactions client frees its server immediately); in session mode the server is PINNED to the client for the entire session and is only released when the client disconnects.
That single difference decides whether a momentarily idle client blocks others. The correct fix for genuine concurrency is to raise DEFAULT_POOL_SIZE (and, if needed, max_connections on Postgres to back it) -- not to raise it blindly, because every server connection in every pool consumes a real Postgres backend (memory + a process). The maxwait figure and the cl_waiting count tell you exactly how much head-of-line blocking the current pool size is causing. The maxwait figure and the cl_waiting count tell you exactly how much head-of-line blocking the current pool size is causing.
Diagnose
Free Tier -- Basic Approach
The one screen an on-call engineer pulls up: SHOW POOLS. cl_waiting and a rising maxwait are the smoke alarm for pool exhaustion -- no database-side query needed.
SHOW POOLS;
database | user | cl_active | cl_waiting | cl_active_cancel_req | cl_waiting_cancel_req | sv_active | sv_active_cancel | sv_being_canceled | sv_idle | sv_used | sv_tested | sv_login | maxwait | maxwait_us | pool_mode | load_balance_hosts -----------+-----------+-----------+------------+----------------------+-----------------------+-----------+------------------+-------------------+---------+---------+-----------+----------+---------+------------+-------------+-------------------- pgbouncer | pgbouncer | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | statement | postgres | postgres | 5 | 7 | 0 | 0 | 5 | 0 | 0 | 0 | 0 | 0 | 0 | 7 | 40666 | transaction | (2 rows)
diagnose_free shows the 'postgres' pool with cl_waiting = 7, sv_active = 5, and maxwait = 7 seconds. Read it as: seven clients are waiting because all five server slots are busy, and the unluckiest has already waited seven seconds.
This is the metric to graph and alert on; cl_waiting flat at zero means a healthy pool, cl_waiting > 0 with maxwait climbing means clients are starved for a server connection.
Fix
Give the pool more server slots and reload PgBouncer. Here that means recreating the bouncer with DEFAULT_POOL_SIZE = 20 (the container-config equivalent of editing pgbouncer.ini and RELOAD). The same 12-client load now fits with room to spare.
SHOW POOLS;
database | user | cl_active | cl_waiting | cl_active_cancel_req | cl_waiting_cancel_req | sv_active | sv_active_cancel | sv_being_canceled | sv_idle | sv_used | sv_tested | sv_login | maxwait | maxwait_us | pool_mode | load_balance_hosts -----------+-----------+-----------+------------+----------------------+-----------------------+-----------+------------------+-------------------+---------+---------+-----------+----------+---------+------------+-------------+-------------------- pgbouncer | pgbouncer | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | statement | postgres | postgres | 12 | 0 | 0 | 0 | 12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | transaction | (2 rows)
SHOW LISTS;
list | items ---------------+------- databases | 2 users | 2 peers | 0 pools | 2 peer_pools | 0 free_clients | 37 used_clients | 13 login_clients | 0 free_servers | 38 used_servers | 12 dns_names | 1 dns_zones | 0 dns_queries | -11 dns_pending | 0 (14 rows)
After raising DEFAULT_POOL_SIZE to 20 and re-running the identical 12-client load, the 'postgres' pool shows cl_active = 12, cl_waiting = 0, sv_active = 12, maxwait = 0. Every client got a server immediately; nobody queues.
SHOW LISTS confirms used_servers = 12 (still well under the database's max_connections). The fix is scoped to the pool, not the application: no client code changed, the latency simply disappears because the head-of-line blocking is gone.
Verify
Confirm the healthy steady state holds: with pool_size = 20 and 12 clients, cl_waiting and maxwait stay at zero and queries flow instead of piling up.
SHOW POOLS;
database | user | cl_active | cl_waiting | cl_active_cancel_req | cl_waiting_cancel_req | sv_active | sv_active_cancel | sv_being_canceled | sv_idle | sv_used | sv_tested | sv_login | maxwait | maxwait_us | pool_mode | load_balance_hosts -----------+-----------+-----------+------------+----------------------+-----------------------+-----------+------------------+-------------------+---------+---------+-----------+----------+---------+------------+-------------+-------------------- pgbouncer | pgbouncer | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | statement | postgres | postgres | 12 | 0 | 0 | 0 | 12 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | transaction | (2 rows)
SHOW STATS;
database | total_server_assignment_count | total_xact_count | total_query_count | total_received | total_sent | total_xact_time | total_query_time | total_wait_time | total_client_parse_count | total_server_parse_count | total_bind_count | avg_server_assignment_count | avg_xact_count | avg_query_count | avg_recv | avg_sent | avg_xact_time | avg_query_time | avg_wait_time | avg_client_parse_count | avg_server_parse_count | avg_bind_count -----------+-------------------------------+------------------+-------------------+----------------+------------+-----------------+------------------+-----------------+--------------------------+--------------------------+------------------+-----------------------------+----------------+-----------------+----------+----------+---------------+----------------+---------------+------------------------+------------------------+---------------- pgbouncer | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 postgres | 12 | 12 | 12 | 300 | 504 | 0 | 0 | 54643 | 0 | 0 | 0 | 1 | 1 | 1 | 27 | 46 | 0 | 0 | 4553 | 0 | 0 | 0 (2 rows)
verify re-reads SHOW POOLS: cl_active = 12, cl_waiting = 0, maxwait = 0 -- stable, no backlog. SHOW STATS shows total_xact_count / total_query_count advancing (work is being done) rather than total_wait_time being the only thing that moves.
The pool now has more servers than concurrent clients, so the exhaustion condition (sv_active == pool_size with clients still waiting) can no longer occur at this load.
⏪ Rollback
Nothing here touches data, so there is no data risk. The fix recreates/reloads PgBouncer with a larger DEFAULT_POOL_SIZE; to roll back you simply restore the previous pool size and reload PgBouncer (or recreate the container with the old value). The real hazard is over-correcting: every extra server slot, multiplied across every pool, becomes a real PostgreSQL backend. If you raise DEFAULT_POOL_SIZE past what max_connections can support, new server connections fail with 'sorry, too many clients already'. Roll back by lowering the pool size and confirming SHOW POOLS sv_active fits comfortably under the database's max_connections.
📦 Version Matrix (PG 14--18)
The honest cross-version axis for PgBouncer is pool_mode, not the PostgreSQL major version -- pooling lives entirely in the proxy. Identical load every time: 6 concurrent clients, DEFAULT_POOL_SIZE = 3, each client running short transactions with a one-second idle gap between them. The mode decides what happens during that idle gap.
| Version | Behavior | Version Notes |
|---|---|---|
database |
user | ✗ Not available -- use stats_reset age |
pgbouncer |
pgbouncer | ✗ Not available -- use stats_reset age |
postgres |
postgres | ✗ Not available -- use stats_reset age |
database |
user | ✗ Not available -- use stats_reset age |
pgbouncer |
pgbouncer | ✗ Not available -- use stats_reset age |
postgres |
postgres | ✗ Not available -- use stats_reset age |
database |
user | ✗ Not available -- use stats_reset age |
pgbouncer |
pgbouncer | ✗ Not available -- use stats_reset age |
postgres |
postgres | ✗ Not available -- use stats_reset age |
⚠ Common Mistakes
Stop this incident before it pages you
The free runbook clears today's failure. Pro adds the standing guardrails -- the alert query, the dashboard, and the managed-service knobs -- so it never becomes an incident again.
- ✓ A ready-to-paste alert query with the right thresholds.
- ✓ The topology change that prevents recurrence.
- ✓ The exact knobs on RDS, Azure Flexible Server & Cloud SQL.
Frequently Asked Questions
PostgreSQL looks fine but connections are slow. Where do I look?
At PgBouncer, not Postgres. Connect to the admin pseudo-database ('pgbouncer') and run SHOW POOLS. If cl_waiting > 0 and sv_active equals your DEFAULT_POOL_SIZE, clients are queuing for a server -- the pool is exhausted even though the database itself is idle.
What is the difference between MAX_CLIENT_CONN and DEFAULT_POOL_SIZE?
MAX_CLIENT_CONN is the total number of CLIENT connections PgBouncer will accept (100 in this lab). DEFAULT_POOL_SIZE is the number of SERVER connections each (db, user) pool may open to PostgreSQL (5 in this lab). When more clients want a server than the pool has, the extras sit in cl_waiting. Raising MAX_CLIENT_CONN lets more clients queue; only raising DEFAULT_POOL_SIZE adds servers.
Should I use session, transaction, or statement pool mode?
Transaction mode gives the best multiplexing for typical web/API workloads: the server is returned to the pool at COMMIT, so idle-between-transaction clients do not hold a server. Session mode pins a server to each client for the whole session (needed if you rely on session state like SET, prepared statements, advisory locks, or LISTEN/NOTIFY) but it makes pool exhaustion far easier. Statement mode is the most aggressive (releases after every statement) and forbids multi-statement transactions.
How high should I set DEFAULT_POOL_SIZE?
To your real peak concurrency for that pool, not higher. Every server slot is a live PostgreSQL backend, so the sum of all pools' server connections must fit under max_connections with room to spare. In this lab raising it from 5 to 20 cleared a 7-client backlog; pushing it to thousands would just move the bottleneck to the database and risk 'too many clients already'.
Does the backend PostgreSQL major version change any of this?
No. Connection pooling lives entirely in PgBouncer; the SHOW POOLS / SHOW CLIENTS / SHOW SERVERS columns and the pool-size mechanics are identical whether the backend is PostgreSQL 14 or 18. That is why this recipe's cross-version axis is pool_mode, not the Postgres version.