A connection pool is a queue you forgot to size

The claim

The default database connection pool in most application frameworks is set to a number someone picked because it looked reasonable, and it is almost always wrong in the same direction: too large. A pool bigger than the database can usefully serve does not add capacity — it converts a manageable queue in your application into an unmanageable stampede at the database, and the failure it produces looks exactly like the database being slow, which sends teams shopping for a bigger instance to fix a problem that a smaller number would have solved.

Why bigger is not more

A PostgreSQL server can only do so much work in parallel, and that limit is set by CPU cores and disk, not by how many connections are open. Past the point of saturation, adding connections does not add throughput; it adds contention. Each connection competes for the same cores, the same locks, the same disk, so the work does not finish faster — every query just takes longer, and the total completed per second actually drops.

The rule of thumb that has held up for years is startling to people the first time they see it: the useful number of active connections is close to the number of CPU cores, plus a small allowance for disk wait. For a 4-core database, an effective working set is not 100 connections; it is closer to a dozen. A pool of 100 against that server does not serve 100 requests faster — it serves them all slower, at once, while the server thrashes.

The arithmetic across your fleet

The trap is multiplicative and teams miss it because each application looks reasonable on its own. Suppose you run 6 application instances, each configured with a pool of 20. That is not 20 connections to the database; it is 120. If the database is configured with max_connections = 100, you are oversubscribed before the seventh instance even starts, and the symptom is connection-refused errors that appear only under load and vanish when you look.

instances x pool_size = total connections opened
6 x 20 = 120  ->  exceeds max_connections = 100

The fix is to size the pool per instance so the sum stays comfortably under the database's limit, leaving headroom for administrative connections and for the brief overlap during a deploy when old and new instances both hold connections.

Move the wait to where you can see it

Here is the counterintuitive part: a smaller pool is not slower, it is more honest. With a pool of 12, when 40 requests arrive at once, 12 run and 28 wait in your application's queue — a queue you can measure, alert on, and reason about. With a pool of 100, all 40 hit the database at once, the database bogs down, and every request including the ones that would have been fast is now slow. The work does not get done faster in the second case; the queue just moved to a place you cannot see it, inside the database, where it degrades everything instead of the excess.

# the metric that matters:
pool_acquire_wait_time   # how long requests wait for a connection
# if this is rising, the pool is too small OR queries are too slow.
# adding connections only helps the first case, and only up to saturation.

A pooler in front changes the maths

When you genuinely have many application instances, the answer is not a bigger pool per instance but a dedicated connection pooler — PgBouncer is the standard — sitting between the applications and the database. In transaction-pooling mode it lets hundreds of application connections share a small number of real database connections, because most application connections are idle at any given moment, holding a slot they are not using.

# pgbouncer.ini
pool_mode = transaction
max_client_conn = 1000      # apps can open this many
default_pool_size = 20      # but only this many reach Postgres

This is the architecture that lets a small database serve a large fleet: the applications think they have a thousand connections, the database only ever sees twenty, and the pooler absorbs the difference. Transaction mode has one constraint worth knowing — it does not support session-level features like prepared statements held across transactions or advisory locks — so confirm your application does not depend on those before switching.

How to size it in practice

1. Find the database's real parallelism. Start near (2 x cores) + effective spindle count as the target for concurrent active queries. On a 4-core SSD-backed instance, that is roughly 10 to 12.
2. Divide across instances. Total pool across all app instances should sum to that target, not exceed it. Six instances sharing a target of 12 means a pool of 2 each, which feels tiny and is correct — add a pooler if that is too tight.
3. Leave headroom in max_connections. Set the database limit above your total pool, with room for deploys, migrations, and a human with psql during an incident.
4. Watch the acquire-wait metric, not the connection count. Rising wait time with the database CPU not yet saturated means the pool is genuinely too small; rising wait time with the database already pinned means the queries are the problem, and more connections will make it worse.

The decision

Before you scale the database instance because it feels slow under load, check the connection arithmetic. Multiply your instance count by your per-instance pool size and compare it to the database's core count and connection limit. If the product is several times the number of cores, the database is not too small — the pool is too large, it is drowning a capable server in contention, and the fix costs nothing but a smaller number in a config file.