Most engineering teams underestimate database connections.
Queries get attention. Indexes get attention. Schema design gets attention. But the connections between your app and the database?
They break first, cause the worst tail latencies, and trigger the nastiest outages.
When we were building our Go backend on PostgreSQL and AWS RDS, we assumed connection handling was “solved.” Go has a built-in connection pool. Postgres is mature. RDS Proxy handles multiplexing.
Reality: it took us months of iteration — and a few painful incidents — to build a connection layer that was truly resilient.
This is the story of that evolution, told in a way that you can reuse for your own systems.
Background
Before diving into the story, here’s the relevant snapshot of our stack:
- Language: Go
- ORM: GORM (for model mapping + ergonomics)
- Driver & Pooling: pgx + pgxpool (for control + performance)
- Database: PostgreSQL hosted on AWS RDS
- Connection Infra: AWS RDS Proxy
- Runtime: Mix of long-running ECS services + short-lived Lambda functions
1.The Naive Beginning: database/sql Defaults
Our first version used:
- the Go standard library (
database/sql) - default settings
- one pool for everything
- no reader/writer separation
On paper this looked fine. Under load it fell apart.
Issues we hit immediately:
- frequent reconnections
- spikes in p95/p99 latency
- connection exhaustion on traffic bursts
- unpredictable behavior during deployment rollouts
The root cause was simple:
The standard library’s pool is very limited.
- No
MinConns. - No health checks.
- Minimal insight into connection states.
With the existing settings, it opens connections lazily and drops them aggressively.
This led us into our first “upgrade”… except it wasn’t really an upgrade.
1.5. The Hacky Connection Warm-Up
Before switching to anything new, we tried to work around the limitations of database/sql.
The biggest missing feature was MinConns.
Our app frequently started with zero open connections, causing a thundering herd of new connections during traffic spikes.
So we wrote a warm-up hack:
for i := 0; i < maxConns; i++ { _, _ = db.QueryContext(ctx, "SELECT 1") }
This forced the pool to open multiple connections early during the application startup.
Why this wasn’t a real upgrade
It was a patch, not a solution and that too a very limited one. It was limited to application startup along with other shortcoming.
It didn’t fix the deeper issues:
- idle timeouts still killed connections
- the pool could still shrink to zero
- behavior varied across environments
- it masked the real problem instead of solving it
This phase was important because it taught us what we truly needed: predictable connection behavior, not hacks.
That realization moved us to the real next step.
2.Switching to pgxpool: Real Control at Last
Migrating to pgxpool was the first time our database layer started behaving like a production system.
With pgxpool, the essentials were finally in our control:
MinConns // maintain warm pool
MaxConns // cap peak usage
MaxConnIdleTime // prevent churn HealthCheck // keep connections alive
Three immediate improvements:
- Warm pools stayed warm. No more reconnect storms during traffic bursts.
- Deployment rollouts no longer caused latency spikes. New tasks pre-initialized connections.
- Idle timeout was no longer a silent killer. We tuned idle time to avoid constant churn.
This alone solved a huge class of problems — but we uncovered a new one.
3.A Subtle GORM Behavior We Learned Along the Way
As our system matured, we eventually learned something about how GORM handles database connections that isn’t obvious at first glance.
In our early setup, we passed a plain DSN into GORM:
setup db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
This works perfectly fine — but what we didn’t realize initially is that:
GORM creates and manages its own internal connection pool.
It wasn’t causing issues or errors, but it did mean:
- we were maintaining two sets of connection pools
- resource usage was a bit higher than needed
- connection behavior wasn’t fully unified across the system
Nothing broke — it was just not the most efficient setup once we started caring about pool tuning.
The fix was simply letting GORM use the same pool we manage with pgxpool
// New setup
sqlDB := stdlib.OpenDBFromPool(pgxPool) db, err := gorm.Open(postgres.New(
postgres.Config{ Conn: sqlDB, // GORM now uses our pgxpool
PreferSimpleProtocol: true, // optional, but helps performance
}))
After this change:
- all connections came from a single pool
- tuning became consistent
- resource usage became more predictable
A small detail — but one that makes the entire architecture cleaner. one pool, one lifecycle, full visibility.
4.Splitting Reader and Writer Pools: The Scaling Unlock
Next up, is an obvious and natural one. But we did not do this because we ran into scaling issue on read queries.
When we procured the AWS RDS instance, we procured a reader instance along with the writer. Anticipating that we will, at some point, have to split out the traffic to reader and writer instance, we did it pre-maturely.
So we made the architectural shift:
Two independent pools: Reader and Writer
Each with its own:
- DSN
- pgxpool instance
- tuning strategy
- monitoring panel
This immediately gave us:
- stable write performance
- better horizontal scaling (especially with replicas)
- clear observability: “read pool is saturated” ≠ “write pool is saturated”
5.Service-Specific Pool Configurations
Different workloads need different pool strategies.
We had a single golang db management implementation that was shared by multiple binaries. We started with a common configuration for all binaries. However, we soon realized that the workload patterns are different from these binaries. We had long running services serving user requests synchronously. Connection pools are very important in these situations to serve user requests quickly. We also had short-lived lambdas that did background work asynchronously. Although, our lambdas could server multiple request in their lifetime, these use-cases are not latency sensitive.
Long-running services (Fargate/ECS)
MinConns = high
MaxConns = high
IdleTime = moderate
Warm pools drastically improve latency.
Short-lived services (Lambda consumers)
MinConns = 1
MaxConns = low
IdleTime = near zero
They shouldn’t hold 25 idle connections between invocations.
Pool configuration is not a one-size-fits-all concept. It’s tied to your service’s shape and runtime environment.
6.The RDS Proxy Two-Level Pool Problem
With multiple binaries talking to the same database instances, we realized we needed a protection layer for our db. The protection layer will shield the db from connection storms and putting a cap on the db’s resource utilization.
The native AWS way to do this is RDS Proxy. Another popular solution is pgBouncer. Since, we wanted low recurring maintenance cost for our engineering team, we decided to go with RDS Proxy.
This definitely introduced more complexity. Connections now flowed through two layers of pooling:
service → pgxpool → RDS Proxy → PostgreSQL
You now have to think about:
- idle timeouts interacting
- which layer “owns” connection health
- how warm pools behave across layers
- how many actual DB connections RDS Proxy maintains
Our conclusion for a high-throughput Go service:
- Keep pgxpool for predictability.
- Use RDS Proxy for multiplexing.
- Tune both intentionally
- Removing pgxpool made the system fragile.
- Relying only on RDS Proxy removed application-level control and poor performance for end user experience
7.Ubiquitous observability
There isn’t any replacement for observability. Once we exposed pgxpool metrics and traced connections, everything snapped into place.
You can’t fix what you can’t see.
stats := pool.Stat()
This made tuning scientific instead of guesswork. We could see:
- pool saturation
- wait time on acquire
- new vs reused connections
- canceled acquires
- idle churn
- connection spikes
- reader vs writer pool imbalance
Takeaways for Anyone Building a Production Go + Postgres System
These lessons came directly from real production incidents and real scaling pain:
Use pgxpool early
The standard library pool is too limited for production systems.
Unified connection pool GORM-PGX
Otherwise you will unintentionally maintain two pools.
Split reader and writer pools
This improves performance and observability.
Tune pgxpool and RDS Proxy together
Treat them as a two-layer system.
Tune pools per service type
Different workloads have different usage pattern and generate different resource consumption.
Add observability
Without pool metrics, you are operating blind.