Can Postgres Really Replace Redis, Kafka and Elasticsearch?
SystemSketch
0:00 / 0:00
Can Postgres Really Replace Redis, Kafka and Elasticsearch?
26 просмотров · 11 дн. назад
SystemSketch
16 подписчиков
26 просмотров · 11 дн. назад
Should you just use Postgres for everything? OpenAI serves 800 million ChatGPT users with a single Postgres primary and about 50 read replicas, yet the same engineering post says no new tables are allowed on that database. Both halves of that story matter.
This video breaks down why Postgres can replace a queue, a document store, a search engine and a vector database: extensible types and index types (GIN, HNSW), extensions like pgvector, PostGIS, TimescaleDB and pg_cron, and one shared transaction system that removes the dual-write problem. You'll see a real SKIP LOCKED job queue, JSONB with GIN, full-text search with tsvector and vector search with pgvector. Then we look at where it breaks: the single-writer ceiling (OpenAI's 10x write spike), MVCC bloat in busy queues, and Uber's write amplification. It ends with the scaling path (PgBouncer, replicas, workload isolation, Notion's 480 shards) and a three-question decision tree.
Made for developers, architects and tech leads deciding whether a second datastore is really worth it.
Chapters
0:00 800M users, one Postgres primary
0:42 The six-database stack
1:52 Why Postgres can do this
2:40 The swap tour: queue, JSONB, search, vectors
4:09 Where it breaks
5:47 The scaling path
6:32 Should you just use Postgres?
Sources
1. OpenAI Engineering, Bohan Zhang (Jan 22, 2026) — Scaling PostgreSQL to power 800 million ChatGPT users: https://openai.com/index/scaling-post...
2. Notion Engineering (2021) — Herding elephants: lessons learned from sharding Postgres at Notion: https://www.notion.com/blog/sharding-...
3. Uber Engineering, Evan Klitzke (2016) — Why Uber Engineering Switched from Postgres to MySQL: https://www.uber.com/blog/postgres-to...
4. PostgreSQL docs — SELECT, The Locking Clause (FOR UPDATE SKIP LOCKED): https://www.postgresql.org/docs/curre...
5. PostgreSQL docs — JSON types and jsonb indexing: https://www.postgresql.org/docs/curre...
6. PostgreSQL docs — Full text search: https://www.postgresql.org/docs/curre...
7. PostgreSQL docs — Routine vacuuming: https://www.postgresql.org/docs/curre...
8. pgvector (HNSW index): https://github.com/pgvector/pgvector
What's your threshold for adding a second datastore, and what metric forced it? Tell me in the comments. Subscribe to @systemsketchhq for more system design breakdowns.
#PostgreSQL #Postgres #SystemDesign #Database #SoftwareArchitecture #BackendDevelopment #pgvector #DistributedSystems #SQL #DataEngineering #SoftwareEngineering #Programming #Coding #TechEducation