← Graph

Big Data Is Usually Smaller Than You Think

concept 1 connections

Heuristic from Stephen Margheim: the number someone has in mind for 'big data' is usually much smaller than what modern Postgres handles comfortably. 30GB is tiny; even hundreds of GB is small. Avoid prematurely optimizing with Elasticsearch, extra indices, or exotic sharding. Hardware speed also keeps growing, so what looks tight today is doubled-capacity tomorrow.

category
practice
about
Big Data Is Usually Smaller Than You Think concept
Stephen pushed back on premature Elasticsearch adoption.

Provenance

Created in
Performance Panel at wroclove.rb 2024 2026-04-17 23:20
Read by
2 extractions