The Evolution of the SQL Query Optimizer
How Cost-Based Optimization Shaped Modern Data Systems

About this book:

Every SQL query you run triggers a hidden miracle: in milliseconds, a declarative request becomes an optimal physical plan spanning indexes, joins, and distributed data shuffles—or it doesn't, and your cluster stalls for hours. *The Evolution of the SQL Query Optimizer* tells the fifty-year story of how that miracle was engineered, from the pointer-chasing nightmares of IMS and CODASYL through the breakthrough of System R's cost-based architecture, the rule-driven extensibility of Volcano and Cascades, and into today's world of learned cardinalities, adaptive re-planning, and autonomous cloud optimizers. You'll see why join enumeration is NP-hard, how histograms evolved into HyperLogLog sketches, and why vectorized execution and hardware-aware cost models are rewriting the rules again.

Written for systems engineers, database architects, and curious practitioners who refuse to treat the optimizer as a black box, this book connects the mathematical foundations—relational algebra, dynamic programming, statistical estimation—to the engineering trade-offs that decide whether your analytical workload finishes in seconds or days. Chapters trace the lineage from Selinger's seminal cost formulas through star-schema OLAP, SQL-on-Hadoop, schemaless document optimization, and the emerging frontier of ML-driven plan selection. Each chapter builds the mental model you need to read an EXPLAIN plan with confidence, diagnose cardinality-estimation disasters, and understand why the optimizer chooses a hash join over a merge join—or when to override it.

Whether you're tuning a latency-critical OLTP query, designing a petabyte-scale lakehouse, or building the next generation of data-intensive systems, *The Evolution of the SQL Query Optimizer* gives you the deep, practical history that turns opaque magic into transparent, controllable engineering.