The Rise of Apache Parquet
How a Columnar Format Reshaped Big Data Analytics

About this book:

Every technological revolution rests on quiet infrastructure—the unglamorous plumbing that makes the extraordinary seem ordinary. *The Rise of Apache Parquet* tells the definitive story of how a columnar file format born at Twitter and Cloudera shattered the I/O bottleneck of the Hadoop era, transforming analytical processing from a grinding, disk-bound slog into the interactive, cloud-native standard we rely on today. From the tyranny of delimited text files and the limits of row-oriented giants like Avro, through the "Format Wars" with ORC, to the architectural breakthroughs of definition levels, predicate pushdown, and vectorized execution, this book reveals why Parquet became the invisible lingua franca of global analytics.

Go beyond the "what" to master the "how" and "why." You’ll peel back the byte-level mechanics—metadata footers, dictionary encoding, bit-packing, and the Dremel-inspired algorithm that tames nested data—while tracing the ecosystem dynamics that cemented dominance: Spark’s embrace, the rise of Presto and Trino, the shift to object storage, and the birth of the modern Data Lakehouse (Iceberg, Delta Lake, Hudi). Later chapters tackle the operational realities every senior engineer faces: schema evolution at scale, streaming ingestion bottlenecks, modular encryption, and the cross-language rewrites in Rust and C++ powering the next generation of data tools.

Whether you are a data engineer fighting cloud costs and query latency, an architect designing disaggregated platforms, or a technologist fascinated by how storage shapes compute, this is your essential guide. Understand the format that turned petabytes into interactive insights—and gain the architectural foresight to navigate the next decade of analytical computing.