All About Parquet Part 09 - Parquet in Data Lake Architectures
Explores why Parquet is the ideal columnar file format for optimizing storage and query performance in modern data lake and lakehouse architectures.
Alex Merced — Developer and technical writer sharing in-depth insights on data engineering, Apache Iceberg, data lakehouse architectures, Python tooling, and modern analytics platforms, with a strong focus on practical, hands-on learning.
418 articles from this blog
Explores why Parquet is the ideal columnar file format for optimizing storage and query performance in modern data lake and lakehouse architectures.
A practical guide to reading and writing Parquet files in Python using PyArrow and FastParquet libraries.
Using GitHub Actions to trigger Airflow DAGs for orchestrating data pipelines across Spark, Dremio, and Snowflake.
Explores using GitHub Actions for software development CI/CD and advanced data engineering tasks like ETL pipelines and data orchestration.
A guide explaining dbt macros, their purpose, benefits, and how to use them to write reusable, standardized SQL code in data transformation projects.
Quarterly roundup of data lakehouse trends, table formats, and major industry news from Apache Iceberg to Delta Lake.
A tutorial on using PyArrow for data analytics in Python, covering core concepts, file I/O, and analytical operations.
A comprehensive guide to using Rust's built-in collection types, including vectors, arrays, hashmaps, and sets, with performance tips and examples.
A guide to performing data operations using PySpark, Pandas, DuckDB, Polars, and DataFusion within a pre-configured Docker environment.
Explains how to implement access control and security for Apache Iceberg tables at the file, engine, and catalog levels.
A comprehensive directory of Apache Iceberg resources, including tutorials, guides, and educational materials for data engineers and developers.
Explores how combining data lakehouse, virtualization, and mesh architectures with Dremio solves modern data scaling and silo challenges.
A comprehensive guide to building interactive data applications using the Streamlit framework, covering setup, visualization, ML integration, and deployment.
A comprehensive guide to Docker Compose, covering file structure, service configuration, networking, volumes, and best practices for multi-container applications.
A comprehensive guide to string handling in Rust, covering types, conversions, operations, and performance best practices.
An introductory guide to Rust, covering its key features like memory safety, ownership, and setup for developers new to the language.
A hands-on tutorial for building a Data Lakehouse on your laptop using Apache Iceberg, Spark, Nessie, Minio, and Dremio.
Explains why data professionals should adopt Dremio and Apache Iceberg for flexible, high-performance data lakehouse architecture.
Explores five key trends shaping the data lakehouse architecture, including storage evolution, table formats, and catalog competition.
A guide on using the alexmerced/datanotebook Docker image for a quick data notebook environment with pre-installed libraries like pandas, Polars, and PySpark.