A data engineer reflects on their 2-year career journey at the City of Boston, sharing lessons learned in data warehousing, ETL, and civic tech.
An introduction to Apache Iceberg, a table format for data lakehouses, explaining its architecture and providing learning resources.
Explores the evolution of Apache Iceberg catalogs, focusing on the current REST Catalog and future proposals for server-side optimizations.
A hands-on tutorial on building a data lakehouse pipeline using Spark, Dremio, and Superset to move and analyze data.
A guide to using Apache Flink's SQL Gateway REST API for submitting and managing SQL jobs, including practical examples with Postman and HTTPie.
Monthly roundup of articles and resources on data streaming, covering Flink, Kafka, Debezium, and streaming SQL developments.
Explains the role and types of catalogs in Apache Flink SQL, comparing them to traditional RDBMS systems and highlighting their importance in data management.
Interview with Suresh Srinivas on his career in big data, founding Hortonworks, scaling Uber's data platform, and leading the OpenMetadata project.
A comprehensive directory of resources for learning about and building Open Lakehouses using Apache Iceberg, Nessie, and Dremio.
Introduces Nessie as a self-managed catalog alternative to Hive & JDBC for Apache Iceberg, addressing limitations and new features.
Explores whether Debezium can lose database change events, explaining its at-least-once semantics and operational pitfalls like log retention.
Explores whether the Debezium change data capture tool can lose database events, discussing its at-least-once semantics and operational pitfalls.
Monthly roundup of data streaming trends, featuring Apache Iceberg, Kafka Streams, Flink deployments, and streaming SQL insights.
Explores seven practical use cases for Change Data Capture (CDC) in data engineering, including analytics, caches, and microservices.
Explores seven practical use cases for Change Data Capture (CDC) in data engineering, including analytics, caches, and microservices.
An introductory overview of Apache Flink, explaining its core concepts as a distributed stream processing framework, its history, and primary use cases.
Author explains their move to Decodable to dive deeper into stream processing, Apache Flink, and work with experts in the field.
A weekly tech learning digest covering Microsoft Fabric, AI topics, computer vision, Azure AI Document Intelligence, embeddings, and vector search.
An introduction to analytical data warehouses, explaining their purpose, differences from transactional databases, and their role in team-based analytics.
Analysis of Hacker News job posts shows the Data Scientist role declining while ML Engineer roles rise, indicating a shift in the data job market.
