What Are Lakehouse Catalogs? The Role of Catalogs in Apache Iceberg
Explains lakehouse catalogs in Apache Iceberg, their role in metadata management, and how to choose between open source and managed options.
Explains lakehouse catalogs in Apache Iceberg, their role in metadata management, and how to choose between open source and managed options.
Explains five ways Apache Iceberg table storage degrades over time, including small files, orphan files, and metadata bloat, with detection methods.
Explores embedding Iceberg catalogs directly into storage, covering AWS S3 Tables and MinIO AI Stor for simplified metadata management.
Guide to maintaining Apache Iceberg tables with compaction, snapshot expiry, orphan cleanup, and manifest rewriting for optimal performance.
Explains Apache Iceberg metadata tables for querying table internals using SQL, covering snapshots, files, manifests, partitions, and practical use cases.
Guide to using Apache Iceberg with Python libraries (PyIceberg, DuckDB, Polars) and MPP query engines like Dremio, Spark, and Trino.
Explores three streaming architectures for Apache Iceberg: Spark Structured Streaming, Flink, and Kafka Connect, focusing on trade-offs between latency and table maintenance.
A practical walkthrough of working with Apache Iceberg on Dremio Cloud, covering table creation, data ingestion, optimization, and AI-powered analytics.
Strategies for migrating data to Apache Iceberg, including in-place, full rewrite, and shadow migration with zero downtime.
Explains Apache Iceberg, a table format that replaces directory-based metadata with file-level tracking for scalable analytics on cloud storage.
Apache Polaris is an open-source catalog service that unifies the Iceberg ecosystem by implementing the Iceberg REST API for vendor-neutral lakehouse metadata management.
Explains the modular Apache Lakehouse architecture using open-source components like Parquet, Iceberg, Polaris, and Arrow for vendor-neutral data management.
A guide to integrating Dremio's data lakehouse with the OpenWork AI agent for local, secure data querying and application development.
Guide on connecting Azure Storage to Dremio Cloud to query data lakes with SQL and AI, avoiding costs of Azure Synapse.
Guide on connecting AWS Glue Data Catalog to Dremio Cloud for querying and managing AWS Iceberg tables with full DML support and federation.
Guide on connecting Snowflake Open Catalog to Dremio Cloud for multi-engine Apache Iceberg analytics, federation, and cost optimization.
Guide on connecting any Apache Iceberg REST Catalog to Dremio Cloud for universal lakehouse data access and management.
Introduces Dremio's built-in Open Catalog for Apache Iceberg, offering a zero-configuration, production-ready lakehouse solution with automated management.
Explores how data modeling principles adapt for modern lakehouse architectures using open formats like Apache Iceberg and the Medallion pattern.
A 2025 year-in-review of key Apache data projects: Iceberg, Polaris, Parquet, and Arrow, detailing their major updates and future roadmap.