Data Lakehouse Open Standards for AI Agents
Explores how open standards like Iceberg and REST catalogs prevent data silos for AI agents in lakehouse architectures.
Alex Merced — Developer and technical writer sharing in-depth insights on data engineering, Apache Iceberg, data lakehouse architectures, Python tooling, and modern analytics platforms, with a strong focus on practical, hands-on learning.
610 articles from this blog
Explores how open standards like Iceberg and REST catalogs prevent data silos for AI agents in lakehouse architectures.
Explains a five-layer architecture for safe agentic analytics, preventing agents from directly querying raw storage.
Explores how GSA MCP servers make federal open data AI-ready, bridging the gap between public datasets and agent-friendly interfaces.
Designing a hybrid lakehouse for regulated markets where data cannot move due to residency or sovereignty laws.
Explains Iceberg v3 deletion vectors and merge-on-read for efficient DML on data lakes, reducing write amplification.
Explains why a native variant type in Apache Iceberg is needed for semi-structured AI data like LLM outputs and agent logs.
Explores how AI agent write patterns stress Apache Iceberg tables and offers patterns like partition isolation and commit queues to maintain performance.
Explains why stateless MCP gateways are essential for scaling data agents, covering deployment, credential delegation, and guardrails.
Explains why autonomous AI agents need a policy layer for security, covering query limits, egress quotas, and enforcement.
Explains credential vending for Iceberg REST catalogs, replacing static cloud keys with short-lived, scoped storage credentials for secure multi-engine lakehouses.
Analysis of Apache Iceberg REST Catalog V2 design addressing protocol debt, scaling challenges, and multi-engine optimization for modern data workloads.
Explores AI-driven semantic view autopilot for data governance, balancing automation with human review to maintain accurate metadata.
Guide to zero-copy mirroring for migrating proprietary data warehouses to open Apache Iceberg tables, assessing conditions and staged migration.
Explores federation vs. lakehouse architectures for unified data access, offering a decision framework and best practices.
An in-depth guide to file compression codecs, covering how data shrinks, codec differences, and practical recommendations for lakehouse architectures.
Analysis of the renaissance in columnar file formats, comparing Parquet, Lance, Vortex, Nimble, and BtrBlocks for modern data workloads.
Explains file encryption for lakehouses, covering Parquet Modular Encryption, Iceberg table encryption, and interoperability challenges.
Explains what open source foundations like Apache, Linux, and Eclipse actually do, focusing on governance differences and their impact on software projects.
Explains the fundamental divide between operational (OLTP) and analytical (OLAP) systems, the physics enforcing it, and the truth about hybrid architectures.
Explores context engineering for AI agents, comparing personal vs. shared context to prevent failures in human and organizational AI use.