REST Catalog V2: Fixing Iceberg Protocol Debt

This article examines the evolution of the Apache Iceberg REST Catalog protocol, focusing on the need for a V2 design to address protocol debt caused by changing workload patterns. It discusses how the original V1 protocol, designed for fewer clients with larger queries, now struggles with overhead from BI dashboards, AI agents, and semantic layers that trigger dozens to hundreds of catalog calls per query. Key topics include batch table loading, caching and metadata freshness, stateless catalog servers for serverless compute, multi-engine optimization without vendor lock-in, and migration plans for existing REST endpoints. The article positions these improvements as strengthening the Agentic Lakehouse architecture, emphasizing that the gap between planning cost and execution cost has become the primary bottleneck in modern data lakehouse environments.