Quacks, psychics and LLMs

This article discusses the author's deep dive into DuckDB and DuckLake, two tools for analytics and data management. DuckDB is described as a column-oriented, in-process database system ideal for running analytics on various file types like CSV, JSON, and Parquet, with a spatial extension noted. DuckLake is presented as a simpler alternative to traditional data lakes, using a metadata database over Parquet in blob storage to enable multiplayer access, change feeds, and time travel. The author draws parallels to their own experience building a lightweight data catalog at Natcap, highlighting the appeal of these tools for simplifying data infrastructure.