AI Has the Opposite Data Problem
Argues that AI's real challenge isn't data scarcity, but the vast amount of generated data that goes unanalyzed, presenting an opportunity for AI.
Argues that AI's real challenge isn't data scarcity, but the vast amount of generated data that goes unanalyzed, presenting an opportunity for AI.
A blog archive listing posts about data visualization, statistical analysis, and data science using the R programming language.
A developer saves their company $4,500/month by replacing an expensive Oracle tool with a free R script for data analysis and visualization.
A lecture on the foundational statistical concept of orderings and ordinal data, exploring their analysis and complications in fields like health research.
The author discusses updates to gssrdoc, an R package that provides integrated help documentation for the General Social Survey (GSS) dataset.
A data-driven analysis of LLM performance on a simple retrieval task, highlighting the need for evidence-based AI testing.
Analyzing pedestrian fatality data using polar coordinate visualizations to reveal cyclical patterns in daily accident counts.
A technical exploration of the ICMM's global mining dataset, detailing the setup, tools, and process for data analysis using Python, DuckDB, and QGIS.
An analysis of Statistics Canada's Open Database of Buildings (ODB) dataset, covering data processing, tools used, and technical setup.
An analysis of Canada's new national building footprint dataset, exploring its sources, technical setup, and initial processing steps.
A statistical reasoning test with three practical problems on sorting uncertain fractions, highlighting anomalies, and estimating population sizes.
Argues that reading raw AI input/output data is essential for developing true intuition about system behavior, beyond just metrics.
Explains the statistical concept of included-variable bias in regression models, challenging the traditional 'omitted-variable bias' framing.
Argues that effective AI product evaluation requires a scientific, process-driven approach, not just adding LLM-as-judge tools.
A technical analysis using R to classify iris images from a dataset, applying PCA and LDA for machine learning classification.
A tutorial on using pandas to calculate scoring streaks or runs in basketball data, demonstrating data manipulation techniques.
Explains the key differences between the = and <- assignment operators in the R programming language, focusing on scoping and side effects.
A hands-on review of the new DuckDB UI, exploring its features for data analysis and comparing it to previous workflows with Rill Data.
A technical walkthrough of loading and exploring UK Environment Agency flood data using DuckDB and Rill for a streaming pipeline project.
A tutorial for R users on mastering data wrangling in 5 progressive levels, using the dplyr package and the Ames housing dataset.