Writing My First Evals
A developer details the process of building evaluation systems for two AI-powered developer tools to measure their real-world effectiveness.
Nick Nisi is a Developer Experience Engineer at WorkOS focused on TypeScript and developer tooling. He’s a JS Party podcast co-host, conference organizer, and frequent speaker sharing practical insights on modern JavaScript.
26 articles from this blog
A developer details the process of building evaluation systems for two AI-powered developer tools to measure their real-world effectiveness.
Introducing the Ideation skill for Claude Code, a structured pipeline for turning vague ideas into actionable, high-confidence technical plans.
Developer shares custom Claude Code plugins 'Essentials' and 'Ideation' that use AI to clean code and structure ideas while preserving personal style and judgment.
Explores the potential of Claude Skills, a new feature from Anthropic that uses markdown files to create deterministic, scriptable AI tools within Claude's ecosystem.
A developer's journey from skepticism to embracing AI coding assistants like Claude, discovering they act as collaborative pair programmers rather than replacements.
A developer experiments with 'vibe coding' using AI to build a GitHub Action without looking at the generated code, exploring AI's role in software development.
A developer reflects on a year of professional growth, including work at Meta, conference speaking, and starting a new role at WorkOS.
A developer reflects on their experience joining and leaving Meta, covering the company's unique dev tools, internal culture, and the reality of working on massive codebases.
A personal reflection on the end of the JS Party podcast, its community impact, and lessons learned over seven years.
JS Party podcast hosts announce their attendance at React Summit 2024, a major React developer conference in NYC/Jersey City.
Announcement about co-emceeing SquiggleConf, a web development conference focused on tools and practices in Boston, October 2024.
A conference speaker and organizer shares tips for preparing and submitting compelling tech talk abstracts, focusing on storytelling and structure.
A developer shares their evolving tooling setup, including terminal emulators, dotfiles, and workflow automation with tools like Homebrew and Neovim.
An introduction to codemods using jscodeshift to automate and standardize code changes across a codebase.
A code review of the Obsidian Clipper plugin, focusing on its TypeScript and Svelte implementation for clipping web content into notes.
A guide to using Git worktrees for managing multiple branches in separate directories, improving developer workflow.
A guide to using Git's includeIf directive to automatically set different user emails for work and personal projects based on directory.
Using tmux to manage separate displays for live coding presentations, reducing stress by syncing projector and local screens.
A developer shares their experience switching to Neovim, highlighting its async plugin benefits and providing basic setup instructions.
A talk and guide on using Vim and Tmux together for development, covering basics, plugins, and configuration for productivity.