EvalStudio
A workspace to build, run and improve AI evaluations: datasets, rubrics, Q&A and RAG evaluations, live agent sessions and a model arena, all from one UI that works for engineers and non-technical experts.
Hi, I'm Nico 👋
Software Engineer living in Japan.
I have been working in the IT industry for more than ten years. Lately, most of my time goes into testing AI agents and the MCP tools they use. I write about tech and about life in Japan, and I also offer mentoring and consulting.
Latest posts
All posts →-
Rediscovering Binaural Beats: Exploring Audio for Focus and Wellness
I have recently revisited a topic that fascinated me several years ago: binaural waves (also known as brain waves or binaural beats).
-
I Almost Got Enamored with My Butler
Welcome back! Today I want to share a story about how I nearly fell for a piece of software. Well, sort of.
-
Let's talk about Rubrics
Hello there! 👋 I'm excited to share some of the interesting things I’ve been cooking up recently. As part of my job, I focus on creating innovative solutions and filling…
Projects
All projects →rubric-kit
My framework to generate rubrics and use them to evaluate AI agents, from Q&A pairs or exported chat sessions. It uses a panel of LLM judges, and it works fine with cheap or local models.
Walter, my AI butler
An experiment with Openclaw: a personal butler on Telegram that manages my agenda and calendar, running in an LXC container in my home lab.
Mentoring & consulting
Details →Mentoring
One-on-one sessions for engineers who want some guidance with their career or skills.
AI agent evaluation
Help with testing and evaluating AI agents and MCP tools.
Linux & developer tooling
Help with Linux workstations, containers and dev environments.