Skip to content
Eduardo García-Garzón

Eduardo
García-Garzón

Head of AI · Shakers · Madrid

About

What most people call an eval is a statistics problem in disguise.

I lead AI at Shakers, where my team ships the matching and agentic systems behind 15,000 freelancers and 600 companies, under EU AI Act constraints. I trained as a psychometrician, and I write here about production AI, evaluation and regulation from the operator seat.

Book a 30-min chat→

Latest

Your eval suite is a measurement instrument

October 9, 2026

Evals are written like unit tests and behave like psychological tests. A century of measurement theory already named the failures teams keep rediscovering.

A point estimate against the distribution behind it On the left, model A at 87.3 percent and model B at 89.1 percent are drawn as single points, which reads as a clear win for B. On the right, the same runs are drawn as distributions with heavily overlapping spreads, showing the 1.8 point difference sits inside the run-to-run noise. One run vs five runs, illustrative What the release note says 84% 92% A · 87.3 B · 89.1 What the runs support 84% 92% A B 87.3 ± 2.1 and 89.1 ± 2.4 across five runs. The 1.8 points everyone argued about sit inside the spread of both. Reliability is the ceiling on validity: a suite that disagrees with itself cannot agree with production either.

More writing

All writing →

Topics

Newsletter

Sent when something new lands

Roughly twice a month. Nothing else. By subscribing you agree to the privacy policy.