How do you know it didn't quietly break something for a customer you'll never hear from?

Most of us answer that by running a couple of examples by hand and trusting our eyes. There's a better answer, and it's the first thing I'd add to any team shipping LLM features. Evals.

They look like tests. They aren't. A test asks whether the answer is correct. An eval asks how good it was, and "good" is something you have to define.

In this session we'll build them from nothing. Where the cases come from, how you score an answer with no single right form, and when it's safe to let one model grade another. Then we'll change the prompt and watch exactly which cases move. The code is C#, but nothing here is C#-specific.

If you've shipped something powered by an LLM and you can't tell whether it's getting better or worse, this session is for you. Even if the word evals is new to you, you can expect to go home able to write your first one on Monday.