About
evalix is a small eval harness that tells you which cases your prompt change broke.
Why
I created evalix because I was tired of building the same loops over and over again to test my prompts. Initially, it was supposed to be a tiny wrapper, but it was growing, I was adding more and more new options. This is what we have now - a pretty decent tool for evaluating prompts quality.