The answer is very much, yes! I'm teaching myself eval statistics using ham exams taken by Claude LLMs as my demonstration vehicle. The picture below shows the disttribution of correct vs incorrect questions Claude Haiku and Sonnet on the extra class exam. The tiny text to the left of the rows is the test subelement, E0A, E1A, E1C, etc... Expect to see much more on this in the future. For a complete look at the data with statistics I can't promise are meaningful yet, see the initial report .