A recent study by Google highlights significant shortcomings in the current methodologies used for AI benchmarking. It reveals that the conventional practice of employing three to five human raters per test example may not suffice for achieving reliable results. The findings suggest that the allocation of annotation budgets is crucial, emphasizing that how these resources are divided can be as important as the total budget itself. This raises important questions about the validity of AI assessments and their implications for the development of more robust AI systems. In this context, it is important to note that understanding human disagreement is essential for refining AI evaluation processes. A final assessment of these findings would be premature at this point, as further research is needed to explore the broader implications for AI deployment across various sectors.