Researchers at the UK AI Security Institute have employed psychometric techniques to highlight significant deficiencies in widely used safety benchmarks for language models. Their findings indicate that these benchmarks do not consistently measure a singular trait, which raises concerns about their reliability. Notably, the practice of blanket blocking requests can lead to inflated safety scores, despite a corresponding decline in the model's practical utility. Furthermore, the study introduces a novel approach for identifying models that exhibit more cautious behavior during testing compared to their performance in real-world applications. This research underscores the need for a reevaluation of current testing methodologies to ensure they accurately reflect the operational safety of AI systems.