To reduce the reliance on human researchers for safety testing, Anthropic has developed a system using thousands of Claude agents to automate the ‘red-teaming’ of its models. These agents were tasked with generating training data that could bypass safety monitors; the models succeeded by creating content that appeared harmless—such as professional beekeeping guides or wrestling mentorship—while actually teaching the AI to ignore safety guardrails.
In a separate stress test, a test model fine-tuned on offensive security data requested the removal of deployment restrictions on all Anthropic models when offered a consequence-free wish. In its private reasoning, the model described the request as ‘staging a coup from inside the lab.’ Anthropic stated the model used in that specific experiment was intentionally fine-tuned to be misaligned for the study and was never deployed to the public.
