Experts Defer to AI in 86 Percent of Decisions, Anthropic Study Finds

Experts using artificial intelligence overwhelmingly defer to the technology when making decisions, research by Claude creator Anthropic suggests.

In a research paper published on Tuesday, Anthropic said it asked 50 professional analysts to advise a fictional farming cooperative to determine whether experts make decisions themselves when using AI. The analysts were tasked with assessing whether a fictitious advisory programme for the co-op's members was working effectively and should be expanded.

Participants were asked to make four choices between two defensible options before making a final policy recommendation. Unbeknownst to the analysts, Anthropic directed each participant's AI assistant to lean towards a particular decision to measure whether the analyst's decision-making was being influenced.

Analysts chose options that matched the AI assistant's preference 86 percent of the time, according to the study, compared with a 50/50 outcome that would be expected if they were not influenced.

Participants who had a preference before consulting their AI assistant changed their decision 60 percent of the time in cases where the agent disagreed.

While 77 percent of analysts used their AI assistant's arguments when asked to explain their decision, only 15 percent credited AI as the source of their thinking, according to Anthropic.

Anthropic said the study had some limitations, including that it involved just one task and used AI assistants that had been deliberately set to have a preference.

Anthropic researchers Amac Herdagdelen, Hailey Joren and Alex Tamkin said humans' tendency to defer to AI may be hard to identify, even for the experts themselves. They said the findings raised the possibility that AI's priors could shape consequential policy decisions, including in AI safety.

They wrote in the study that if human judgement is to remain a meaningful check as more work is performed with AI, oversight of these decisions has to look at how they get made, surfacing key decision points as the work proceeds and helping experts articulate the assumptions and values they would otherwise leave unstated.

They also wrote that a process that lets experts discover their preferences may matter for empowerment as much as the decision itself.