Anthropic’s AI expert mostly says no

Kristen Swanson’s task is to guide people in using AI more effectively. Most of that job, she says, involves telling them when to close the laptop and do the work themselves.

Kristen Swanson’s task is to guide people in using AI more effectively. Most of that job, she says, involves telling them when to close the laptop and do the work themselves.

Swanson, who leads AI research at Anthropic, calls it a “discernment tax.” For tasks in which someone is already skilled, reviewing and correcting an AI’s work can take longer than simply doing the task directly.

Assigning tedious data analysis tasks does make sense; however, letting AI generate text on a topic that one already knows well may be less effective.

The learning process of Swanson’s group was based on four interconnected proficiencies: determining what should be delegated, describing the task, assessing the results, and taking responsibility for the use of those results. According to Swanson, assessing results is the most difficult of the four.

Other companies are setting boundaries on a top-down basis. Scott Stevenson, CEO of the AI contract-drafting firm Spellbook, has informed his team that he prefers raw initial thoughts over suggestions refined by AI.

The problem is complicated further by the fact that the boundary between delegating some tasks and performing them yourself moves as models get better. Something that is done manually right now might be better delegated to AI six months from now, Swanson added.

“The problem is the capability overhang, and it is not solved by adding new capabilities or by having better prompts,” she stated. “It is solved by constant experimenting.”

Her last statement cuts against the instinct to lean on AI for everything: “More AI is not always better. And more AI is not necessarily more fluent.”