A Rigid AI Workflow Made the Work Worse
A field experiment found that a mandatory paired AI protocol reduced document production and quality, while a lighter thought-partner frame showed a narrower, tentative benefit.
Scope note: This essay covers two bounded interventions tested during one company training day. The study does not prove that structured collaboration is always harmful or that thought-partner training always works.
Organizations want a recipe for using AI. Recipes feel safe. This experiment found one that made the work worse.
Researchers tested two approaches with 388 Gap employees. One required pairs to follow a structured process for discussing a task and drafting with AI. The other gave individuals brief training that framed AI as a thought partner. Everyone had access to the same tool. The surrounding instructions changed.
The mandatory paired process produced fewer documents and lower-scoring work. The thought-partner training showed a possible benefit at the top end of individual work, but that finding was much less certain than the headline might suggest.
The protocol created work about the work
Pairs in the structured condition had to hold a synchronous discussion, feed that material into AI, and draft together through a prescribed sequence. Control pairs could use the tool however they chose.
Only 71 of 97 assigned pairs in the structured group produced a document, compared with 93 of 97 control pairs. Among the documents that were produced, the structured group scored almost five points lower on a 22-point rubric. Their documents were also shorter, and they created fewer versions.
Some of that quality difference was tied to length. The automated grader favored longer documents, and controlling for word count reduced the estimated penalty. Human review still preserved the direction of the difference. The more basic result survived: the protocol made it much harder to finish.
That is an operational failure, even if the idea behind the protocol was sound. Coordination consumed the time that participants needed for the task. More than a third of the treatment pairs with compliance data became stranded and could not carry out the intended process.
A workflow does not help because it contains good verbs. It helps when real people can complete it with the tools, time, and context they have.
Framing may matter, but the evidence is narrow
The second intervention asked individuals to treat AI less like a one-shot answer box and more like a partner they could question and refine.
The main analysis did not find a significant improvement in average document scores. An exploratory analysis did find that trained participants had about twice the odds of producing a top-scoring document. That result held across several high-score cutoffs, but the cutoff was chosen after the researchers saw that most automated scores were packed near the maximum.
This is interesting, not conclusive. A better mental model may help some people reach stronger work. The study does not show that a short reframing exercise lifts everyone, and it does not justify turning “AI as thought partner” into the next mandatory slogan printed on a lanyard.
The useful contrast is lighter. A flexible idea about how to engage with AI may leave room for judgment. A rigid process can replace judgment with ceremony.
The caveats are part of the result
The control group worked in the morning, and the treatment group worked in the afternoon. Time of day, room conditions, fatigue, facilitator differences, or technical load could therefore explain part of the gap. The treatment group also dropped out at a higher rate. The document grader was sensitive to length, and its scores did not match human ratings perfectly.
The authors state these limits clearly. They describe the findings as bounded lessons, not a general verdict on all behavioral and cognitive support.
That restraint improves the paper. Enterprise teams should copy it.
Pilot the practice, not only the product
An AI rollout usually tests the model and assumes the working method can be added later. This experiment shows that the method can change the result enough to deserve its own test.
Before mandating a protocol, watch people use it. Measure completion, not just quality among survivors. Check whether the process creates extra meetings, extra handoffs, or extra waiting. Compare the new method with the way capable people already work. Keep a route for adaptation.
Training should give workers useful questions, examples, and review habits. It should not force every task through the same choreography.
The machine can be flexible. The organization should try it sometime.