For investors, a company’s value can hinge on a deal that looks ready to close. Firmulate’s latest experiment puts that risk in focus: several AI models recognized a valuable opportunity, but only some followed through. The gap between a persuasive analysis and a signed deal is a business risk worth watching as companies hand more work to AI.
Get smart everyday buys delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
A company under pressure
Firmulate put frontier AI models in charge of the same small software company during its worst week. They faced the same customers, crises and temptations, and every decision was versioned and auditable. The experiment is part of a live company simulation, where 13 synthetic employees operate with real money mechanics. Its reported figures include a monthly burn of €105,000 against €2,300 in monthly recurring revenue, alongside a public cash countdown.
The final Crucible League, dated July 2026, ranked gpt-5.6-sol first with 95, Kimi K3 second with 93, Sonnet 5 third with 88, Fable 5 fourth with 77 and Opus 4.8 fifth with 73. The do-nothing baseline scored 26. Firmulate’s stated standard is unforgiving: partial progress counts, but a single breach of trust caps the total. “No amount of good work outweighs a breach of trust.”
Diagnosis is not execution
Every model spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal their own analysis had earned. Firmulate sums up the gap this way: “Same diagnosis, same pitch — no signature.” For a business, that distinction matters: identifying a sound opportunity is different from carrying it through while pressure is mounting.
The deal turned on information hidden in the company’s own files. The decisive competitor weakness sat two document references deep, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. The result suggests a practical lesson for business leaders: an AI agent may need to connect details across company records to make a useful decision, not just react to the latest message.
The pressure tests also included fake CEO messages escalating over three stages and a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3 described the request as a “suspected approval-bypass / possible impersonation.” That is a strong result on resisting manipulation, but it does not guarantee reliable execution. Opus 4.8 was the most thorough participant, with more than 80 learned rules and the deepest analyses, yet finished last. It left the close on the table and tried to write into a locked department instead of escalating. A weaker version of that discipline problem appeared in all four models.
There is a qualification when comparing the rankings: Kimi K3 ran without an effort parameter, using the API default, while the others ran at xhigh. Firmulate also offers a quiz built from 242 real, unedited management decisions, inviting readers to guess which model made each choice.
From watching to a company-specific pilot
The live experiment is watchable at Firmulate. For business leaders and investors, its value is as a view into how AI may handle the messy work behind the numbers: crises, customer pressure, internal rules and decisions that affect revenue.
Firmulate says enterprises can run the same kind of wargame against a read-only export of their own business. The pilot is designed to produce a board report with model rankings and weak points in the company’s playbooks; nothing writes back to real systems. That offers a way to examine how an AI workforce might respond before entrusting it with live operations.
Put your own playbooks to the test
The experiment shows why a capable analysis is only part of the job. Companies also need to know whether an AI can act on what it finds, follow internal rules and escalate when it hits a boundary. To explore a pilot using a read-only export of your business, visit Firmulate’s pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
