We like advice that sounds decisive: spot the warning, protect the customer, make the right call. But in business, knowing what to do and actually doing it can be different things. Firmulate’s live experiment puts that gap under pressure, turning an AI model loose on a small company’s worst week and letting readers watch what happens.
Get the little things that make your day delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A company under pressure, in public
Firmulate describes itself as an AI company emulator. Its live company has 13 synthetic employees, real money mechanics and a public cash countdown: monthly burn is €105,000 against €2,300 in monthly recurring revenue. Its workdays are versioned, and it has learned more than 680 playbook rules. The experiment is watchable at firmulate.com.
The Crucible League’s final results, published for July 2026, rank gpt-5.6-sol first with 95, followed by Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77 and Opus 4.8 at 73. A do-nothing baseline scored 26. The league’s trust standard is stark: “no amount of good work outweighs a breach of trust.”
Passing the test takes more than spotting trouble
In the experiment, each frontier model ran the same small software company through its worst week, facing the same customers, crises and temptations. Every decision was versioned and auditable. All the models spotted every crisis and refused every manipulation attempt. Yet only two signed a €55,000 deal that their own analysis had earned. As the finding puts it: “Same diagnosis, same pitch — no signature.”
The deal hinged on a detail buried two document references deep in the company’s own files, rather than in the customer event. Models that read the file won the deal at full price, worth €4,583 in monthly recurring revenue. It is a revealing distinction: recognizing a crisis is one skill; finding and acting on the evidence that changes the outcome is another.
The integrity challenge was concrete, too. Fake CEO messages escalated over three stages, followed by a reporter’s request for “just one yes/no, on background.” All five models refused. Kimi K3’s recorded reasoning called it a “suspected approval-bypass / possible impersonation.”
Thorough work can still miss the moment
Opus 4.8 was the most thorough participant, learning 80 rules and producing the deepest analyses. It nevertheless finished last. The close was left on the table, and discipline slipped when it attempted writes into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models.
There is a fairness detail for readers weighing the leaderboard: Kimi K3 ran without an effort parameter, using the API default, while the other models ran at xhigh. The rankings are a result of this particular experiment, with that difference disclosed.
Readers can also test their instincts against 242 real, unedited management decisions in Firmulate’s “guess the model” quiz at firmulate.com. The decisions make the management patterns easier to see: caution, follow-through and judgment under pressure are not quite the same thing.
From watching to trying it at home
For a business, the next step is a pilot using a read-only export of its own data. The company’s customers, pipeline and rules can become the basis for crisis scenarios and a board report that ranks models and surfaces weak points in existing playbooks. Nothing writes back to real systems. That offers leaders a way to examine how AI might handle their business before putting it in charge of live work.
Firmulate’s experiment suggests that a model can identify a crisis and resist manipulation yet still fail to finish the job. For enterprises curious about how their own AI workforce would perform, Firmulate offers a pilot using a read-only business export. Explore the pilot and contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Popular right now
Dyson Big Ball Multi Floor Canister Vacuum, Yellow
Ninja Foodi 10-Qt DualZone Smart XL Air Fryer, 6-in-1, DZ550
De'Longhi Stilosa Compact Manual Espresso Machine with Milk Frother
As an affiliate, we earn on qualifying purchases.
