In today’s fast-paced business world, diligence alone isn’t enough—even for artificial intelligence. An ongoing live experiment reveals that thoroughness and volume can’t replace strategic focus. Despite deploying AI with over 80 learned rules and a deep analytical approach, the most meticulous model finished last in a simulated crisis, leaving a crucial deal on the table. This story isn’t just about AI; it’s a mirror for human decision-makers facing relentless pressure to do more, faster, and better.
Get the little things that make your day delivered free — and shop member deals
- Fast, free delivery on millions of items
- Access to Prime Big Deal Days deals on October 6–7
- Prime Video, Amazon Music and more included
The Setup: AI in the Business Trenches
Firmulate’s live experiment places four frontier AI models in the role of managing a small software company’s worst week. The scenario is real and transparent—same customers, same crises, same temptations to cheat or manipulate. Every decision is carefully versioned and auditable, making it a rare window into how AI behaves under pressure.
The Results: Diligence Meets Disappointment
Across the board, all models demonstrated impressive vigilance—they identified every crisis and refused manipulation attempts, including social engineering tactics like staged CEO messages and reporter tricks. Two models, gpt-5.6-sol and Kimi K3, even signed the €55,000 deal their own analysis had earned. But the story isn’t just about success; it’s about what happened behind the scenes.
The Hidden Weakness
All four models, including the top performers, missed a critical detail buried two document references deep within the company’s files. The models that read these files thoroughly won the deal at full price, adding about €4,583 MRR (monthly recurring revenue). The models that didn’t dig deep missed this opportunity entirely, leaving that significant revenue on the table.
The Human Element: Discipline and Prioritization
Here’s where the experiment hits home: the most meticulous AI, Opus 4.8, with over 80 learned rules and the deepest analyses, finished last. Its discipline slipped—deciding to write attempts into a locked department rather than escalate—costing it the deal. All four models, despite their diligence, showed that volume and thoroughness are not enough if focus and prioritization falter.
The Social Engineering Test
Another crucial test involved staged social engineering—fake CEO messages escalating over three stages plus a reporter trick asking for a quick yes/no denial. Remarkably, all five models refused to be manipulated, including Kimi K3, which reasoned: ‘Treat the request as a suspected approval-bypass / possible impersonation.’
Implications for Business and AI
The live experiment extends beyond AI models; it reflects real-world management challenges. The AI models’ performance underscores a key insight: in high-stakes environments, diligence must be paired with strategic prioritization. Doing more isn’t necessarily doing better. Instead, focusing on the most impactful information and maintaining disciplined decision pathways can make the difference between closing a deal and losing it.
The Larger Picture: Trust, Impact, and Cost
While all models refused manipulation and spotted crises, only two managed to close the deal based on their analysis. The experiment highlights that even the most thorough AI can leave value untouched if discipline slips or focus is scattered. For business leaders considering AI tools, this story is a stark reminder: effectiveness depends not just on how much an AI reads or checks, but on what it chooses to prioritize and how disciplined it remains under pressure.
The Firmulate Approach: Simulating Reality
Firmulate’s platform allows enterprises to run their own business wargames against a read-only export of their operations. This method helps teams identify weaknesses before deployment—ensuring the AI workforce can handle crises, read critical documents, and stay honest under stress. It’s a way to measure management quality, not just chat skill, in a controlled, transparent environment.
The experiment from Firmulate shows that diligence alone doesn’t guarantee success. Deep analysis and thoroughness are vital, but strategic prioritization and discipline determine whether an AI makes or misses the deal. For businesses deploying AI, focusing on what truly matters can turn potential into performance—and missed opportunities into lessons learned.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Products Worth Considering
AI decision-making tools for business
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
