From 14 Stalled Pilots to One Production Win in 90 Days

The situation

Eighteen months into its GenAI program, a North American automotive OEM ($10–20B revenue, 30,000+ employees) had 14 pilots running: warranty operations, supplier documentation, dealer support, engineering knowledge search, marketing content. Every one of them had a demo. None of them had shipped.

The pattern will be familiar to anyone who has read MIT's finding that 95% of enterprise GenAI pilots deliver no measurable P&L return. Each pilot had a sponsor; almost none had an owner whose budget would feel the result. Meanwhile the run costs — vendor licenses, cloud spend, contractor hours — kept compounding quietly in the background.

A new CIO gave the program one quarter to show a path to production. The team took Delzey's free Readiness Score and self-assessed at 68 out of 100. Then they booked the Readiness Audit to find out whether that number was real.

The audit

Two weeks, fixed fee ($19,500 at this scope tier — 14 initiatives, eight interviews, regulated supply chain). Evidence review of the full pilot inventory, business cases, architecture docs, and two vendor contracts, plus eight stakeholder interviews — engineering leads to the CFO's office.

The verified score came back at 51 — a 17-point delta against self-reported. The weakest domain by a distance was Domain A: pilots and ROI discipline.

• 3 of 14 pilots had a named executive owner
• 2 had a baseline measured before the build
• 0 had kill criteria
• 2 pairs of pilots were unknowingly building the same thing

The readout delivered a Ship / Fix / Kill triage of the actual portfolio: 2 ship candidates, 7 fixable, 5 kills.

What we did

Delzey is one senior consultant. The OEM has hundreds of engineers. The engagement was structured accordingly: their people build; we audit, spec, referee, and verify.

Kill. Five pilots were shut down within two weeks of the readout — the duplicates, one tool the vendor's own roadmap had made redundant, and two initiatives no owner was willing to claim. Roughly $900K in annualized vendor, cloud, and contractor spend came off the books.

Fix. The seven fixable pilots each got what the audit found missing: a named owner, a measured baseline, and kill criteria.

Ship. Warranty-claims triage won the production slot — repetitive volume, clean success criteria, and a cost line someone already owned. A directed pilot-to-production sprint ($35K, six weeks) followed. Delzey wrote the spec, the architecture decision (the OEM's existing document-AI vendor plus in-house routing — refereed, not sold), the eval plan against a labeled 500-claim test set, the guardrails (human review above a dollar threshold), and the go-live gate.

The OEM's engineers built it. The first gate attempt missed the extraction-accuracy threshold; two weeks of fixes later, it passed. That is what the gate is for.

The results

Ninety days after audit kickoff, warranty-claims triage was in production — the OEM's first GenAI system to make it there.

• In its first eight weeks, the system auto-triaged roughly 70% of inbound claims, with human review on the rest
• Manual triage hours dropped 31% against the pre-build baseline
• Routing accuracy held at parity with the human baseline — the goal was as-good-and-faster, not magic
• The ~$900K freed by the kills was redeployed into the data-foundation work the Fix pile needed, and covered the $35K all-in Delzey fees many times over (the audit fee was credited against the sprint)

The OEM moved onto an Advisor retainer for monthly portfolio steering. A second sprint — supplier-document processing, promoted from the Fix pile — is scoped and waiting on its baseline.

"The triage table was uncomfortable for exactly one meeting. Then it became the most useful page in the building." — Chief Information Officer