Yonas Yohannes just posted a deep dive into the ‘Architectural Mismatch’ holding banks back.
He’s right that ‘bespoke bot chaos’ is a mess, but I disagree that architecture is the primary culprit.
In my work with Pragmatic Data Services, I’ve found that if you define the business problem correctly and the model hits its accuracy targets during a POC, the ‘success’ is already there.
The question we need to answer isn’t just ‘How do we build a more complex fabric?’ it’s ‘Why are we letting technical hurdles block a solution that we’ve already proven works?’
If the output is deterministic and the metrics are solid, that’s not a pilot—that’s a product. Let’s stop treating AI like a science experiment and start treating it like the business tool it is.”
Yonas is someone who I cherish as a mentor, and someone who has had significant influence on the message and trajectory of Pragmatic Data Services. His article focuses on what many are trying to unravel: The failure pattern of AI initiatives. What many business leaders find frustrating, is that *if* the POC (Proof of Concept) works and the metrics are solid, it should be a straight line to production – and ultimately ROI.
However, in enterprise environments—especially banking—the gap between a successful POC and a successful “Project” is rarely about the AI’s accuracy. I am of the opinion that it is more about the entire organization’s understanding of the complex fabric, technical hurdles, and the adaptability necessary to drive success behind these projects.
Here is what often happens behind the scenes that keeps a winning POC from becoming a production success:
- The “Data Drift” Trap
In a POC, you are often working with a “gold dataset”—a clean, static slice of data.
- The Reality: In production, data is “noisy” and constantly changing.
- The Failure: The model that was 98% accurate on Monday might drop to 70% by Friday because the upstream data format changed or the customer behavior shifted. Without the Architecture Yonas mentioned (Data Fabric/Agentic RAG), the project fails because it can’t maintain its POC performance in the “wild.”
- The “Last Mile” Integration (The Deterministic Wall)
Starting with a deterministic output expectation is a good start, but “believing” the output is deterministic and “enforcing” it are two different things.
- The POC: A human reviews the AI output and says, “Yes, this is correct.”
- The Production: The AI has to talk directly to the Core Banking System.
- The Failure: IT and Risk teams often block the project here because there is no “hard-coded” safety net. If the AI makes one mistake in 10,000 transactions, who is liable? Without a Risk Operations Center (ROC) layer to catch that 1-in-10,000 error, the project stays in the “demo” phase forever.
- The Scalability of Trust (The Human Factor)
A POC usually involves a small, excited team. Production involves the “Middle Office”—people whose daily workflows will change.
- The Failure: Even if the AI is accurate, if it doesn’t fit into the existing Operating Model (how people actually do their jobs), they will bypass it. If the AI’s “deterministic output” requires a human to spend 20 minutes double-checking it, the “measurable business impact” vanishes.
- Regulatory and Audit “Hard Stops”
In BFSI, “it works” isn’t enough. You have to prove why it worked.
- The Failure: Many GenAI projects fail at the production gate because they lack Explainability. If a regulator asks why a commercial loan was denied, and the answer is “the model said so,” the project is shut down. This is why Yonas emphasizes Segment Ontologies (BIAN/FIBO)—they provide a standardized language that auditors can actually follow.
- DATA Provenance: This is a term that isn’t commonplace yet, but the ability to ‘show receipts’ for the data source, the connection to the output, and a clear chain of custody matters. Especially for zero tolerance projects that cannot allow hallucination impact.
Clear understanding of the Business Problem and associated Metrics must come first. If those aren’t defined, the architecture is just expensive over-engineering.