Think of your data platform like an international airport. If every airline used different jetways. Each gate enforced its own security rules. And baggage systems couldn’t talk to each other. Nothing would scale. Planes would idle. Costs would spike. Delays would be constant.
That’s effectively how many enterprise data estates operate today. And it’s why the open data lakehouse has moved from an architectural option to an operational necessity. This is especially true as AI shifts from experimentation into production.
For CDAOs, CIOs, and IT architects, the move toward an open lakehouse isn’t philosophical. It’s a response to stalled AI projects, cost pressure and operational reality. Across industries, organizations tend to arrive at the same four reasons. Let’s explore them.
1. Lower total cost of ownership (without slowing teams down)
First, most enterprises still fund multiple parallel platforms. A data lake for raw data. A warehouse for BI. Separate environments for data science and machine learning are connected by pipelines that duplicate data and quietly rack up infrastructure, licensing and operational costs.
Open lakehouse architectures address this directly by separating storage from compute and allowing multiple engines to work in place on open tables. The savings are not limited to cheaper storage. They come from retiring redundant systems, reducing extract, transfer, load (ETL) sprawl and eliminating the constant churn of refreshes.
In regulated industries such as banking, this impact appears quickly. When risk, finance and data science teams work on the same governed data instead of maintaining parallel copies, organizations stop paying a coordination tax every time someone asks for a new cut of the data. Costs fall and iteration speeds up.
2. Unified access beats silos (and keeps AI from stalling out)
Next, silos do more than slow analytics. They quietly undermine AI. When models are trained on one version of the data and finance reports on another, definitions drift and trust erodes. AI initiatives stall not because the models are wrong, but because no one can agree on what data is correct.
An open lakehouse provides a single, governed substrate with consistent permissions, lineage and auditability, so analytics and AI operate from the same foundation. Gartner has warned that many AI initiatives fail because organizations lack AI-ready data management practices, not because of model limitations, but because the underlying data estate is not trustworthy or unified enough to scale.
This dynamic is especially visible in banking. Fraud or credit models rarely fail audits due to poor performance. They fail because teams cannot clearly explain which data were used, how they changed, or how decisions were made. Unified access with built-in governance keeps AI usable beyond the pilot phase.
Learn how SAS integration with DuckDB empowers smarter access to open data. Watch this webinar.
3. Simpler data estates (because complexity compounds)
The third point is that most modern data stacks did not become complex overnight. Complexity accumulated as teams added tools to solve point problems. Each new platform introduced its own metadata, interfaces and failure modes. Over time, the estate became fragile and expensive to change.
Standardizing on open file and table formats simplifies the data estate by reducing moving parts and increasing portability. Data remains accessible even as tools evolve, which matters as AI workloads introduce new requirements around observability, lineage and reuse. There is a shift toward reusable, consumable data products rather than bespoke pipelines stitched together behind the scenes.
In practice, this is what allows teams, whether in banking, healthcare or other industries, to test new analytics or AI approaches without standing up yet another platform or copying data into a new silo.
4. Self-service at scale, without losing control
Finally, speed and governance are often framed as trade-offs. In reality, the organizations that move fastest are the ones that enable governed self-service.
Open lakehouse architectures make this possible by enforcing policies such as access controls, quality checks and lineage directly on shared data. Analysts, engineers and data scientists work on the same foundation and under the same rules. Friction is reduced rather than created.
Banking provides a familiar example. When AML, payments fraud and risk teams share a governed data substrate, audit trails can span data to model to decision without heroic effort. The same pattern applies anywhere that trust and accountability matter (meaning basically everywhere).
Open isn’t the goal – it’s the enabler
Organizations don’t move to open lakehouse architectures because they want open formats for their own sake. They do it because openness is the only sustainable way to lower TCO, unify access, simplify operations and enable self-service, while avoiding a new generation of vendor lock-in.
Sounds ideal, right? Unfortunately, the reality doesn’t always live up to the hype. Many open data lakehouse initiatives stumble or fail outright due to architectural missteps, governance gaps or unrealistic expectations. Learn more about this reality and how to avoid any gotchas in my next blog.
DuckDB and SAS® Viya®
One of the strongest enablers of the trusted, open lakehouse is the integration of SAS Viya with DuckDB. It’s a high-performance, open-source OLAP engine built for fast SQL analytics on columnar data.
