End of the EDW era?

Last decade we fuzzed about Enterprise Data Warehouses (EDW). A data warehouse concept where all the enterprise data is collected to one central data warehouse – preferably to one data model, the one and only truth of the company and its’ business landscape. EDW has been the Holy Grail of data warehousing last 20 years, and we finally saw those solutions in reality during last ten years.

Efforts building (and maintaining) EDW systems are significant. Long waterfall projects beginning with long and painful logical data modelling exercises following months of data, integration and physical modelling work. And when system is finally “ready”, business has a bunch of new requirements and company needs to start tweaking its’ data model again and build new multilevel Extract-Transform-Load processes (ETL). Slow, risky and costly!

Last few years we have been forced to rethink the EDW approach. Gartner introduced term “Logical data warehouse” in 2011. And the question was how this “big data architecture” will impact the Holy Grail of data warehousing – EDW. Logical data warehouse concept is closer to reality with solutions such as SAP HANA. HANA’s multi-tenant architecture support and Hybrid Transactional/Analytical Processing (HTAP) makes possible to access and use your all data from one integrated platform without moving the data around and duplicating it to several places. Data virtualization, HTAP, logical data warehouse, multi-tenant systems, modern data architecture – just to name few concepts which will change the data warehousing and analytics architectures fundamentally.

Also Internet-of-Things and other data streaming solutions are creating new challenges in real-time data processing. Amount of data can get so high, it is semi-structured, or it is coming in so frequently that traditional EDW approach is way too expensive for any business case. These new challenges and requirements forces to rethink the architecture and where again modern data architecture and logical data warehousing concepts come in the picture. In many cases relational database is not even the best feasible solution and NoSQL databases without schema and strict relations is more agile and flexible option.

As enterprises’ data needs, variety and amount of data grows and the overall environments gets more demanding, it is many times even impossible to centralize and combine the data into a single database.

Upcoming modern data architecture trends:

SAP HANA is a platform, which is designed to support logical data warehouse concept. For some time now it has been possible to use data directly and seamlessly from its’ original location through Smart Data Access. In the latest service pack SPS09 multi-tenancy has taken a step further as well. SAP introduced multi-tenant database concept, and you can run several applications e.g. ERPs on the same HANA appliance – each system having own tenant database. And great new feature for big data is write-back to HDFS. These features are not only supporting modern data architecture, but truly helping companies to reduce IT costs significantly and giving them a lot of new options when planning their enterprise architecture.

One thing is clear and obvious: there is no magical “One Size Fits All” solution. Architecture needs to be designed fit for purpose – now we just have a bunch of new tools, systems and applications supporting modern solutions and architectures. But the end of the day the dream remains the same as before: one truth of business and easily accessible data through a single access point.

Last but not least: in few years we need to rethink our approach again when we see live SAP S/4s. Are data warehouses for business analytics dead for businesses running SAP? One doesn’t have to be a wizard or have a crystal ball to predict that at least SAP BW’s role is going to change…

Contact Person

Blog writer

Tuomas Autio