Publication: Data Management Review

Column Title: Data Warehouse Delivery

Author: Douglas Hackney

Headline: When to Federate

Issue: May 2000

[callout: The federated BI architecture is the “big tent” ]

[begin copy]

In the past few months’ columns we’ve explored the reasons why a single, monolithic datawarehouse (DW) architecture is incapable of supporting the reality of today’s heterogeneous,speed driven market (past columns are available at www.egltd.com and www.dmreview.com).This month, we’ll examine when it is appropriate or required to move to a federated BIarchitecture.

As we have seen by the rapid growth and diversification of the business intelligence (BI) market,our current and future reality is driven by a wide panoply of business driven solutions focused onsolving domain specific problems. There is little doubt that the BI market has outgrown its humbleand relatively simple technologist and architecture roots. In fact, the market research companysurvey.com projects that the business intelligence market will be worth $148 billion dollars by2003. In a market of this size, complexity and impact, a simple, monolithic architecture isdesperately inadequate.

This current and future BI market is built on the foundation of a modern BI infrastructure,consisting of a federated BI architecture accommodating all the components of a contemporary BIsystem: packaged/turnkey DWs and data marts (DM), packaged/turnkey analytical applications(AA), custom built DWs and DMs, custom built AAs, data mining, online analytical processing(OLAP) tools, query and reporting (Q&R) tools, production reporting tools, data quality tools,extraction transformation and load (ETL) tools, system management tools, information deliverytools, enterprise information portals, reporting systems, knowledge management systems,database systems, etc. The federated BI architecture is the “big tent” that provides the foundationand environment to facilitate and enable business information flow, analysis and decision makingin the typical organization’s heterogeneous environment.

Sites that are just embarking on the design and implementation of a BI system will most likelychoose a federated architecture by default, as their system will be composed of a mixture ofpackaged and custom solutions. Sites that have started by attempting to develop a single,monolithic system are often highly stressed when faced with business driven scenarios that theirold-think architecture is not optimized to support.

You should design and develop a federated BI architecture when you are faced with the followingscenarios:•Mergers and Acquisitions (M&A)The wave of M&A activity across all industry segments has presented the BI professional with avery challenging scenario. Often, teams that are happily humming along building their custom,

hub-and-spoke monolithic system awake one day to learn that they are the proud owners of anew company or two, along with their accompanying BI systems. If very rarely makes sense toflush a fully functioning data warehouse infrastructure down the toilet, so these teams are forcedto adopt a federated BI architecture.

•Turnkey ERP DW(s) + Custom DW(s)Today’s market is quickly moving from the “build” stage of evolution to a “buy” mode. Just as weall used to develop general ledger and order entry systems, but would never even consider ittoday, BI is moving from hand built, custom systems to packaged offerings. Teams that have apartially completed monolith data warehouse system (which means 99.99% of the systems onthis planet) are often presented with a packaged data warehouse as an extension to the ERP orOLTP system that the business has just invested $20-40 million into. In this case, it is notconducive to career development to reject this turnkey system. Don’t look a gift horse in themouth as it were, and at a minimum consider it to be a wonderful way to extract data from thearcane innards of the ERP package.

•Multiple DW / DM SystemsOrganizations of any size, especially those that have been involved with data warehousing formore than a year or two, all have multiple DW / DM systems. A typical large scale site hasbetween 10 and 40 data warehouse systems and countless scores of data mart systems ofvarying degrees of architecture. These sites must adopt a federated BI architecture to achieveany semblance of data integration across the multiple systems.

•Turnkey analytical applicationsTurnkey analytical applications (AA) at extremely low price points ($5 – 50,000) are sweeping themarketplace. These systems are sold directly to the business users and are usually dumped onthe doorstep of IT as soon as the purchaser is promoted or moves on to greener pastures. Mostof these systems are non-architected and many are only marginally capable of being integrated atall. This scenario requires a federated BI architecture to accommodate these high-power, high-impact, politically sensitive solutions.

•Speed to market driven solutionsThere is nothing more important in today’s business world than speed to market. It will trumparchitecture every time. In a survey of over 1,500 BI professionals I’ve found one example of abusiness delaying an incremental income adding/margin affecting BI solution to allow forarchitecture. In order to be part of the solution, and to facilitate speed to market, you mustaccommodate the time requirements of the business in your federated BI architecture.

Be ready for these inevitable scenarios and be ready to be part of the solution instead of part ofthe problem by laying the foundation of a federated BI architecture today.

[end copy]