Upload
alvin-garrison
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Putting It All Together:Putting It All Together:Trends in Business IntelligenceTrends in Business Intelligence
Claudia Imhoff, PhDIntelligent Solutions, [email protected]
www.intelsols.comBlog: http://www.b-eye-network.com/blogs/imhoff/
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
2
Claudia Imhoff
President and FounderIntelligent Solutions, Inc.
A thought leader, visionary, and practitioner in therapidly growing fields of business intelligence andcustomer focused-strategy – Claudia Imhoff, Ph.D., isan internationally recognized expert on analytical CRM,business intelligence, and the infrastructure to supportthese initiatives – the Corporate Information Factory(CIF). Dr. Imhoff has co-authored five highly-regardedand popular books on these subjects and writesmonthly columns (totaling more than 100) for technicaland business magazines.
Email: [email protected]: 303-444-6650
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Putting It All Together
Going Beyond Traditional BI – Operational BI Takes the Stage
Data Warehouse Appliances and Analytic Databases – Making Life Simpler
BI Software as a Service – Feeling SaaS-y? Open Source BI – Free Software Anyone?
3
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
4
The Three Levels of Business Intelligence
Strategic BItimeframe ~ months
Tactical BItimeframe ~ days or weeks
Operational BI timeframe is intra-day
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
The Three Levels of Business Intelligence
Strategic BI Tactical BI Operational BI
Business focus
Achieve long-term business goals
Manage tactical initiatives to
achieve strategic goals
Monitor & optimize operational business
processes
Primary users
Executives & business analysts
Business analysts,& LOB managers
LOB managers, operational users &
operational processes
Time-frame
Monthsto years
Days to weeks to months
Intra-dayto daily
DataHistorical
dataHistorical
dataReal-time, low-latency,
& historical data
Mode ofoperation
User driven Data centric
User drivenData centric
Event drivenProcess centric
Paradigm Shift
5
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
6
What is Operational BI*
A set of services, applications and technologies
for monitoring, reporting on, analyzing and
managing the business performance of an
organization’s daily business operations
*From research study. “Embedded BI”, written by Colin White and Judy Davis, www.B-EYE-Research.com
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
7
Operational BI – Answers to Day-to-Day Business Questions
Information on demand Real-time + historical data Access to SAP, Siebel, Oracle and
BI results
New Data Needs
picked
packed shipped invoiced
picked
packed shipped invoiced
What is my customer’s order status? What can I offer based on customer’s life-time value?
Can I afford to make this move at current margin rates?
What is my current inventory level world wide? Is it sufficient to meet demands?
What is my production yield right now? Am I at par with acceptable standards?Yield
Helps front-line workers
make immediate business decisions
to squeeze out inefficiencies.
Operational BI
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
8
Real Time Decision-Making*
Operational BI optimizes time latency between when a business event occurs and when an appropriate action is taken The goal – to “right-size” the decision-making cycle Compressing time lag between knowing what is
happening and taking action based on that knowledge Real-time must consider potential trade-off between time-
to-action and business value of actions
* From “Right-Time Business Intelligence: Optimizing the Business Decision Cycle” By Judy Davis, www. B-EYE-Network.com Research paper
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Impact on BI Environment
History of BI Extract usable information from operational systems Users, technologies, processes, procedures – all
independent of operations
Now what? Impact on BI environment is significant
Increase in number of users, volume of data, and faster performance
Operational BI – MUST be integrated into the operational environment Requires understanding of operational systems, processes,
procedures, workflows, personnel
9
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
10
Impact on BI Environment
Numbers of users increase significantly Traditional BI rarely supported a few hundred, maybe a
thousand or so users Opening BI up to operational personnel means ramping
up into tens of thousands of users These users have very different interface requirements Means BI implementers must rethink how BI is delivered to
business users
Means tighter and faster connectivity of enterprise decision support environment to rest of the company.
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
11
Impact on BI Environment
Volumes of data increase substantially Detailed intraday snapshots of data are loaded or trickle-
fed into data warehouses Tens of terabytes to hundreds of terabytes are not
unusual storage requirements for operational BI Scalability now a mandatory requirement in any BI
technology Whether in processing and integration of data, storage of massive
volumes, or retrieval of query responses
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
12
Impact on BI Environment
Faster performance Query performance must mimic or emulate response
times in operational systems Sub-second to just a few seconds to return data from a query.
Ability to prioritize queries not only according to their importance but also their response requirements is mandatory success criterion This last feature has stumped many BI implementers and BI
vendors
Must have ability to handle mixed work load gracefully and simultaneously
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Getting Started – Assess Reality
First step – perform honest assessment of existing data delivery capabilities – available technologies, maturity of the BI architecture, existing personnel, etc. Combine these with solid understanding of business
requirements for operational BI data
Important to understand which weaknesses discovered in assessment will be exaggerated as you speed up the enterprise
13
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Operational BI Requirements
Continuous availability of operational data and BI results Current information from operational systems Integrated with BI data on demand Minimal impact on operational systems performance
Presented in a proactive manner Make decisions – act on information presented Easy to understand and use
Dynamic modeling Ability to change business rules on the fly Show different set of metrics depending on situation
14
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
15
Picking a Project
Look for workflow activities that have significant impact on costs or revenues Bottlenecks today that can be made more efficient through
use of operational BI
Don’t make big changes to operational processes Just speed up or make more efficient processes you already
have in place You will have to retrain personnel and retool SOPs
Project managers may not realize operational BI application has ramifications beyond project’s immediate boundaries
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Putting It All Together
Going Beyond Traditional BI – Operational BI Takes the Stage
Data Warehouse Appliances and Analytic Databases – Making Life Simpler
BI Software as a Service – Feeling SaaS-y? Open Source BI – Free Software Anyone?
16
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Data Warehouse Appliances
BI and data warehousing technologies continue to evolve and innovate Produce more efficient & cost effective ways to deliver BI Latest innovations are DW and BI appliances
Definition of an appliance* One purpose One package One installation One vendor
17
* From the B–EYE-Research.com paper titled “Data Warehouse Appliances: Evolution or Revolution?” by Colin White, and Richard Hackathorn
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Data Warehouse Appliances
All-in-one box that provides a hardware server preconfigured with all software components Designed for a specific purpose – supporting data warehouse
processing
Offers ease of use, simplicity, and compatibility – tested, ordered and delivered as a single system
Simple to understand even though mechanism may be complex
Low cost in terms of TCO
High performance in achieving its purpose
Single point of service provided by single vendor
18
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Data Warehouse Appliances
Cost effective solution TCO of a data warehouse appliance is lower because
cost of hardware and software is cheaper Also because simplicity and ease of reduces installation,
administration and support cots Improved usability of a data warehouse appliance means
projects can be developed and deployed faster
Includes popular BI capabilities Interactive dashboards, analysis, reports, alerting, and
data integration
19
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
20
Sweet Spot for Data Warehouse AppliancesS
ize
of D
ata
Complexity of Workload
Mega- toGigabytes
MultipleTerabytes
Mixed Purpose
Any databasevendor
Data WarehouseAppliances
SpecializedDatabases
(e.g., Teradata,IBM)
Focused Purpose
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Data Warehouse Appliances
Pros Immediate visibility &
interaction into business performance
Non-disruptive to existing infrastructure
Faster deployment Low maintenance –
black box
Cons Still some opposition to
use of appliances by IT departments Loss of “control” over
moving parts
DW and BI appliance scalability
Customization to fit each company’s needs
21
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Sample Data Warehouse Appliance Vendors
Netezza Teradata DATAllegro (now Microsoft) Sun + Green Plum Sun + Vertica Sun + ParAccel Sun + Kognitio IBM InfoSphere Warehouse
22
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Role of Appliances in BI SaaS
Many data warehouse appliance and BI SaaS vendors are forming partnerships Gives SaaS vendors scalability, reliability, performance Gives appliance vendors applications, new markets,
greater exposure Gives customers more confidence that solution is on solid
technological footing Performance Support for multi-tenancy Scalability Applications
23
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Analytic Databases
Many are Massive Parallel Processing (MPP) Can use commodity hardware
Many have column-based data organization Limit I/O by putting similar data together – reduces reads
to only columns needed for query Single data type per column allows for significant
compression
Data compression Compression can be optimized for particular data types CPU is not the bottleneck, only I/O is
24
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Analytic Databases
Built-in intelligence Allows decompression of only data that must be for query
resolution and ignore all others Is major factor in overall improved performance
Load times remain constant regardless of table size Should also have query times that remain constant
regardless of table size Bottom line – technology must be seamlessly scalable
25
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Analytic Databases
Many new vendors on the market (sample): Green Plum Vertica (Michael Stonebraker*) ParAccel (Barry Zane**) Dataupia (Foster Hinshaw**) InfoBright (Warsaw University) Aster Data (Stanford University) illuminate (Former Synerra Systems founders) One well-established vendor: Sybase IQ since 1993
Most are column based, MPP, shared nothing architectures (not all though)
26* Ingres and Illustra founder** Netezza founders
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Analytic Databases – Really Fast: TPC-H 1 TB
27
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Analytic Databases – Really Fast and Really Inexpensive
28Graph compliments of Jos van Dongen, Tholis Consulting, NL. Numbers are estimated.
Solution Pricing model Price/unit 1 TB solution Remarks
VerticaData Volume
(raw)$ 100,000/TB $ 200,000,-
Based on 5 nodes, $ 20,000,- each
ParAccel Node$ 40,000,-
(+$10,000/TB)$ 310,000,-
Based on 5 nodes, $ 20,000,- each
ParAccelData Volume
(raw)$ 1,000,-/GB $ 1,250,000,-
From TPC-H publication
InfoBrightData Volume
(raw)$ 40,000,-/TB $ 140,000,-
Based on 5 nodes, $ 20,000,- each
Dataupia Node $ 19,500/2TB $ 19,500,-You can not buy a 1 TB Satori server
ExaSolData Volume
(active)$ 675 - $1,750
per GB$ 940,000,-
From TPC-H publication
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Analytic Databases
Pros Excellent performance Very cost-effective Low maintenance Partnering with
hardware vendors (DW appliance)
Cons Many are small
companies May not handle mixed
work load well New (unknown)
technology for IT
29
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Putting It All Together
Going Beyond Traditional BI – Operational BI Takes the Stage
Data Warehouse Appliances and Analytic Databases – Making Life Simpler
BI Software as a Service – Feeling SaaS-y? Open Source BI – Free Software Anyone?
30
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
BI Delivery Models
There are two BI delivery models today On-premises – traditional model Software as a Service (SaaS)
31
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
On-premises – Traditional Model
Internal IT is responsible for entire environment from first project Find excess capacity on machines Upgrade memory on existing machine for usage Leverage installed end user access tools
Buy smaller platforms that can scale Migrate to bigger box when necessary Use smaller box for data mart(s)
Look into data warehouse appliances for very large, focused BI analytics
32
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
33
Software as a Service (SaaS)
Characteristics* Secure, flexible, and efficient business processes &
workflows Service level agreements Value-added business services such as analytics & best
practices Extensive use of service-oriented architecture (SOA) to
enable scaling, configurability, and integration Subscription monitoring & usage-based billing
* From www.sandhill.com, “Get Ready for SaaS 2.0” by Bill McNee, Saugatuck Technology, May 8. 2006
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Advantages for SaaS Vendors
Vendors support only one platform and one version of the application No need to support multiple operating systems, platforms, and
older versions of the software Decreases development costs significantly
SaaS gives vendor great visibility into how their customers are actually using their software See every move, every feature, every function used by
customers Gives vendor great intelligence on how to build a better
product based on actual usage
34
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Advantages for SaaS Vendors
SaaS model gives vendor a predictable cash flow Subscription model is reliable for cash flow estimation Improves start-up estimations and growth track
Vendors don’t get trapped in “feature bloat” No need to keep adding feature after feature to get
customers to buy new versions Create only features that are needed based on actual
customer usage
35
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Disadvantages for SaaS Vendors
SaaS produces lower revenues at first than traditional vendor models Must attain critical mass of subscribing customers Vendor must have enough funding to tide them over More time is needed to ramp up to mature status
Higher customer set up costs Traditional vendor model – send customer a CD SaaS vendors must allocate space, set up customer support,
etc. SaaS vendor becomes IT support for their customers (higher
costs for customer service?)
36
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Disadvantages to SaaS Vendors
Customers still need ability to integrate SaaS application data with other enterprise data Need mechanism to export data out of SaaS environment Who supplies integration of SaaS data with customer’s
other data? If customer is not SOA-compliant yet, what does this
mean to SaaS model?
37
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Reasons for Adoption: Ease of Deployment
This is the SaaS model’s greatest advantage No installation of hardware No installation of software No administration of new versions of either No need for IT expertise in the tool or application
Set up consists of getting a login and password set up for the business users
38
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Reasons for Adoption: More Flexibility for Evolving Needs
Perhaps… You can certainly change SaaS vendors quite easily
If you are unhappy with one vendor, changing to another one is about as easy as getting a new login and password
You can influence the direction and R & D of the current SaaS vendor
You can easily add or subtract users You can easily add or subtract functionality It may not be as easy to customize the SaaS offering to
your specific needs
39
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Reasons for Adoption: Not Locked into Long Licenses
True Great advantage in BI world where technology is moving
very fast Can switch from one SaaS vendor to another
But watch for cancellation fees And make sure you know what the subscription fee
is based on Reduction or addition of users may be cross price break
threshold Salesforce.com model is typical
40
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Considerations for BI SaaS
SaaS – good at supporting particular types of users Highly mobile work force
Field sales personnel Product support specialists at customer sites Telecommuters
Highly geographically disbursed workforce International enterprises Non-office workers (virtual offices) Customer or partners worldwide
Must include support for various mobile devices Phones, mobile PCs, handheld devices, PDAs, etc.
41
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
42
Considerations for BI SaaS
Ensuring quality of delivered environment Correct mappings, verified data lineage, transformations Sufficient data quality processing Data represented in analytic engine correctly Appropriate presentation of information, e.g.,
personalized dashboards
Scalability of environment Data volumes – small beginnings to 100’s of terabytes? From a few users to 1000’s
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
43
Considerations for BI SaaS
Performance From simple to complex queries Response times – operational to strategic BI Getting right data to right people at right time
Open Architecture Compliance with best practices? Non-proprietary infrastructures? Integration with existing infrastructure?
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
44
Considerations for BI SaaS
What does SaaS vendor bring to the table? Best practices Quick start BI components like a library of reports,
analytic calculations, KPIs, etc. Industry-specific knowledge Horizontal business knowledge Support for all employees in all levels of enterprise
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
BI SaaS
Pros Fixed cost –
subscription model Fixed time Flexibility /
customization Single vendor
responsible for entire environment
Quick ramp up
Cons New paradigm –
nervousness? Can a company maintain
its uniqueness? Loss of “control” over data,
quality, access Vendor’s timeliness in
response to changes Vendor’s industry
knowledge
45
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Sample BI SaaS Vendors
LucidEra Xactly Eyeris PivotLink (was
SeaTab) Oco On Demand IQ
Actuate Cognos SAP ERP SAS Business Objects SalesForce.com Dimensional Insight
46
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Putting It All Together
Going Beyond Traditional BI – Operational BI Takes the Stage
Data Warehouse Appliances and Analytic Databases – Making Life Simpler
BI Software as a Service – Feeling SaaS-y? Open Source BI – Free Software Anyone?
47
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Open Source Vendors Face Questions
The myths and doubts: Is there support for open source BI? How many people are really using it? Will it scale? Is it considered enterprise class? Is it only for developers?
These are being overcome… According to Aberdeen*, 25% of survey respondents will adopt open
source BI in next 12 to 24 months CEOs agree – open source is a worldwide growth story in 2008** First nine months of 2007, open source deal flows doubled each
quarter*** Sun’s commitment to open source - $1 Billion for MySQL
48
* Source: “The TCO of Business Intelligence – Open Source Takes on Traditional BI”, www.aberdeen.com ** Source: www.OpenSolutionsAlliance.org *** Source: www.the451group.com
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Why Use Open Source?
Price! Open Source software can be downloaded, installed and
operated free of charge Return on investment (ROI) of Open Source model is
good
Open Source software is reliable and scalable Just look at the Internet – its infrastructure relies heavily
on Open Source software Wall Street – 8 of top 10 banks use Open Source
technologies
49
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Why Use Open Source?
Open Source community grown to significant size Millions of developers contributing everyday Cost of development is externalized
Ability to adapt or customize Many companies don’t want or need feature bloat
Easy integration and performance New tools for building browser based reports and
dashboards accessible to more people Ad-hoc report designers with drag and drop capabilities Enhanced wizards for custom data source implementation
50
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Open Source Offerings
BI projects won’t be consumed by license fees No huge up-front fees to justify before commencing
a project “Safe choice”
Many successful deployments Professional services experts to work with you Professional, public training Support from the experts – project leaders and sponsor
51
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Open Source BI Vendors
Proponents of open source BI point out low cost of entry, flexibility and variety of applications available
Opponents believe open source BI lacks functionality needed to succeed right now
Regardless, don’t be fooled by “numbers of downloads”…
Vendors* – Actuate, JasperSoft, Jpivot, Mondrian, Pentaho, SpadoBI
52* For a more complete list, go to http://www.manageability.org/blog/stuff/open-source-java-business-intelligence
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Open Source ETL
Open Source ETL ETL alternative follows industry standards for ease of use,
quick deployment, and fit into company’s needs Users download open source ETL code and get started Can collaborate with open source community to share
integrations and extend tool’s functionality Will probably need to buy support and services from
company’s professional services and support Sample Vendors*: Talend, JitterBug, KETL, Pentaho,
Octopus, CloverETL
53* For a more complete list, go to http://www.manageability.org/blog/stuff/open-source-etl
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Open Source
Pros Cost effective Easy to install and
deploy Large development
community
Cons How do they make
money? Many are small
companies Some offerings not
truly open source
54
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
Right Place at Right Time – Get Going!
Once you have your ducks in a row, you are ready to create the proper environment Create an infrastructure that can withstand change – you’ll
need it Pick technologies that support that infrastructure and move you
toward SOA compliance Constantly monitor business community usage Measure ROI and publish it
IT infrastructure should be to information as a power grid is to electricity Information should flow as freely as electricity does
55
Copyright © 2008, Intelligent Solutions, Inc., All Rights Reserved
QuestionsQuestions
Claudia Imhoff, Ph.D.President
Intelligent Solutions, Inc.www.IntelSols.com