When every shot counts - can you make a difference or will you fail

Slide 1

By

Paul Ormonde-JamesTwitter: @pormondejamesLinkedin http://au.linkedin.com/in/ormondejames

When every shot countsWhen every shot counts

Intelligence for Corporate Survival

Markus Evans

Consumer Intelligence & Analytics

Conference

Park Hyatt, Melbourne, Australia

22-23 August 2013

http://au.linkedin.com/in/ormondejames

http://au.linkedin.com/in/ormondejames

Slide 2Copyright 2005 Paul Ormonde-James, [email protected]

Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s

We live in a digital

world


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sUbiquitous Computing

More data from more machines


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sVirtualisation

Storage getting

cheaper by the minute


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sCloud Computing

Data now physically all over

the world and growing at an increasing rate


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sSocial Media…. BIG

DataMore people

communicating with more people, social data

increasing


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s

InformationOverload


Every

shot

coun

tsEvery

shot

coun

ts

8

Needle in a hay stack

Finding the right customer data is likeLooking for a needle in a hay stack


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sHow do we makes sense of all this???


Every

shot

coun

tsEvery

shot

coun

tsThe Challenge?

How do I find the right

customers, at low cost to

make millions?

1 OFFER = 1 ACCEPT


Every

shot

coun

tsEvery

shot

coun

ts

How do companies understand WHO to market to?? CONTACT

How do companies approach YOU? CHANNELS

How do companies understand YOUR needs SEGMENTS

I’m an individual

I’m an individual

I’m an individual

I’m an individualI’m an

individual I’m not


Every

shot

coun

tsEvery

shot

coun

ts

12

The Ying & Yang

Data Quality

Analytics

TODAY I will address


Every

shot

coun

tsEvery

shot

coun

ts

13

My Experience

Data Quality

Insight Services Group

PostConnect

Data & InsightsAustralia’s largest fact based address & lifestyle profile database

• Postal address File• AMAS data• National Change of Address data• Movers data• Australian Lifestyle survey• eProducts & insights• Parcels insights >>


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s

Bringing insights from data across the whole of Australia Post

Retail insights

Geospatial insights

Address insights

Parcel insights

Movers insights

Campaign insights


Every

shot

coun

tsEvery

shot

coun

ts

16

My Experience

Analytics


Every

shot

coun

tsEvery

shot

coun

ts

17

The World Bank

Head of Global Business Intelligence, The World BankWashington DCUSA

• Manage modeling & analysis teams for global impacts on banking sectors

• Managed global teams collecting data from 182 countries• Managed predictive analysis teams on portfolio risks &

cash flows• Built & managed Global teams for end to end data

control & analysis


Every

shot

coun

tsEvery

shot

coun

ts

18

The Analytic 4 P’s

PEOPLE

PLAT

FORM

S

CUSTOMER

PRO

CESSES

PERFORMANCE

Slide 19

AnerdLYTICS – The human face of numbers

and analysis


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sAna Lytics

Ana

Ana Lytics – Power through people


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sBusiness Intelligence – Simple form


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s

Build the technology like pieces of a jigsaw

puzzle

22


Every

shot

coun

tsEvery

shot

coun

ts

23

My Experience

Data Quality


Every

shot

coun

tsEvery

shot

coun

tsData Quality imperitives

24

• The data is consistent - • The databases are well designed - A well-designed database will

perform satisfactorily for its intended applications, it is extendible and it exploits the integrity capabilities of its DBMS.

• The data is not redundant - • In actual practice, no organization has ever totally eliminated

redundant data. In most data warehouse implementations, the data warehouse data is partially redundant with operational data. For certain performance reasons, and in some distributed environments, an organization may correctly choose to maintain data in more than one place and also maintain the data in more than one form.

• The redundant data to be minimized is the data that has been duplicated for none of the reasons stated above but because:

| Copyright – Paul Ormonde-james 2013


Every

shot

coun

tsEvery

shot

coun

ts

25

• The creator of the redundant data was unaware of the existence of available data.

• The redundant data was created because the availability or performance characteristics of the primary data were unacceptable to the new system. This may be a legitimate reason or it may also be that the performance problem could have been successfully addressed with a new index or a minor tuning effort and that availability could have been improved by better operating procedures.

• The owner of the primary data would not allow the new developer to view or update the data.

• The lack of control mechanisms for data update indicated the need for a new version of the data.

• The lack of security controls dictated the need for a redundant subset of the primary data.

– In these cases, redundant data is only the symptom and not the cause of the problem. Only managerial vision, direction and a robust data architecture would lead to an environment with less redundant data.



Every

shot

coun

tsEvery

shot

coun

ts

• The data follows business rules - • As an example, a loan balance may never be a negative number.

This rule comes from the business side and IT is required to establish the edits to be sure the rule is not violated.

• The data corresponds to established domains - • These domains are specified by the owners or users of the data.

The domain would be the set of allowable values or a specified range of values. In a human resource system, the domain of sex is limited to "male" and "female." "Biyearly" may be accurate but still not an allowable value.

• The data is timely - • Timeliness is subjective and can only be determined by the users

of the data. The users will specify that monthly, weekly, daily or real-time data is required. Real-time data is often a requirement of production systems with online transaction processing (OLTP). If monthly is all that is required and monthly is delivered, the data is timely.


Every

shot

coun

tsEvery

shot

coun

ts

• The data is well understood - • It does no good to have accurate and timely data if the users

don.t know what it means.

• Naming standards are a necessary (but not sufficient) condition for well-understood data.

• Data can be documented in the meta data repository, but the creation and validation of the definitions is a time-consuming and tedious process.

• This is, however, time and effort well spent.

• Without clear definitions and understanding, the organization will exhaust countless hours trying to determine the meaning of their reports or draw incorrect conclusions from the data displayed on the screens.


Every

shot

coun

tsEvery

shot

coun

ts

28

The data is integrated - • If the data is integrated, meaningful business information can

be readily generated from a combination integration generally requires the use of a common DBMS.

• There is an expectation (often unfulfilled) that all applications using the DBMS will be able to easily access any data residing on the DBMS.

• An integrated database would be accessible from a number of applications.

• Many different programs in multiple systems could access and, in a controlled manner, update the database. Database integration requires the knowledge of the characteristics of the data, what the data means, and where the data resides.

• This information would be kept in the meta data repository.



Every

shot

coun

tsEvery

shot

coun

ts

29

• The data satisfies the needs of the business -

• The data has value to the enterprise. High quality data is useless if it's not the data needed to run the business. Marketing needs data on customers and demographic data, Accounts payable needs data on vendors and product information.

• The user is satisfied with the quality of the data and the information derived from that data -

• While this is a subjective measure, it is, arguably, the most important indicator of all. If the data is of high quality, but the user is still dissatisfied, you or your boss will be out of a job.

• The data is complete - • All the line items for an invoice have been captured so

that the bill states the full amount that is owed. All the dependents are listed for an employee so that invoices from medical providers can be properly administered.



Every

shot

coun

tsEvery

shot

coun

ts

30

• There are no duplicate records - • A mailing list would carry a subscriber, potential buyer or

charity benefactor only once. You will only receive one letter that gives you the good news that "You may already be a winner!“

• Data anomalies - • From the perspective of IT, this may be the worst type of

data contamination. A data anomaly occurs when a data field defined for one purpose is used for another.

• For example, a currently unused, but defined field is used for some purpose totally unrelated to its original intent.

• A clever programmer may put a negative value in this field (which is always supposed to be positive) as a switch.



Every

shot

coun

tsEvery

shot

coun

ts

31

My Experience

Analytics


Every

shot

coun

tsEvery

shot

coun

ts

32

• “Analytics is the combustion engine of business, and it will be necessary for organizations that want to grow, innovate and optimize efficiency,”

• “Given its far-reaching impact, it is one of the few software markets that thrive even in adversity.”

• Gartner analyst Rita Sallam



Every

shot

coun

tsEvery

shot

coun

tsDecline of the Business Intelligence Stack

33

• Mega vendors IBM, SAP, Oracle, Microsoft and SAS still own the dominant share of the business intelligence market. SAP leads in business intelligence platform revenue

• Mega vendors own two-thirds share today versus one-third in 2007. But the area of analytics is still an open playing field. Data discovery vendors such as QlikTeck, Tableau and Tibco Spotfire grew more than the others

• “A stack-centric mentality and single vendor standardization policies won't cut it,” she said. “Understand that your organization needs a portfolio of analytic capabilities.” Gartner

• Companies now differentiating between the idea of selecting a single business intelligence vendor and establishing business intelligence standards.


http://www.tibco.com/


Every

shot

coun

tsEvery

shot

coun

ts

34

• Standardization is becoming less about using the same toolset and more about employing specific tools, metrics, and processes for certain capabilities and use cases.

• This shows up in a decline in the number of users identifying vendors as their business intelligence standard.

• All the mega vendors experienced a drop, some by as much as 19 percent in only three years.



Every

shot

coun

tsEvery

shot

coun

tsDemographics Drive Data Discovery

35

• Another area that is changing the business intelligence market is demographics.

• Millennials (ages 20 to 30) now comprise 20 percent of the workforce, but their ranks will swell to 40 percent by 2020.

• “The graduating high school class of 2011 spent all of their school years with pervasive access to the Internet – they don’t know a world without information at their fingertips,” said Sallam.

• “You tell them to go to the library to use the card catalog, and they look at you if you told them to go use an abacus to calculate the square root of 1,058.”

• These younger employees are driving the consumerization of IT, which includes how business intelligence is delivered. They want business intelligence to be as intuitive, social and collaborative as the tools in their personal life.


http://en.wikipedia.org/wiki/Generation_Y

http://www.datamation.com/datbus/article.php/3890361/Consumerization-of-IT-Poorly-Understood.htm

http://www.datamation.com/datbus/article.php/3890361/Consumerization-of-IT-Poorly-Understood.htm


Every

shot

coun

tsEvery

shot

coun

tsDemographics Drive Data Discovery

36

• Thus traditional reporting and ad hoc query are flat or declining, whereas data visualization in dashboards and interactive visualization are experiencing growth. “BI is becoming more visual as data discovery needs intensify,” Sallam said.

• The data discovery market will rise from $591 million currently to $1 billion by 2013, Sallam said.

• Data discovery tools typically include a mix of in-memory analytics, data mashup capabilities, dashboards, self-service delivery, light footprint and speed of deployment. These tools have become so popular that established vendors are copying them.

• Demographics are also contributing to the growth of mobile business intelligence. Gartner predicts that 33 percent of analytics will be consumed on handhelds by 2013.

• By the end of this year, 55 percent of organizations using business intelligence either have or plan to deploy mobile BI. As a result, mobile BI projects will outnumber traditional workstation projects by four to one within three years. This has everything to do with competitive advantage.


http://www.enterpriseappstoday.com/business-intelligence/business-intelligence-is-going-mobile-1.html


Every

shot

coun

tsEvery

shot

coun

tsThe Rise of Predictive Analytics

37

• Gartner surveys show most users still focus on measurement of the past, with only 13 percent of users making extensive use of predictive analytics. Less than 3 percent use prescriptive capabilities such as decision/mathematical modeling, simulation and optimization.

• Gartner advises organizations to develop a plan to support new data volume, variety and velocity requirements. By being able to correlate, analyze and present insights from structured and unstructured information, organizations will be able to personalize customer experiences and exploit new opportunities.

• “Those that can do advanced analytics on top of Big Data will grow 20 percent more than their peers,” said Sallam. “The explosion of data volume, as well as its variety and velocity, will enable new, high-value advanced analytic use cases that drive growth and productivity.” | Copyright – Paul

Ormonde-james 2013


Every

shot

coun

tsEvery

shot

coun

ts

38

How is it done


Every

shot

coun

tsEvery

shot

coun

tsThe how is important

• Regardless of methodology, most processes for creating predictive models incorporate the following steps:– 1. Project Definition: Define the business objectives and desired

outcomes for the project and – translate them into predictive analytic objectives and tasks. – 2. Exploration: Analyze source data to determine the most appropriate

data and model – building approach, and scope the effort.– 3. Data Preparation: Select, extract, and transform data upon which to

create models. – 4. Model Building: Create, test, and validate models, and evaluate

whether they will meet – project metrics and goals. – 5. Deployment: Apply model results to business decisions or processes.

This ranges from – sharing insights with business users to embedding models into applications

to automate – decisions and business processes.– 6. Model Management: Manage models to improve performance (i.e.,

accuracy), control access, promote reuse, standardize toolsets, and minimize redundant activities.


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

sWhat does research show us


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s


Every

shot

coun

tsEvery

shot

coun

ts

• Barriers to Usage. A host of barriers can prevent organizations from venturing into the domain of predictive analytics or impede their growth. This “analytics bottleneck” arises from: – 1. Complexity.

• Developing sophisticated models has traditionally been a slow, iterative, and labor intensive process.

– 2. Data. • Most corporate data is full of errors and inconsistencies but most

predictive models require clean, scrubbed, expertly formatted data to work.

– 3. Processing Expense. • Complex analytical queries and scoring processes can clog

networks and bog down database performance, especially when performed on the desktop.


Every

shot

coun

tsEvery

shot

coun

ts

• 4. Expertise. – Qualified business analysts who can create sophisticated models

are hard to find, expensive to pay, and difficult to retain.

• 5. Interoperability. – The process of creating and deploying predictive models

traditionally involves accessing or moving data and models among multiple machines, operating platforms, and applications, which requires interoperable software.

• 6. Pricing.

– The price of most predictive analytic software and the hardware to run it on is beyond the reach of most midsize organizations or departments in large organizations.

Fortunately, these barriers are beginning to fall, thanks to advances in software, computing, and database technology.


Every

shot

coun

tsEvery

shot

coun

ts

45

We are known for our unique offering – enabling

smarter connections.


Every

shot

coun

tsEvery

shot

coun

ts

Pa

ul O

rmo

nd

e-ja

me

s

Conclusions.....

46

Slide 47

ByPaul Ormonde-JamesTwitter @pormondejamesLinked inEmail: [email protected]

When every shot countsWhen every shot counts

What will you do to differentiate?