Agile Enterprise Data Management: Terms, Models, Universal Warehouses

Phasic Systems Inc

www.phasicsystemsinc.com703-945-1378

2

Phasic Systems Inc Management

• Geoffrey Malafsky, Ph.D, Founder and CEO▫Research scientist▫Supported many organizations in their quest to access the right

information at the right time• Tim Traverso, SVP Federal

▫Former Technical Director, Navy Deputy CIO• Kevin Moran VADM (Ret), COO

▫Former Deputy Chief of Naval Personnel• Marshall Maglothin, SVP HealthCare• Deborah Malafsky SVP Business Development

3

Our Agile Methods

• Why be Agile?▫Provide flexibility and adaptability to changing business needs while

maintaining accuracy and commonality▫Segmented approach is too slow, rigid, and costly

• How?▫Treat data lifecycle as one continuous operation from governance to

modeling to integration to warehouses to Business Intelligence▫Emphasize value produced at each step and overall coordination▫Seamlessly fit with existing organization, procedures, tools but add Agility,

commonality, flexibility, and reduced cost and time• We are Agile and comprehensive

▫Typical 60-90 day engagement▫Deliver completed products not just plans or partial results

4

Methods and Tools• DataStar Discovery: Agile system modeling, data modeling, data semantics

▫ Policy, governance, standards and design▫ Fully integrated metadata: each element, table includes attributes for

business, IT, and BI▫ Point-select data models, codes, rules, in Data Lifecycle Model▫ Easily build common term standards using all variations with clear

understanding of business use scope, meaning▫ Rapid collaborative agreement; line-of-sight to operations

• DataStar Unifier: Agile Implementation ▫ Agile warehousing and aggregation▫ Simplified, common semantics using Corporate NoSQL™▫ Aggregate data using all use case and system variations simply and easily

into standard or NoSQL databases▫ Universal warehouse with flexibility, extensibility, adaptability

5

“As a COO of a Wall Street firm and a former US Navy Vice Admiral in charge of a large integrated organization of thousands of people and numerous IT systems, I have seen firsthand the critical role that high-quality enterprise data plays in day-to-day operations of an organization. Without timely access to reliable and trusted data all of our operations were vulnerable to poor decision making, weak performance, and a failure to compete. With Phasic Systems Inc.’s agile methodology and technology, we were finally able to solve our data challenges at a fraction of the time, cost, and organizational turmoil that all the previous and more expensive, time-consuming approaches failed to do.”

PSI Customer Testimonial: VADM (ret) J. “Kevin” Moran

6

Agility Across the Data Life CycleTraditional►Slow (9-24 months)► Fractious (business,IT, BI)► Expensive► Inflexible► Poor practicalsemantic mapping

Agile EDM►Fast (1-3 months)► Coordinated (business, IT, BI)► Large decrease $► Flexible (daily)► Easy, adaptive semantic mapping

Products►Data models (all)► Unifying, concise Data System Model► Glossaries► Codes►Semantic mapping (data, XML,Legacy, SOA)► Feed MDM, BI► Universal Data Warehouse►Managed governance – integration - analytics

7

Agile: Overcome Hurdles

• Group rivalry▫Embrace important business variations; recognize no valid reason to

force everyone to use only one view exclusively. • Terminology confusion

▫Use a guided framework of well-known concepts to rapidly identify, and implement variations as related entities.

• Poor knowledge sharing ▫Use integrated metadata where important products (business models,

data models, glossaries, code lists, and integration rules) are visible, coordinated, and referenceable

• Inflexible designs▫Use a hybrid approach (Corporate NoSQL™) for Agile warehousing and

integration blending traditional tables and NoSQL for its immense flexibility and inherent speed

Data Meaning Not Just Metadata

Must be agile in order to adapt quickly to new business needs ▫Continuous change is norm: requirements, consolidation▫We must use all the important business variations of key terms (e.g.

account, client, policy) – No such thing as single version for all!

GovernanceDesign MDM

Integration

Which Value? Whose? My “customer” or your “customer”?

Sales, Accounting

CEO/CFO/CIO SAP/IBM/ORACLE

How is data used?

Ontologies

9

Data System Model

10

Real Estate Listing Example

•Seems simple and well-defined▫Each house has a type, id, address, etc..▫Industry standards: OSCRE, RETS

•Yet, data systems are very different▫Data model tied tightly to business workflow▫Extensions and “make-it-work” changes added over time

•Similar to customer relationship mgmt, ERP, and many other fields

11

NKY HomeSeekers Texas

Main Hurdle to Enterprise Data Standards and Warehouse: Data Values Not Synchronized with Metadata

Different Meanings (Legal and Business Activities)

12

13

Fully Integrated Metadata for Business, IT, and BI

14

15

16

Corporate NoSQL™•NoSQL gives large systems flexibility & high performance

▫Simple key-value pairs: little data modeling▫ Inherent hardware performance: no database joins (10000x faster)▫But, poorly suited for corporate use – lacks connection to business

•Corporate NoSQLTM

▫Blends traditional techniques and NoSQL ▫Tables provide direct alignment to business concepts▫Key-value pairs eliminate need to delineate every attribute▫Business driven terminology in production model▫Easily handles semantic variations ▫Updates do not require changes to data models or physical stores

17

Position Data Model

Results•Applied to production data:

▫Fully cleaned & integrated data governance approved Requirement: 500,000 records in 2 hrs on Sun E25K Actual: 50 minutes on 3 year low-cost server

•Governance documents produced and approved▫Legacy data models – first time in ten years▫Common data model – directly derived from ontology.

Position-Resume model•Standing governance board created with short decision-

making monthly meetings▫Position-Resume Governance Board

•Process approach and technology applied to new IT systems