Upload
zita
View
127
Download
0
Tags:
Embed Size (px)
DESCRIPTION
CMR. Building a Corporate Metadata Repository. Past Present Future. Sam Highsmith Principal Researcher Statistical Research Division U.S. Census Bureau 301-457-1928 [email protected]. Agenda. The Past The Present Requirements for 2000 Questions. - PowerPoint PPT Presentation
Citation preview
Building a Corporate Metadata Repository
Past
Present
Future
CMRCMR
Sam HighsmithSam HighsmithPrincipal ResearcherPrincipal ResearcherStatistical Research DivisionStatistical Research DivisionU.S. Census BureauU.S. Census [email protected]@census.gov
Agenda
• The Past
• The Present
• Requirements for 2000
• Questions
What is the Corporate Metadata Repository?
• Electronic card catalog• Developed in conjunction with work from
Sweden, Canada, Australia, and UN/ECE Metadata Workshop
• Implements Business Data Model for Surveys and Censuses
• Provides a process for registering data set, variables, and documents for access and reuse
Past Milestones
• Built a Metadata Repository model • Built a Web enabled Prototype• Formal Agreements with other Bureau users• DADS adopted the Metadata Repository model• Developed a Transition to Production plan• Contract with Oracle to build a CMR • Economic Directorate Pilot Application
Metadata Repository Model
• Business Data Model– CASRO, SRD, consultant, Reinvention Lab– Survey and Census business model
• Data Element Registry– ISO 11179
• Built BDM and DER, then merged to CMR
• Modeled with ERD tool
Milestones PROTOTYPE
• Interface to Multiple Statistical Information Systems - DADS, FERRET, PCDOCS
• Table of Contents organization
• Search capabilities
• Oracle generated database for repository
• Completely Web based
Milestones
• Formal Memorandum of Understanding with major Bureau directorates
Milestones
• DADS adopted the Metadata Repository model• Implemented by Oracle Consulting for AFF
– Dissemination tool for 2000 Census data
– Accessible from www.census.gov
• DADS American Fact Finder is completely metadata driven
• !!!!!!!!!!!!
Milestones
• Developed a Transition to Production plan• Metadata support staff
• metadata providers in directorates
• Still waiting…...
Milestones
• Built a Metadata Repository model • Built a Web enabled Prototype• Formal Agreements with other Bureau users• DADS adopted the Metadata Repository model• Developed a Transition to Production plan• Contracted with Oracle to Build CMR
Review of CMR Implementation StrategyReview of CMR Implementation Strategy
Current Business Process does not include an Integrated Current Business Process does not include an Integrated
Metadata Business ProcessMetadata Business Process
Review of CMR Implementation StrategyReview of CMR Implementation Strategy
Current Business Process does not include an Integrated Current Business Process does not include an Integrated
Metadata Business ProcessMetadata Business Process internallydevelopedsystems
customizedcommercial
systems
CASES
variety ofprogramming
languages
individualtool of choice
internallydevelopedsystems
customizedcommercial
systems
CASES
variety ofprogramming
languages
individualtool of choice
CATICAPIMailPAPIOCRICR
CADECSAQTDE
PFIRS
CATICAPIMailPAPIOCRICR
CADECSAQTDE
PFIRS
internallydevelopedsystems
SAS
DEVSURV
COBOLFORTRANDECForms
StEPSECON DW
individualtool of choice
internallydevelopedsystems
SAS
DEVSURV
COBOLFORTRANDECForms
StEPSECON DW
individualtool of choice
AFF
CENSAS
FERRET
Econ DW
CD-ROM
Internet
AFF
CENSAS
FERRET
Econ DW
CD-ROM
Internet
Census 2000 ACS
DemographicSurveys
Econ Census
Econ Surveys
Design Collect Process Share
Review of CMR Implementation StrategyReview of CMR Implementation StrategyBenefits of a CMRBenefits of a CMR
Review of CMR Implementation StrategyReview of CMR Implementation StrategyBenefits of a CMRBenefits of a CMR
A C T I V E
CorporateMetadata
Repository(CMR)
-------------------Statistical MD
Geographic MDTechnical MD-------------------
Business Rules-------------------
Product Layouts
A C T I V E
CorporateMetadata
Repository(CMR)
-------------------Statistical MD
Geographic MDTechnical MD-------------------
Business Rules-------------------
Product Layouts
Launch Dissemination Systems
Support One View to Internet Users
Support “Virtual Statistical Agency”and sharing data/MD across web sites
Support Generation of All Kindsof Products, including Reimbursables
Support sophisticated analysisof BOC data (e.g. time series and crossdataset comparisons)
“Faster to Market” through reuse
Proposed Technical/Software Architecture Four Ways an Application Can Use CMR MetadataProposed Technical/Software Architecture
Four Ways an Application Can Use CMR Metadata
Tightly Coupled with CMR
Loosely Coupled with CMR
1. Application written against CMR - uses it directly for metadata access and maintenance.
2. Application uses same extensible CMR physical model - can replicate metadata from CMR. (e.g. DADS/AFF,ECON,LEHD)
3. Application communicates with CMR through an API to exchange metadata. (e.g. FERRET)
4. Application communicates with CMR using a standard XML-based metadata interchange. (future - e.g. FEDSTATS,BLS...)
CMR Meta-ModelCMR Meta-Model
• Meets ISO/IEC 11179 Data Element Registry standards
• Meets FGDC Content Standard for Digital Geospatial Metadata (CSDGM)
• Can support emerging metadata standards
• Extensible
• Shareable
• COTS where possible
Economic Directorate Pilot
• Pilot versus Prototype
• Focus on two stages of the information collection life cycle of the ASM
• Added functionality to two existing METADATA applications– Dissemination and Refics
• Used JAD, RAD, code generation• The “workers” jumped on board!• Found multiple existing metadata applications
A METADATA EXAMPLE FROM
The Annual Survey Of Manufactures
MetadataReuse
Computerized Self Administered Questionnaire
MetadataReuse
Editing Annual Survey of Manufacturers
BALANCING SIMPLE:B1:SW=WW+OW YYN 00.00 00.00 N N B2:TE=PW+OE YYN 00.00 00.00 N N B3:PW=(PW1+PW2+PW3+PW4)/4 YNNNN 00.05 00.05 00.50 N N TTHVB4:LC=LE+VP NYY 00.05 00.00 N N B5:CM=CP+CR+CF+EE+CW YNNNNN 00.05 00.05 00.50 N N NTTHVB6:TIB=WIB+MIB+FIB YNNN 00.25 00.25 00.50 N N NTTHVB7:TIE=WIE+MIE+FIE YNNN 00.25 00.25 00.50 N N NTTHVBALANCING T-ADJUSTMENTS:B03A 0,4.1,4.1,4.1,4.1B03B 0,3.2,3.2,3.2,3.2B05A 0,1.0,0.8,0.5,0.5,0.9 0,4.4,4.6,4.6,4.5,4.6B05B 0,1.0,0.8,0.5,0.5,0.9 0,2.0,2.0,2.0,2.0,2.0B06A 0,4.3,4.3,4.3B06B 0,2.0,2.0,2.0B07A 0,4.3,4.3,4.3B07B 0,2.0,2.0,2.0BALANCING FENCE-CONSTANTS:0 0 11 0.9 1.12 0.75 1.253 0.5 1.54 0.2 55 0.05 106 0.0 20
MetadataReuse
ASM IRACS
MetadataReuse
MetadataReuse
A FewViews of the Pilot Application
Year 2000 Planning
What are we building this year?
Hint: A production system!Hint: A production system!
Metadata Portal SiteMetadata Portal Site• Web-enabled
• Access to all CMR tools
• Promote knowledge management
• Secure access (role-based & by individual)
• Support Collaboration and Personalization:
• BOC Public Access to Tools and Folders
• Collaboration areas for BOC Projects and Organizations
• Personalization
– Customize own CMR Web Site to short-cut access and add personal work
– Publish personal work, if desired
Metadata ManagementMetadata Management
DEPARTMENT
EMPLOYEES SALARY
DEPT
EMP SAL
Meta-ModelMeta-ModelManagementManagement
RepositoryRepositoryManagementManagement
Architectural Architectural SolutionsSolutionsManagementManagement
AdminTools
Bulk Load& UnloadServices
BrowsingTools
MetadataExchangeServices Publish on Web
Search structured & unstructured MDDrill-through to actual dataLaunch Applications
Security
NetworkSystemRepositoryMetadataFine Grain Ownership
Object-level VersioningMulti-path VersioningObject-level Check-in/Check-out
ConfigurationManagement
V1.0
V1.1 V2.0
Synchronize Repositories Master-Slave Relationships Same Meta-ModelMeta-Model BridgesXML-based Exchange
Formatted/Delimited ASCII FileXML-based FilePublish in many different standard formats
Create, Read, Update, DeleteArchiveBackup/Recovery
Object Layer and Open APIObject Layer and Open API
• Object layer over the CMR physical repository tables– The underlying meta-model could change with minimal
impact to the CMR tools and interfaces accessing the object layer.
• Provide fine-grain, role-based metadata security for access by CMR tools
• Publish an object oriented Open API for developers• Use wizards and generation tools as much as possible• Use meta-model generation tools (preferrable)
Metadata ManagementMetadata Management
• Web-enabled• Meta-model generated application• COTS repository administration• Supports CRUD manipulation of metadata• Supports copying and versioning metadata objects• Supports storage and retrieval of unstructured
metadata• Implements business rules, audit trails, and metadata
status for simple work flow
The CMR Metadata Management tool will enable the BOC business metadata owners and stewards to maintain their own metadata.
Oracle Designer Generated HTML/JS Applications
LogicalModels
MiddleTier
Deployment
ServerTier
Deployment
PhysicalModels
ClientTier
Deployment
FunctionalRequirements
UseCases
UMLObjectModel
ServerModel
WebModules
CMRRepository
TAPI(PL/SQL)
PL/SQLgeneratingHTML & JSApplication
Code
OASEnvironmen
tw/
PL/SQL Cartridge
&HTTP
Listeners
Web BrowserHTML
ApplicationHTTP
Net8
Net8
View Layer
Created/Generated usingOracle DesignerHand coded
Created/Generated usingRational Rose
Metadata Browsing and ReportingMetadata Browsing and Reporting
• Web-enabled• Meta-model generated application• Supports textual and structured metadata searches• Supports nested searches• Supports hierarchical, tree-based browsing (drill
up/down)• Supports search and browsing of Tier 1 documents
(accessible in the CMR repository or on the Census Intranet/Internet)
The CMR Metadata Browsing and Reporting tools will provide read-only metadata searching, hierarchical browsing, and advanced analysis and reporting of the CMR metadata.
Metadata Open InterchangeMetadata Open Interchange
• Use an open standard technology (such as XML) to interchange metadata with external systems
• Support bulk metadata object level refreshes• Support metadata validation and correction in staging
area prior to final load• Enforce role-based, object level security• Support web-enabled log of input/output activity• Web-enable UI as much as possible
The CMR Metadata Open Interchange provides an open, standards-based method for interchanging metadata between the CMR and external systems.
CMR <-> base SAS CMR <-> base SAS
• Web-enabled UI• Output metadata for a statistical dataset for use by
base SAS• Input metadata for a statistical dataset from base SAS• Support metadata validation and correction in staging
area prior to final load• Enforce role-based, object level security• Support web-enabled log of input/output activity
CMR and SAS metadata interchange tool will enable the metadata owner to perform a bi-directional metadata exchange, and will establish a common mechanism of loading metadata into SAS system and pushing it back to CMR repository.
Target: Production in 2000
?Questions
?