CIM Certification Program - Distributed Management Task …€¦ · · 2018-03-13CIM Certification Program Deborah May The Open Group ... test suite or certification system and

CIM Certification ProgramCIM Certification Program

Deborah MayThe Open Group

[email protected]

DMTF 2002 Developers' Conference June 10-13, 2002 Page 2

AgendaAgenda

• Certification Program– Overview of program

• Test Suite– Overview of Test Suite Beta Release


The Open GroupThe Open Group

Global Vendor/Technology Neutral IT Consortium

Develop consensus and facilitate

interoperability

Evolve & integrate specs and open

source technologies

Provide certification services and facilitate

procurement of certified products

Provide consortia infrastructure

services

Bringing IT suppliers & buyers together to solve enterprise interoperability problems

CONSORTIA


Certification Certification Experience Experience -- ExamplesExamples

Free Standards GroupLSB (Linux)


AgendaAgenda


• Summary of program deliverables• Certification Policy document



Program SummaryProgram Summary

• Certification Policy• Compliance Specification

Core Documents- form foundation of program

• Certification Program Guide• Problem Reporting & Interpretations System• Registration Package (Registration Form, Agreements)

Implementation Components used by suppliers

• Operations Manual• Test Suite Validation Procedures

Implementation Components used by operational staff


Certification PolicyCertification Policy

• Creation and operation of the program– Achieving and maintaining certification– Processes to address problems during the

certification process

• Broad enough to cover future certification needs– Does not include anything:

• That is likely to change frequently• Is not subject to DMTF review & approval processes


CIM CertificationCIM Certification

Certifying a MOF• MOF file and a unique identifier string

• Version of the standard CIM Schema to which the MOF is compliant

• MOF may represent the model implemented in a software service/feature of the supplier’s product

• Requirements for conformant MOF are specified in CIM Compliance Specification

• For DMTF members only


TerminologyTerminology

Specification Authority(SA)

Develops, maintains, interprets specifications

Certification Authority(CA)

Manages day-to-day operations of program

Test Suite Maintenance Authority (TSMA)

Maintains test suites

Ensures repeatability, reproducibility, and objectivity (fairness)


Certification ProcessCertification Process

Pre-certification activity

1b. Supplier performsdevelopment & QA testing

1c. Supplier becomesfamiliar with program

requirements

CertificationAgreement

CertificationPolicy

CertificationProgram

Guide

ComplianceSpecification

1a. Supplier obtains testsuite

A



RegistrationRegistration

Form

CertificationAgreement

(signed)

2. Supplier registers forcertification

3. CA checks initialcertification information

Fees(if any)

Problem withRegistration

OK

MOF

B

A



Testing and Audit4. Supplier performs formal testing

5. CA audits certificationinformation

Testresults

6. Notification & publication ofcertification

ConformanceRequirements

not met

Source ofproblem?

Testingproblem

CertificateCertificationRegister

Auditsuccessful Supplier may hold back

publication for period of time

Supplier submits PRvia Problem Reporting

& InterpretationsProcess

Specification,Test Suite orCertification

System problem

B


CertificationCertification

Awarded based on validation of both test results and supplier’s claim of conformance

Conformance demonstrated through use of test suites

Testing provides an indicator of compliance


Supplier ObligationsSupplier Obligations

Which means that:– MOF conforms to an industry standard specification– MOF will remain conformant throughout the life of

the MOF– If there is a non-conformance, the MOF will be fixed

in a timely manner

Supplier must “warrant and represent” that MOF meets, and will continue to meet, the

applicable conformance requirements


Duration & RenewalDuration & Renewal

• Certification is valid only during period in which supplier is DMTF member

• Certification must be renewed annually in conjunction with renewal of DMTF membership

• Supplier must demonstrate that Certified MOF continues to meet conformance requirements


Certification RegisterCertification Register

Includes:• Name of supplier• Unique identifier for MOF• Version of the standard CIM Schema

against which the MOF was certified• Contents of MOF file

Public, web-accessible record of all Certified MOFs


Test suite Test suite acceptance processacceptance process

• Documented process for formally accepting new test suites for use in the program

• New test suite releases must:– Undergo a beta testing process

• Faults found are rectified prior to approval process

– Be “approved” by DMTF prior to use for formal testing

• CA maintains list of currently approved test suites on its web site


Test Suite updatesTest Suite updates

Maintenance release

• Fix test suite problems or interpretations of the spec

• 2-week beta test period• Once approved, replace

existing releases after 3-month overlap period

Enhancement release

• Functional upgrade as well as extensions to coverage

• 3-month beta test period• Once approved, replaces

existing releases after 6-month overlap period


Problem ResolutionProblem Resolution

Answers must be timely

Answers must have “industry consensus”

Answers must be available to all members

Anonymity must be preserved

Allows any supplier to raise questions about the specification, test suite or certification system and get definitive answers

Problem Reporting & Interpretations Process


Problem ResolutionProblem ResolutionSupplier files a Problem

Report for a problemfound during certification

process

Preliminary Review byPR review group

Opinion Submitted byTSMA or CA

3 businessdays max

5 businessdays max

CAcategorizes

PR

Specificationissue

Test Suite or CertificationSystem issues

A


Problem ResolutionProblem Resolution

FinalResolution?

Detailed Review byPR review group

CA Notifies Supplierof Resolution

Yes

SupplierSatisfied?

Supplier invokesappeals process

NoProvide preliminaryresolution to Supplier

(if possible)

No

10 businessdays max

Return toCertification

ProcessYes

Initial PRresponse

A

Resolution(Interpretation, TSD,

CSD, Rejected)

FurtherConsideration


Appeals ProcessAppeals Process• Suppliers can appeal decisions made by CA or DMTF

• Two-level appeals process:

– Technical Review & Board Review

• Deterministic timeframes for response, escalation

Supplier raises appeal

via CA CA arranges review & monitors process

Yes - CA represents

supplier

No - Supplier presents case

for review

Anonymous Review?


ConfidentialityConfidentiality

• All information is confidential between supplier and CA during certification process

• Once certification is achieved, it’s public information - supplier info and MOF entered into certification register– Test results always confidential– Option to remain confidential for up to 6 months


ReRe--certificationcertification

• Scope of certification: submitted MOF• Any modifications to MOF constitute new MOF

– Includes modification, addition, or removal of any class, property or method

– Requires re-testing and re-certification– Option to replace existing entry in certification

register or add new one

• Renamed MOF (modify identifier) with no changes to MOF requires amendment to certification register only


AgendaAgenda




Test SuiteTest Suite

Formal Testing• Test suite hosted on

CA’s web site• Testing performed by

supplier• Results submitted to CA

for audit

Pre-certification TestingDownloadable command-

line version of test suite available

Validates submitted MOF file against CIM Specification and a specified version of the

standard CIM Schema


ExampleExample

• Walkthrough of test suite execution using the web-based version– User enters URL of test suite server into their

browser– Initial test suite interface will prompt for user and

MOF information


User InformationUser Information


Upload MOF fileUpload MOF file

• User specifies version of standard CIM schema to validate against

• Browse file system to select MOF file to upload• Selecting “Submit” will upload the MOF file to

test server & execute test suite• MOF to be tested must be contained in a single

file


Upload MOF fileUpload MOF file


View ResultsView Results

• Upon completion of test execution, results will be displayed in browser by class

• Each property & method in the class will be listed with detail on the name, result and information on failures (if any). – Properties will also indicate attribute as required,

key or extended


View ResultsView Results


Submit results to CASubmit results to CA

• At bottom of results window, user has option to submit results to CA for certification or delete the results

• If user is satisfied with results– Enter brief description of results for CA– Select “Submit”– Results stored on test suite server & email

notification sent to CA

• If user wishes to delete results, they will be removed from server & user may repeat testing


Submit results to CASubmit results to CA


CIM Certification Test SuiteBeta Release

Accessible via the WBEM tools area of the DMTF web site:http://www.dmtf.org/standards/cimtools.php?section=member

Questions?Questions?