Upload
amberlynn-mills
View
213
Download
0
Embed Size (px)
Citation preview
A Study of the Model and Methodology for A Study of the Model and Methodology for
Institute Evaluation in CASInstitute Evaluation in CAS
Xiaoxuan Li, Bing Shi, Jianzhong ZhouInstitute of Policy and Management, CAS
Management Innovation and Evaluation Research Center, CAS
ContentsContents1.1. Brief introduction of CAS and Evaluation Brief introduction of CAS and Evaluation
Practice in CAS InstitutesPractice in CAS Institutes
2.2. Three-hierarchy Evaluation ModelThree-hierarchy Evaluation Model
3.3. ConclusionsConclusions
Brief Introduction of CAS and Brief Introduction of CAS and Evaluation Practice in CAS InstitutesEvaluation Practice in CAS Institutes
Brief introduction of CASBrief introduction of CAS The Chinese Academy of Sciences (CAS) was founded in Beijing on
November 1, 1949. Consisting of the Academic Divisions and various subordinate institutions, it is the lead national academic institution in natural sciences, a major advisory body to the government on science and technology related issues, and a national comprehensive research and development center in natural sciences and high technology areas in China.
The general objectives of CAS are to develop into a base for scientific research, for training high caliber scientific talent and for incubating high-tech industries in China; to become a national scientific think tank and to evolve into a national research institution that boasts “first-class achievements, first-class efficiency, first-class management and first-class talent.”
CASCAS
Academic DivisionsAcademic Divisions CAS HeadquartersCAS Headquarters
Main InfrastructureMain Infrastructure
- - 17 Large-scale Scientific Resear17 Large-scale Scientific Resear
ch Facilitiesch Facilities
- - 7 National Labs7 National Labs
- - 5 Field Stations Networks5 Field Stations Networks
- - 36 National Engineering Center36 National Engineering Center
ss
- - 273 Technology Transfer Center273 Technology Transfer Center
ss
- - 317 Journals 317 Journals
- - 46 National Associations and So46 National Associations and So
cieties cieties
–Division of Mathematics and Division of Mathematics and PhysicsPhysics
–Division of ChemistryDivision of Chemistry–Division of Life Sciences and Division of Life Sciences and
MedicineMedicine–Division of Earth Sciences Division of Earth Sciences –Division of Information Technical Division of Information Technical
SciencesSciences–Division of Technological SciencesDivision of Technological Sciences
Members 709Members 709
Foreign Members53 Foreign Members53
Committee for Consultation Committee for Consultation
and Review and Review
Committee on Scientific EthicsCommittee on Scientific Ethics
Committee for Science Committee for Science
Popularization and PublicationPopularization and Publication
Institutions Directly Institutions Directly under CASunder CAS
-- 94 Research Institutes94 Research Institutes
-- 2 Universities and Schools2 Universities and Schools
-- 2 Supporting Units 2 Supporting Units
-- 3 Botanical Gardens 3 Botanical Gardens
-- 12 Branches12 Branches
-- 2 Press and Publication Comp2 Press and Publication Comp
aniesanies
-- 1 Assets Management Compa1 Assets Management Compa
nyny
-- 22 Holding Enterprises 22 Holding Enterprises
Distributed in 22 Distributed in 22 provinces and cities provinces and cities
over China.over China.
Overview of CASOverview of CAS
In 1998, with the approval of the Chinese government, the CAS launched the Knowledge Innovation Program (KIP), and has since turned over a brand new page in its history of development.
In the process of conducting the KIP, in recognition of the national strategic requirements and the world trend in science and technology, the CAS has made the most profound and extensive adjustments in its disciplinary deployment and organizational structure since its founding in 1949. Scientific restructuring and management innovation has led to the preliminary formation of a structure and mechanism that are both geared to a socialist market economy and aligned with international standards.
Changes of Concepts in Institutes Changes of Concepts in Institutes Evaluation in CASEvaluation in CAS
Changes of Concepts in Institutes Changes of Concepts in Institutes Evaluation in CASEvaluation in CAS 15Years/3Period15Years/3Period
From 1993 to 1998, the institutes evaluation in CAS was
focused on research outputs ,it is called output evaluation.
From 1998 to 2004, the CAS launched the Knowledge
Innovation Program (KIP), the institutes evaluation in CAS was
focused on achievements and level of accomplishment, it is called
performance evaluation.
From 2005 to now, the institutes evaluation in CAS was focused
on S&T creativity and concerned performance in integrity, it is
called quality-orientated comprehensive evaluation.
Three-hierarchy Evaluation ModelThree-hierarchy Evaluation Model
Background and Tasks
• Comprehensive Complementary Reform Pilot Project
– In 2007, seven pilot institutes have been selected to investigate modern R&D institutions system
– Four thematic issues haven been launched in the following aspects respectively: Resource Allocation, Human Resources, Organizational model of R&D, Evaluation.
– Basic Tasks of Evaluation Issue: To study evaluation models for different types of institutes.
Categories of 7 Pilot Institutes
Institute of Institute of High Energy High Energy PhysicsPhysics
Cold and Arid Cold and Arid Regions Regions Environmental Environmental & Engineering & Engineering Research Research InstituteInstitute
Shanghai Shanghai Institute of Institute of Technical Technical PhysicsPhysics
Institute of Institute of Computing Computing TechnologyTechnology
Dalian Dalian Institute of Institute of Chemical Chemical PhysicsPhysics
Institute of Institute of MicrobioloMicrobiologygy
Institute of Institute of Physics Physics
• Large scale scientific Large scale scientific facilitiesfacilities
• Resources, Environment Resources, Environment and institute in western and institute in western ChinaChina
• Engineering Engineering and big teamsand big teams
• Basic Basic researchresearch
• High-tech High-tech ResearchResearch
• Comprehensive Comprehensive instituteinstitute
• Biological Biological researchresearch
Yearly quantitative and qualitative monitoring
Peer review
At the bottom hierarchy are indicators that can reflect common characters of the institutes, including yearly quantitative monitoring and yearly communication review
At the second hierarchy are the key indicators and benchmarks, which reflects the characters of certain category of institute. It can be used in both diagnosing evaluation for individual institute and comparison between institutes of the same category.
At the top hierarchy is qualitative evaluation by experts review, and it is also a hierarchy of individual evaluation. Experts review include two stages: diagnosing evaluation for individual institutes and comparing evaluation among the institutes of the same category.
Quantitative Qualitative
Three-hierarchy Evaluation Model
Key indicators
Key indicators and benchmark
The selection of key indicators: Based on the orientation and charac
teristics of institutes, we selected no more than 6 key indicators for e
ach pilot institute. Totally, there are 3 common key indicators and 8
specific key indicators.
There are four methods to set the benchmark in our research : 1) th
e more the better, for example the more awards, the better; 2)differ
ent standards will be given based on international comparison; 3)be
chmark will be set according to the development level of institutes;
4)judged by experts experience.
key indicators of 7 pilot institutes
共性指标:
不同研究所的共性因素
Common key indicators
S&T Talents
per capita funding
Awards
Specific key indicators
Invited reports of important conference or workshop
Accomplishment of Major Tasks
Construction and operation of large research facilities
Platform of resource and data
Intellectual Property
High quality papers
Engineering Application and Demonstration
Advisory Reports
The methods to set common key indicators of benchmark
CountryInstitutio
nper capita funding (ten
thousand dollars)per capita funding for researc
her (ten thousand dollars)
USA NIH 15.4 46.2
USA NIST NA 31.8
German MPG 18.8 51.9
German HFG 12.4 39.6
German FhG 13.8 NA
France CNRS 12.5 34.6
Japan RIKEN 32.3 41.4
Japan AIST 28.7 36.9
Korea KIST 17.7 27.5
Australia CSIRO 13.9 15.6
indicators The method to set benchmark
S&T Talents Make use of the definition of quantitative monitoring on innovation capacity about the S&T Talents
per capita funding
With reference to international research institutions and the real per capita budget of this year.
Awards More the better
The methods to set the specific indicators of benchmarks
Specific key indicators are determined by separately by
the 7 pilot institutes.
The study is expected to be accomplished by the end of
this year.
Conclusions
Conclusions
——Through the study of 7 pilot institutes, key indicators and bench
marks regarding different types of institutes were proposed, includin
g key quantitative indicators and benchmarks as well as key qualitati
ve indicators and anchoring method. Thereby, three hierarchy evalu
ation model has been set up.
——the new model puts more emphases on the character of evaluati
on by category and individual features of institutes.
—— In summary, this model has developed Quality-orientated Comp
rehensive Evaluation system in methodology.
ThanksThanks
Contact information:
www.casipm.ac.cn