Upload
aubrey-pierce
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Proposal EvaluationPrasit Palittapongarnpim, M.D.Mahidol University
Proposal Is a description of R&D activities,
supposedly, in advance. Is a management tool for resources
allocation. Can be used as TOR between
researchers and sponsors. Hardly be accurate.
Sponsors’ purposes of evaluation Are the proposal goals fit to the funding goals? Is the methodology appropriate? Are the researchers capable to finish the
projects? Is the hosting institute ready and supportive
enough? Is the budget realistic? In some particular cases
cost-effectiveness may be considered? Other considerations
Ethics Safety Capacity building
Proposal Evaluation Methodology depends on The goals of the funding programs Funding strategy The number of applicants How many stages will proposals be
prepared? Levels of intellectual rights needed to be
protected
What are the goals of R&D MFU? Is MFU a Provincial University National University Regional University International University
Are the proposal goals fit to the funding goals and scope? What is the required output?
Proof of concepts Information Papers Patents Materials Products. Capacity building Policy translation (usually with papers) Industrial translation Proof of concepts Hypothesis formulations
What is the scope?
Possible Goal settings and Funding Strategies for various categories of R&D Descriptive information seeking (e.g., incidence,
prevalence): Top down/Commission Hypothesis-driven research (e.g., basic, target
discovery, etc.): Researcher initiatives/fully competitive
Candidate discovery (e.g., drug or vaccine candidates): Researcher initiatives/ scoped competition.
Product development (Preclinical and clinical): Major: Top down/commission with stage-gate
protocols Minor: Bottom up/competitive
Translation to policy(e.g., cost-effectiveness study, QOL): Top down/ Scoped competition
Translation to industry(e.g., Industrial technology assistance): Customer-based/commission and competition
มู�ลค่�าของ outputs จากโค่รงการเป็�นสิ่��งที่��น�กวิ�จ�ยค่าด แต่�มู�ป็�จจ�ยอ��นๆเก��ยวิข!องมูาก มู�กจะเก�ดข#$นต่�อเมู��อหน�วิยงานลงมู�อจ�ดการ
มู�ลค่�าแลกเป็ล��ยน (exchange value) เช่�น ขายสิ่�ที่ธิ�บั�ต่ร เก�ดธิ)รก�จ/รายได!
มู�ลค่�าใช่!สิ่อย (use value) เช่�น การให!บัร�การมู�ลค่�าสิ่�ญญะ (sign value) เช่�น เก�ดการ
ป็ระช่าสิ่�มูพั�นธิ.หน�วิยงาน
How many kinds of goals are needed in the proposals? Long term goals (visions) Outputs Operational objectives
In any cases it should be stated and clear Outputs and operational objectives
should be measurable and promised. Avoid overpromising
How are the proposals selected? Open competitive Scoped competitive
By areas of research By eligibility of applicants
Self-competitive Commission Concept- full -revised
Intellectual Rights in Proposal Contents. How much should be disclosed? How much can be protected during
submission? How confidential is the reviewing
processes?
Sponsors’ purposes of evaluation Are the proposal goals fit to the funding goals? Is the methodology appropriate? Are the researchers capable to finish the
projects? Is the hosting institute ready and supportive
enough? Is the budget realistic? In some particular cases
cost-effectiveness may be considered? Other considerations
Ethics Safety Capacity building
Evaluation of Methodology Flawed or unclear methodology is usually
amendable. Competitive granting may not allow that at the times of funding decisions.
Some (S&T) projects required un-established methodology, which require strong technical capacity of researchers.
Researchers may be more familiar with suboptimal technology. Decision is required to force researchers to better but unfamiliar methods or stick with the familiar method.
Methodology Evaluations Do we start from a right place? Is the
material good and most advanced? Literature review is NOT a part of project
methodology! Try to go beyond “me too”.
Is the pathway to the goal feasible and optimal?
“Optimal” may means Fastest Conclusive vs. Supportive vs. Suggestive Including risk management
Secrets in Methodology Secret recipe. Vague strategy
Common Pitfalls in Methodology Writing Wrong strategy/methods Inferior strategy/methods Vague strategy/methods Methods that researchers have not
enough experiences Too sketchy
Common Pitfalls in Methodology Evaluations MOST COMMON: Reviewers do not understand or
misunderstand the methods Diagrams usually help. Groups discussions are usually very helpful.
Reviewers have own methodology preferences Reviewers subsequently use it in a different
project but finish it before the applicant!? Should funder help improve the methods? YES. Is it fair? Depending on the goal and process of
funding. It is nor fair for competitive funding.
The purposes of evaluation Are the proposal goals fit to the funding
goals? Is the methodology appropriate? Are the researchers capable to finish the
projects? Is the hosting institute ready and
supportive enough? Budgets. Other considerations
Ethics Safety
Supporting Facility Space Equipment Special facility
Clinical cohorts Sample banking systems BSL3 Animal lab Pilot Production PD-PK facility Genomics facility High-performance computing
Research Budget Personnel Equipment Chemicals and consumables Travels Miscellaneous (+Contingency) Overhead/Utilities
Ethics Absolutisms, e.g.
Most major religions Relativisms, e.g.
Utilitarianism
Biosafety The purpose is to manage the risks at
the acceptable level, by the COMBINATION OF 3 methods Safe experimental protocols (Standard
operational protocols) Personal protection device Physical containment
All the textbooks are guidelines. Nothing can replace RISK-BENEFIT ANALYSIS.
Product Development A sequential multidisciplinary process, usually
need several institutes and good management. Have definite goals but can be of multiple
pathways Highly quality oriented (GLP (good laboratory
practice), GMP (..manufacturing..), GCP (..clinical..), GAP (..agricultural..))
Usually with strict experimental protocol with limited innovation.
Cannot be done very well in most university settings. Requiring good organization, discipline and infrastructure.
There are at least two ways of product development Backward engineering Evolutionary approach
Creating diversified candidates Selection
Figure 2 Positioning pharmacogenomics and pharmacogenetics in the drug discovery and development process.
Diagnostics Vaccines Therapeutics Health
promotives Medical/
Health Services
Biomedical/genomicsKnowledge
Genomics/High-throughput
Technology
Clinical/EpidemiologyKnowledge
Biomedical Informatics/ Robotics/
Nanotechnology
DevelopmentInfrastructure
ProductDevelopment
Suprastructure:Policy commitment/
Government Resource Allocation
International Infrastructure
International Input
HRD
Managing Product Developments
Product development needs Development team specialized at each step.
Foreign teams can be used in some steps. Infrastructures usually go with the development
teams. Product development manager/management team Sponsor/financers
Financing Product Development Product development are costly. Financer
is the game controller. Financer can be Private companies Government Institutes Philanthropic organizations International organizations
Product development are costly, one can do very few at a time. Which one would you do? How can you end up with the conclusion? Is it generally agreed?
Concept of Execution the goals : Focus on Wildly Important Goals :(WIGs)
“WIGs: The Goals must be achieved. Failure to achieve this goals renders any of the other achievements inconsequential.”
4 Disciplines of execution
Discipline 3Keep a Compelling Scoreboard
Discipline 1 Focus on the Wildly Important Goals (1-3 goals)
Discipline 2Action on the Lead measures
Discipline 4 Create a cadence of Accountability
The enemy of the best are the goods. There are many good products that can
be developed, but usually one can only do one.
The average cost of developing a drug is about $800,000,000 USD. The cost for clinical trials are at least a few to several dozen millions USD.
A poor clinical trial is not better than doing nothing, e.g., BCG vaccine.
When to stop: Exit Strategy Since we should devote all resources to
a few products, it is extremely important to stop the work immediately when appropriate or in the case of failure.
Tool: Stage-Gate Protocol. Development is divided into stages, each
which clear end-point (Gate Review Criteria).
Allowance to pass the gate is judged by “gate keeper”.
Stage and Gate
37
37
Discovery Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
R&D
idea
idea
Gate 1
Gate 2
Gate 3
Gate 4
Gate 5
Testing LaunchBusiness plan
Feasibility
study
R&D FunnelStage gate
Technology Readiness Levels Provide a Common Language
http://trl.sandia.gov
TRL1 Basic Research “Experimental data revealing useful information about the basic principles observed”
TRL2 Applied Research “Model that explains the underlying science and how it could be applied to solve a particular application’s problem”
TRL3 Research Result “Experimental or analytical demonstration that shows that the key elements of an approach are likely to be feasible”
TRL4 Research Demonstration (Lab demo)“Experiment in a Laboratory”
TRL5 Research Prototype (Demo Unit) “Looks like a Product, Hand-built by PhDs, Breaks a Lot”
TRL6 Engineering Prototype (Alpha Unit)“Research Prototype that is Rugged and Repeatable”
TRL7 Flight / Field Prototype (Beta Unit) “Engineering Prototype that is Reliable and Manufacturable”
TRL8 WR / Hi-Rel (Production Unit)“Field Prototype that has cost “wrung out” (if applicable), and has completed qualification”
TRL9 Proven Product “Product that has been used successfully in a system before and is being adapted for use in a similar application.
Funding and Outputs of Product Development
Product development succeeds only with adequate Infrastructures (animal testing lab, pilot production,
quality assurance/regulation, etc.) Personnel (engineers, chemists, etc.) Funding (at least opportunity to get funding all the
ways of development) Material supplies (esp. the ones need to be imported
or without commercial suppliers) Usually need a full time manager.
Outputs are usually visible only when development finishes. But can be visible during development by Stage-Gate management processes (Meeting or not meeting Gate Review Criteria).
Take home message. Proposal writing is an integral part of
R&D process. Most Proposal fails to get funded Do not be discouraged.