Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Millis_Stanford_GCEP.ppt 1
Managing High-Risk/High-Gain Research
Marc G. MillisNASA Glenn Research Center
At Lewis FieldCleveland OH 44135
Prepared for the
2005 Design for Breakthrough Research in an Academic SettingStanford University
2005, November, 11-12
Breakthrough
Millis_Stanford_GCEP.ppt 2
Managing Breakthrough Research
Experience Base
• “Vision 21” Lewis Research Center (1988-1994)– Initiated by mid level managers to sustain organization’s technological preeminence
– Provide an environment for the open exchange of ideas and a process to explore ideas that are too risky to warrant formal sponsorship
– “Free Idea Forums” - “PINS” - “Rules of Engagement”
• NASA Breakthrough Propulsion Physics Project (1996-2002)– Credibly addressed the incredible goal of interstellar travel
– Examining emerging science to circumvent limited technology1. Propulsion without propellant or beamed momentum
2. Achieve ultimate transit speed (faster-than-light?)
3. Onboard energy production
• Physics Entrepreneurship Masters Program (2003-present)
– Case Western Reserve University, Cleveland OH, http://pep.phys.cwru.edu/– Master’s Thesis: “Interstellar Flight Foundation”
Millis_Stanford_GCEP.ppt 3
BACKGROUND
NASA Breakthrough Propulsion Physics Project
GOAL: Exceed the fundamental limits of existing propulsion by further advancing physics to discover the breakthroughs that could revolutionize spaceflight and enable interstellar voyages.
ProgrammaticConduct visionary
research in a credible manner.
TechnicalTarget the greatestchallenges of deep-
spaceflight.
Millis_Stanford_GCEP.ppt 4
NASA BPP Project Accomplishments(Total of $1.6M spread over 1996-2002)
• Assessed 8 research approaches– 2 suitable for sequels– 3 unresolved– 3 dead ends
• Produced 14 journal publications
• Produced award-winning public education website
Warp Drive, When?http://www.nasa.gov/centers/glenn/research/warp/warp.html
Millis_Stanford_GCEP.ppt 5
NASA Breakthrough Propulsion Physics Project
Key Operating Philosophy
• Reliable – Define success as gaining reliable knowledge rather than claiming breakthroughs (puts emphasis where needed).
• Immediate – Focus on immediate make-or-break issues, unknowns, or curious effects (just enough for “go/no-go”), not the whole thing.
• Iterated – Gain knowledge via cycles of short-term, incremental tasks.
• Diversified – Support multiple, divergent research (not just hot topics).
• Measured – Track applicability and progress with the cycles.
• Impartial – Research selected via competitive peer assessments, where reviewers judge reliability of results, not feasibility of concept.
• Empirical – Emphasis on experiments over pure theory or studies.
• Published – Results published, both pro and con.
Millis_Stanford_GCEP.ppt 6
Overall Findings on Breakthrough PropulsionMillis (2004) Prospects for Breakthrough Propulsion From Physics, NASA/TM-2004-213082
Figure 3Hooper's Self-Cancelling Flat Coil
Point where wire is folded back onto itself
Null Findings Unresolved Next Steps
Millis_Stanford_GCEP.ppt 7
Managing Breakthrough Research
Lesson 1: Shift Away From Diminishing Returns(Foster, Innovation, The Attacker’s Advantage, 1986)
To exceed the limits of prior technology (or markets),seek entirely different technology (or markets).
Per
form
ance
Investment
Bottom line: Must venture beyond the familiarBottom line: Must venture beyond the familiar
Millis_Stanford_GCEP.ppt 8
Per
form
ance
Investment
“Masters”
“Pioneers”The “Breakthrough”
Managing Breakthrough ResearchA Closer Look at Your Place in the Curve
Millis_Stanford_GCEP.ppt 9
Pioneers• Expose ignorance - knowledge gaps• Create new knowledge• Can NOT quantify comparisons• Intuitive progress• Highest-Gain benefits sought
Masters• Maintain the knowledge base• Refine established knowledge• Quantify comparisons, “trades”• Procedural progress• Minimum Risk sought
Institutions & Management• Must sustain preeminence, reputation• Difficulty comprehending research details• Must assess risk -vs- benefits• Must stay within budgets• Must produce progress
Managing Breakthrough Research
Contrasting Motivations and Skills
Middle Minority“Might work, might not…
Let’s check.”
Qua
ntity
Pioneers | Masters
Pedantic Prudes-- The Establishment --
“That’s impossible”
“You’re Crazy”
“You just don’t understand”
Pathological Pundits-- The Lunatic Fringe --
“My theory is great!”“I can’t be wrong!”
“If you disagree, you are part of the suppression conspiracy”
(Not the show from the 90’s, but not very different from it either)
More CredibleMore Visionary
Millis_Stanford_GCEP.ppt 11
Managing Breakthrough Research
CHALLENGES• Breakthrough-Seeking Research
– High-Risk means likely to fail (easy stuff’s been done already)
– High-Gain implies reaching beyond the familiar, i.e. CRAZY IDEAS!• Unpredictable (venturing beyond the known)• Cannot quantify benefits rigorously• Evokes hype
• Supporting Institutions & Management– Difficult to comprehend and assess the unfamiliar– Must demonstrate progress in sync with funding cycles– Dealing with those edgy researchers– Dealing with success (typically disruptive)
• Researchers– Competing against nearer-term, more certain technology refinements– Limited sources– Clearly communicating the value of your work– Predicting when you’ll have a breakthrough
Millis_Stanford_GCEP.ppt 12
Managing Breakthrough Research
CHALLENGES
Long-Term Goals Near-Term Progress
Visionary Credible
Divergent Options Need to Focus
Sufficient Investment Affordable
Millis_Stanford_GCEP.ppt 13
Managing Breakthrough Research
Examining Methods for Breakthrough Research
• Defining the Problem• What are the “Important Problems”?• Horizon Mission Method• Special challenges of hunting breakthroughs
• Collecting Data• S-Curve lesson• On revolutions
• Forming Hypotheses• Human creativity patterns• Science fiction inspirations• Brainstorming• Critical filtering
• Testing Hypotheses, just one small step at a time• Affordable• Non-threatening
• Share Findings and Try Again• Publish dead ends• Clearing house of findings
Millis_Stanford_GCEP.ppt 14
Managing Breakthrough Research
Defining THE Problem
• Grand ChallengesLong-range, seemingly impossible goals as catalysts for sustained innovations. These are the goals whose solutions will make a huge difference
• Important ProblemsThose aspects of the Grand Challenges that can be attacked today, that will distinguish the significant research from the mundane
Millis_Stanford_GCEP.ppt 15
Managing Breakthrough Research
Distinguishing Between Good & Great researchers(Richard Hamming, “You and Your Research” Lecture at Morris Research & Engineering Center (May 7 1986)
• Great researchers tackle the “Important Problems” not the safe problems• Initially develops independent thoughts• Courageous, focused, driven, & steady• Converts what is difficult into something easier• Tolerates ambiguity:
– believe enough in self to proceed,– but doubt self enough to see flaws
• Learns things beyond their own work: “Knowledge is like compound interest”• Sells themselves:
– Writes well– Technical presentations– Executive level presentations
• Honest with self - Overcoming personality flaws (convert liabilities into assets)
Millis_Stanford_GCEP.ppt 16
1. Pick Impossible Goals, with well-defined boundaries– Prevents mere extrapolations– Forces “out of box” thinking
2. Assume it can be done– Imagine you are in the future; imagine solutions– Use inspirational imagery (including science fiction)
3. Look back from the future; identify limiting assumptions4. Identify the genuine critical challenges5. Identify knowledge gaps
– What questions still remain unanswered?– How do we seek those answers?
Managing Breakthrough Research
“Horizon Mission Method” To Define Problems(Anderson, "Leaps of the Imagination: Interstellar Flight & the Horizon Mission Methodology," JBIS, 49:15-20, 1996)
Millis_Stanford_GCEP.ppt 17
“Space travel is utter bilge.”- Dr. Richard van der Riet Wooley (one year before Sputnik 1)
“The secrets of flight will not be mastered within our lifetime, not within a thousand years.”
- Wilbur Wright (two years before Kittyhawk)
“Heavier than air flying machines are impossible,X-rays are a hoax”
- William Thomson (Lord Kelvin)
But then, don’t forget that there were also crazy ideas that were just plain crazy.
Managing Breakthrough Research
Reflexive Response to Potential Breakthroughs
Millis_Stanford_GCEP.ppt 18
Challenges
High payoffs evoke High emotions
+Edge of knowledge evokes
Controversial ideas
=Hot debates with premature conclusions
And eventuallyBreakdown of constructive discourse
Challenges
High payoffs evoke High emotions
+Edge of knowledge evokes
Controversial ideas
=Hot debates with premature conclusions
And eventuallyBreakdown of constructive discourse
Mitigation
• Dissect big gains into small issues
• Aim for discriminating tests
• Consider results objectively
• Constrain conclusions within irrefutable data
• Openly publish and discuss
Mitigation
• Dissect big gains into small issues
• Aim for discriminating tests
• Consider results objectively
• Constrain conclusions within irrefutable data
• Openly publish and discuss
Managing Breakthrough Research
Dealing With Revolutionary, Provocative Research
Millis_Stanford_GCEP.ppt 19
• Business Driven: Revolutions Come From OutsideHisrich (2003) ENTP 429 Class Note– Technological Myopia (Incumbent focus on their own, old, technology )– Focus on Existing Customers (diminishing pool or shifting to different customers)– Capital Allocation Processes
• Not interested in projects that might not work• Insist on hard data where it is not possible
Managing Breakthrough Research
Lessons for Seeking Revolutions
• Concept Driven: “Paradigm Shifts”Kuhn (1962) The Structure of Scientific Revolutions– Copernicus– Newton– Darwin– Einstein– Quantum Mechanics
• Tool DrivenDyson (1997) Imagined Worlds– Telescopes… Galileo– X-Ray Diffraction… Crick/Watson– Hubble Telescope…– High Temperature Superconductors…– Micro, Nano, etc…– Relativistic Heavy Ion Collider…– Computation (Includes Internet collaboration)…
Millis_Stanford_GCEP.ppt 20
Consider this as a model for planning discussions, workshops
1. Define the problem
2. Deluge with information
3. Gestation period... “Ah-Ha!”
4. Brainstorm multiple paths (Diverge)
5. Gestation period... “Ah-Ha!”
6. Critically filter down to workable set (Converge)
7. Try it out and iterate
Managing Breakthrough Research
Lessons About Creative Process(multiple sources)
Millis_Stanford_GCEP.ppt 21
• Sci-Fi has been found to be a common, significant inspiration to pioneers.• Sci-Fi has not been found to be an accurate predictor of the future.
Managing Breakthrough Research
Science Fiction Inspirations for BrainstormingEmme, E., ed, (1982) Science Fiction & Space Futures Past & Present, Am. Astronautical Soc. History Series, V. 5.
InspirationJ. Verne
VisionVon Braun
RealityApollo
Millis_Stanford_GCEP.ppt 22
Convert Objections into Objectives
Be Visionary!
• Assume it can be done• Imagine the possibilities• Draw on inspirations• Pattern past successes
Be Visionary!
• Assume it can be done• Imagine the possibilities• Draw on inspirations• Pattern past successes
Be Credible!
• Be constructively skeptical• Identify unsolved challenges• Build on known science• Aim toward testable concepts• Use scientific method
Be Credible!
• Be constructively skeptical• Identify unsolved challenges• Build on known science• Aim toward testable concepts• Use scientific method
Managing Breakthrough Research
Effective Brainstorming
Millis_Stanford_GCEP.ppt 23
Rules of Engagement
1. No scoffing at presenter. Remember that every breakthrough started as an incomplete, crazy idea.
2. Use “PINS” sequence:1st Identify what is Positive.2nd Identify what is Interesting.3rd Identify what is Negative, AND4th Suggest ways to overcome flaws.
Give suggestion in writing later.
3. Feel free to add ideas.
Responding to New Ideas Presenting New Idea
1. No scoffing at critics. Remember skepticism is a normal, necessary part of shaping ideas into products.
2. Seek constructive suggestions to further advance the idea. Shape objections into objectives.
3. Be open to adapt ideas to take advantage of new information.
HUMOR is Constructive!
Managing Breakthrough Research
Effective Brainstorming
Millis_Stanford_GCEP.ppt 24
Managing Breakthrough ResearchFinding the Middle Ground Between Credibility and Visions
1. Zero Propellant
2. Faster-Than-Light
3. Onboard Energy
Vacuum Battery ?
GrandChallenges
Concepts& Devices
Curious Effects, Unknowns, & Issues
Space Drives
Warp Drives & Wormholes
Quantum Fluctuations
General Relativity
Conservation Laws
Quantum Mechanics
Cosmology
Anomalous Rotation Rates(“Dark Matter”)
Anomalous Red-shifts(“Dark Energy”)
mpvp = mrvr
FoundationalPhysics
Compare Emerging Physics to Visions
Millis_Stanford_GCEP.ppt 25
General Physics�SRL-1
Curious Effects, Unknowns & Issues�SRL-2
Concepts & Devices�SRL-3
BPP�GOALS
Mapping Physics to BPP Goals
COLOR KEY:
Marc G. Millis Draft Ver. 2004-10-08_BPP_Map
= Dismissed
SRL-n.0 (Noticed) SRL-n.1 (Problem Defn) SRL-n.2 (Collect Data) SRL-n.3 (Hypothesis) SRL-n.4 (Tested) = TRL-1 (Op Principles) TRL-2 (Concept Design) TRL-3 (Design Tested)Place-Holder
= Caution = Continue Study = Info Included Blk Frame = Title or Empirical+Theoretical Gry Frame = Empirical Wht Frame = Theoretical= TBD
Goal-1�MASS
Goal-2�SPEED
Goal-3�ENERGY
Thrusting
Short Cuts
a priori�FTL
Energy in Space
Alt Nuc
Alt Chem
Sail Drives
NET TRANSFER METHOD
Field Drives
Alt Emission�e.g. photon rocket
Brute Fast
Metric Engineering
Alt Dimensions
Tachyon Drive
Transport via�QM NonLocality &
Entanglement
KE Coupling
ZPE
Alt Antimatter
Alt Fusion
Casimir Echo�(LaPointe)
DISCIPLINES
Arrow of Time
ANOMALIES
Hi-Energy�Cosmic Rays
Gamma Ray Bursts
Arbitrary Parameters in Std. Mdl.
Matter/Antimatter Asymmetries
Anomalous Galactic Rotation Rates�
(aka Dark Matter)
Anisotropy of EM Propagation
Anomalous Red Shifts�(aka Dark Energy)
Solar Neutrino Flux
Sonolum Spectrum
Saxl Charged Torque Pendulum
Josephson Junction Sinusoid / Yr
Pendulum @ Eclipse
Cosmological Constant
Conservation of Momentum
Conservation of Energy
MOND?
Maxwell Eq
2nd Law Thermo
Zero Point Energy
Standard Model
Lorentz Invariance
Brans-Dicke
Yilmaz
Electromagnetism
Quantum Mechanics
Particle Physics
Special Relativity
Strings & Branes
Thermodynamics�& Energy
Differential
Diode
Induction
Diametric
Pitch
Bias
Disjunction
Alt-Wave
Grav-Wave
Casimir Battery�(Forward)
Sono-Fusion
f-sec Laser Fusion
LENRs
(Black Light)
Deep Dirac
Space Warps
Wormholes�(Visser)
Hyperspace
Tachyon Space
Translocation
Inertia Cycle�(Woodward)
Rectified Slepian�(Corum)
PHENOMENA USED
Extreme Energy
Negative Energy
Hadron Collider�Mini-Blk-holes
Causality
ISSUES
MASS OPTIONS Inertial Mass
Charged MassDark Matter?Virtual Pairs?
Higgs particles?
EMCBR
Higgs?GR Grav Waves?
MOMENTUM-ENERGY WAVES
SPACE PROPERTIES�-or- SPACETIME STRUCTURESInertial Frame�(Mach Space)
EtherÓ?Dark Matter?
Higgs Fields?QM Foam?
ZPEs (EM, G, etc)?
Grav Potential
Space Energy?
Alt Dimension?
Vibrating ZPE Mirror�(Maclay/Forward)
OF INTEREST
Stellar Aberration
Focualt Pendulum�& Coriolis Effect
Hypercharge�5th Force
Mach Principle
Tunneling Speed?
Schlicher Thruster
Free Energy claims
Anomalous TrajectoriesRetarded Potentials
Vacuum Optical Analogy
Equivilance Principle
Gravito-Magnetism
Cosmic Background Radiation
Local Stnd of Rest
Anomalous Forces Near RF-Pumped Superconductors
Quantum Nonlocality
Nullor Space Drive (Fwd)
GR g Dipole (Fwd)
Biefeld-Brown, et. al.
UNKNOWNS
Scalar G Waves?
ZPEs (EM, G, etc)?
VARIANTS / CLAIMS
CONTROL METHOD
Extreme Boundary (superconductors)
Motion of Masses� - Rotation� - Off Cycle
EM Modulations
Extreme V or A
Superconductor Force Impulse Beam
Net External Force
Reaction Mass?
Energy Transfer Mechanism
Transient Inertia?�1st & 2nd Term�
(Woodward)
Neutrino - Mass2?
Speed of Spacetime?
Eq. Reaction Mass in Space ?
Abraham-Minkowski Controversy
Tachyons?
Detectability of�LSR Motion ?
Violations of Local Position Invariance
?
Reliability of Gravito-Mag data
w/ Superconductors
Violations of Weak Equivilance Principle ?
Gravity & Inertia of Antimatter
EM-GravWave transduction via Superconductors
Space Drives�(Millis)
Boundary Mod
ELF EMG (deAquino)
Mech. Osc. Drives
Antigravity
Speed of Gravity?
Anomaly @ RHIC�Quark-Gluon Plasma
MEG (Bearden)
Kinematics
Imaginary time (Minami)
Pondermotive Force�(Smith)
Minkowski EM mv�(Brito)
EM-G Alternatives�(Cravens)
EM-Tensors�(Kron)
EM-G Alt, SU3 ?�(Barret)
Grav Gauge Sym �(Tucker)
Novel Force Engine (Allen)Detect e, m = f(G)
Torsion, EM- ST�(Ringermacher)
ZPE - Inertia ?�(Haische)
Polarizable Vac ?�(Puthoff)
EM ZPE Tangible?
Heim 8D TheoryGeneralization of
Cerenkov radiation to vacuum, et.al.
Strong Field limit to Euclidean Analogies
Inertia of Bose-Einstein Cond.
Wormhole Induction�(Davis)
Warp Drive�(Alcubierre)
Tubes�(Krasnikov)
FTL Self-Ref QM (Goff)
General Relativity� - Geometric� - Euclidean� - Quantum Grav
Astronomy &�Cosmology
Brane Propulsion (Minami, Loup)
Fusor
Brady Asym Fluid
Marchese tests
Find the middle groundwhere goals and foundations
Intersect.
Then answer theImportant Questions.
Foundationsof
Knowledge
DesiredHigh-Gain
Achievements
Managing Breakthrough Research
Converting This Into Research Tasks
Millis_Stanford_GCEP.ppt 26
Typical “Instinct” Approach• Lead person picks pet projects or the latest hot
topics.• Funds sought for selected approaches only.• Other approaches not comparatively assessed
in a rigorous manner.
Typical Results• Success is defined in terms of whether the
approach worked.• Negative results not published.• In the event of a null result, support ebbs.• Window closes on all other approaches for
addressing these same challenges.
Typical “Instinct” Approach• Lead person picks pet projects or the latest hot
topics.• Funds sought for selected approaches only.• Other approaches not comparatively assessed
in a rigorous manner.
Typical Results• Success is defined in terms of whether the
approach worked.• Negative results not published.• In the event of a null result, support ebbs.• Window closes on all other approaches for
addressing these same challenges.
Contrasting Typical -vs- Strategic Approach
Better Strategic Approach• Lead person acts as an impartial broker.• Funding set to what the challenge is worth.• Various approaches solicited and
comparatively evaluated.• Selection criteria concurred with customers and
practitioners.• Scope of each task set to the minimal effort
needed to resolve an immediate “go / no-go”question.
Results Sought• Success defined as gaining reliable knowledge
to guide next steps.• Results, pro or con, published to set foundation
for future decisions.• Opportunity open for sequels to the positive
results, and to redirections around null results.
Better Strategic Approach• Lead person acts as an impartial broker.• Funding set to what the challenge is worth.• Various approaches solicited and
comparatively evaluated.• Selection criteria concurred with customers and
practitioners.• Scope of each task set to the minimal effort
needed to resolve an immediate “go / no-go”question.
Results Sought• Success defined as gaining reliable knowledge
to guide next steps.• Results, pro or con, published to set foundation
for future decisions.• Opportunity open for sequels to the positive
results, and to redirections around null results.
Managing Breakthrough Research
Turning This Into A Project
Millis_Stanford_GCEP.ppt 27
• Goals– Promise to deliver measurable progress– Do not promise to deliver breakthroughs
• Broad participation– Acquire the pioneering skill mix (Combine Masters and Pioneers)– Get the researcher pool– Get the advocacy pool
• Digestible pieces– Devise and propose short-term, low-cost, and incremental tasks– Non-threatening increments of progress– Provide means to monitor progress and judge value regularly
• Resonate Funding Level– Large enough to make progress– Small enough to avoid being a target
Managing Breakthrough Research
Suggestions for Making Research Proposals
Millis_Stanford_GCEP.ppt 28
Commercetechnology applied, sold
Technologyscience applied
SciencePhenomena understood
SpeculationA knowledgeable guess
ConjectureA wishful vision
“Too risky.” “We don’t need it.”
“It’s not practical.” “It’ll never work.”
“That doesn’t fit with existing theories.”
“You’re Crazy.” “You don’t know what you’re talking about.”What do we need to learn?
How dowe make a
useful device?
How can we use this new
science?
How do we test this?
Managing Breakthrough Research
Stepping Over Classical Barriers
Millis_Stanford_GCEP.ppt 29
TRL 7 Engineering model tested in space.
TRL 6 Prototype / engineering model tested in relevant environment.
TRL 5 Component / breadboard tested in relevant environment.
TRL 4 Critical function / characteristic demonstration.
TRL 3 Conceptual design tested.
TRL 2 Conceptual design formulated.
TRL 1 Basic principles observed and reported.Conjecture
Speculation
Science
Technology
Routine
Managing Breakthrough ResearchMeasuring Progress: “Technology Readiness Levels”
Hord, M., CRC Handbook of Space Technology: Status & Projections, CRC Press, FL, © 1985
Millis_Stanford_GCEP.ppt 30
Scientific Method (as readiness levels):SM-4 Hypothesis empirically confirmed / dismissedSM-3 Hypothesis proposedSM-2 Data collectedSM-1 Problem formulated (identify relevant knowledge gaps)SM-Ø Pre-science:
• Anomalous effect noted, or• Correlation between goal & knowledge recognized.
Conjecture
Speculation
Science
Technology
Routine
Relevance of science topic (as readiness levels):R-3 Directly relevant to a technologically desired effectR-2 Critical make-break issue underlying the desired effectR-1 Underlying general physics
First, specify the degree of relevance of the emerging science, and then specify the progress achieved within this relevance using the Scientific Method levels.
R-3/SM-4 = TRL-1
Managing Breakthrough Research
Measuring Applied Science ProgressMillis (2004) BPP Project Management Methods, NASA/TM-2004-213406
Millis_Stanford_GCEP.ppt 31
• Devise criteria using team of practitioners and customers– Practitioners know what can be done
– Customers know what is needed
• Grading proposals– Peer (practitioner) reviews constrained to use criteria, plus subjective score
– Grade relevance and reliability, not the predicted technical outcome.
– Have proposal summary sheet matched to criteria
– Multiple reviewers per proposal
• Selecting awards– Compile scores to rank and flag disparate reviews (standard deviations)
– Customer (not practitioner) selects
– Select diversified research portfolio, not just highest-ranking set
Managing Breakthrough Research
Suggestions for Research Selections
Millis_Stanford_GCEP.ppt 32
• Multiplicative criteria (failed criteria fails whole proposal)
• Scholastic gradations (A through F) where possible
• Independent, minimum criteria concurred by team
TotalScore =ANA
⎛
⎝ ⎜
⎞
⎠ ⎟
aBNB
⎛
⎝ ⎜
⎞
⎠ ⎟
bC +Cmin
NC
⎛
⎝ ⎜
⎞
⎠ ⎟
c
Where:– A, B, C represent criteria scores– a, b, c are weighting factors
(where 1 is the maximum value, and lower priorities are fractions of 1)– NA, NB, NC are normalizing functions– Cmin is a preset value to prevent the parenthetical term from equaling
zero, thereby making criteria C non-mandatory.
Managing Breakthrough Research
Suggested Proposal Grading Method
Millis_Stanford_GCEP.ppt 33
Technical Relevance1: Gain – Magnitude of performance improvement, assuming the technology ultimately
reaches fruition.
2: Empiricism – Tangible effects or just theory?
3: Readiness – The present maturity of the topic/concept under study.
4: Progress – Magnitude of progress to be achieved, as measured by the difference in the readiness now, and the anticipated readiness upon completion of the task.
Credibility5: Foundations – Based on credible references.
6: Contrasts – Compared to current credible competing work.
7: Tests – Leading toward a discriminating test.
8: Results – Probability that the task will result in a reliable foundation for future decisions.
Resources9: Triage – Will it be done anyway or is it unique to this Project?
10: Cost – Funding required (reciprocal scoring factor).
11: Time – Time required to complete task (reciprocal scoring factor).
Managing Breakthrough Research
Sample Criteria from NASA BPP Project
Managing Breakthrough Research
Clearing House Research Summary Form ?A: Submitted by: Submission ID#: Reviewer ID#:
B: Title of submission (for published results, list full reference citation here) C: Submission Type: SCORESCalculated Composite:
Published Results(blocks R,S optional)) F D C B A 0-4
D: Central issue, unknown, or observation under study (paragraph) Subjective Score:Proposal
(Skip blocks E,F) F D C B A 0-4
Automated Pre-Score:Suggested Inquiry
(Fill in at least A-D,G,K,N) F D C B A 0-4
E: Findings [Only required when citing published results] (paragraph & check box F) F: Sequels Expected? SEQUELS JUSTIFIED?
RELEVANCEG: Relevance to Project (paragraph & check-boxes H) H: Ultimate Improvement 1: Gains toward Goals?
Mass M
0 1 2 3 4 0 1 2 3 4 0-4
Speed S
0 1 2 3 4 0 1 2 3 4 0-4
Energy E
0 1 2 3 4 0 1 2 3 4 0-4
I: Increment of work proposed or reported (paragraph & check boxes J,K,L,P,S) J: Type of Research 2: Empiricism
Study Theory Expmt Ex&Th S T E T&E 1-4
KL: Readiness & Progress 3-4: Readiness & Progress
�TRL2 �TRL2 �TRL2 �TRL2
Test Test
Hypoth Hypoth
Data Data
Defn Defn
PreSci PreSci
M: Contrasting or skeptical challenges to the proposed (or reported) approach (paragraph) Test Test
Hypoth Hypoth
Data Data
Defn Defn
PreSci PreSci
Test Test
Hypoth Hypoth
Data Data
Defn Defn
PreSci PreSci
Science Readiness Levels(4) Calculated Progress
N: Founding References (list) O: Representative Graphic (optional) P: Related Disciplines(Check all that apply) F D C B A 0-4
Basic Motion & EnergyElectromagnetismSpecial Relativity CREDIBILITYQuantum Mechanics 5: FoundationsQM: VacuumQM: Non-Locality F D C B A 0-4
QM: Gravity 6: ContrastsParticle Physics
Q: Contrasting or Skeptical References (list) Micro/Nano Physics F D C B A 0-4
Cryogenic Physics 7: TestabilitySuperconductorsCosmology/Astron F D C B A 0-4
General Relativity 8: ResultsGR: Metric EngrGR: GravitoMagnetic F D C B A 0-4
Strings/Branes
RESOURCESR: Prior publications to reflect proposed researchers' qualifications (list) S: Resources 9: Triage:
Triage: D C A 1-4
Likely other ? BPP unique 10: Cost
Cost: $ K F D C B A 0-4
T: Performing Organization U: Other Sponsors (optional)Time: 11: Time (yrs)
�3y 2y 1y <1y D C B A 1-4
Critica
l Is
sues
Gen
eral
Phys
Des
ired
Eff
ects
Critica
l Is
sues
Gen
eral
Phys
Des
ired
Eff
ects
Before AfterK: Before L: After
No YesMaybe
REVIEW COLUMN
Red Yel Grn
Findings Summary SheetsIn-House GRC Research (1993-1994 GRC Director’s Discretionary Funds)
Experimental Evaluation of Hooper’s Gravity-Electromagnetic Coupling ConceptMarc Millis & Gary Scott Williamson, NASA GRC.
Critical Issue:Are the claims of gravitational effects from self-canceling EM coils (as Patented) genuine?
BPP Relevance: Goal 1 (Mass)
Founding Reference:• Hooper, W. J., “All-Electric Motional Electric Field Generator,”
US Patent 3,610,971 (oct 1971).
Type of work:Experimental assessments.
Increment of Progress:
CONCLUSION: The claimed effect was not found in replication experiments within the sensitivity of the apparatus (40µg).
DETAILS: Millis & Williamson, Experimental Results of Hooper’s Gravity-Electromagnetic Coupling Concept, NASA TM-106963 (June 1995)
How Supported
Topic & PI
Relative Maturity(Color coded findings)
Graphic IconsRelevance
Starting Point
Concluding Reference
Key Finding(Color coded)
Test anomalous gravity claims with dual microstructure YBCO DiskG. Hathaway, et.al, Hathaway Consulting, Ontario, Canada
Independently Funded Research
Critical Issue:Independent test of unconfirmed anomalous gravitational effects with YBCO superconductors.
BPP Relevance: Goal 1 (Mass)
Example Founding References:Podkletnov E. and Nieminen R., Physica C 203 (1992) 441
Concluding Reference:Hathaway, Cleveland, & Bao, “Gravity modification experiment using a rotating superconducting disk and radio frequency fields,” Physica C, 385 (2003) pp. 488-500.
Type of work:Experimental tests.
Increment of Progress:
FINDING: “… no evidence of a gravity-like force…”with sensitivity “50 times better than… Podkletnov.”
Null Finding: Gravity Shield
Introducing the Interstellar Flight FoundationUsing the dream of reaching other worlds as a long-range goal and a catalyst for near-term progress, the Interstellar Flight Foundation supports incremental advancements in science, technology, and education. As a private nonprofit (501c3) corporation, supported mainly through philanthropic donations, the Foundation seeks out and directs support to the best practitioners who can make credible progress toward this incredible goal and educate the public throughout this journey of discovery.
TM
Interstellar Flight Foundation - Plans• Tracking and explaining relevant
advances (Centauri Dreams website)• Building network of practitioners• Awards for significant advancements• Educational outreach
– Web resources– Milestone Books– Inspirational Documentaries
• Sponsoring research through competitive selections
• Invitational Research InstituteTM
Millis_Stanford_GCEP.ppt 39
Managing Breakthrough Research
Misc. Lessons From Business Innovation Ventures• Don’t reorganize, start from scratch (Henderson, Clark, Architectural Innovation…1990)• Change orientation away from refinement to one of active search (Henderson, Clark, Architectural
Innovation…1990)• Firms that conduct in-house R&D are better able to use external information and collaborations (Cohen,
Levinthal, Absorptive Capacity…1990)• Expose broad range of employees to broad range of external information [travel for more than just for
presenting papers] (Cohen, Levinthal, Absorptive Capacity…1990)• Balance inward-looking and outward-looking research, else poorly functioning organization (Cohen,
Levinthal, Absorptive Capacity…1990)• Cyclical Innovation new business units, (total = 6; market strategists & engineers, whose ratio evolves
from definition [market] to development [engineering] to launch [market]) (Tabrizi, Walleigh, Defining Next-generation products…1997)
• Separate innovation from taking care of existing markets (Chesbrough, Designing Corporate Ventures…2000)
• Utilize venture capital structures, in-house (Chesbrough, Designing Corporate Ventures…2000)• Broad swath on markets, do not limit to existing base (Chesbrough, Designing Corporate Ventures…2000)• Look for disruptive technologies outside the usual places (Ken Zinda, 2004, CWRU lecture)• Lessons from the “Bike Shop” ...
– Focused on solving particular needs– 15 handpicked engineers & scientists described as creative, eccentric and bored.– Direct purchase credit card privileges ($250K/month!)– Machinists and world-class equipment available (not farmed out)– Scientist and engineers actually play with their contraptions– Low-tech feel to surroundings– Secretive unit (presumed for Defense sensitivities and Commercial competitiveness)
Millis_Stanford_GCEP.ppt 40
Managing Breakthrough Research
ABSTRACT
Based on the experiences of the NASA Breakthrough Propulsion Physics (BPP) Project, management techniques for addressing high-gain and high-risk research are presented. This includes the operating philosophies for casting long-range and uncertain prospects into more reliable, near-term steps. The methods for reviewing, prioritizing, and selecting research tasks are also included. Specifically, these methods employ the following key features: (a) invited research tasks are constrained to only address the immediate unknowns, curious effects or critical issues, (b) reliability of practitioners ranks higher than the implications of claimed technical benefits (c) reviewers judge credibility rather than feasibility, and (d) total scores are obtained by multiplying the criteria scores rather than by adding. Lessons learned are discussed. This presentation is a summary of the following recent report:
Marc G. Millis, "Breakthrough Propulsion Physics Project: Project Management Methods", NASA TM-2004-213406 (2004 Dec.)
http://gltrs.grc.nasa.gov/cgi-bin/GLTRS/browse.pl?2004/TM-2004-213406.html