National Institute of Standards and Technology U.S. Department of Commerce

Technology Program Evaluation:Methodologies from the

Advanced Technology Program

Gems: Success stories on a page

Prasad GupteTechnology Innovation Program

National Institute of Standards and Technology

Collaborative Expedition WorkshopNational Science Foundation

March 18, 2008

National Institute of Standards and Technology U.S. Department of Commerce

What is a “gem”?

A one-page brief that describes the realized economic impact of funded ATP project.

A hybrid document Elements of economics, technology (lay-level),

and public relations Qualitative and quantitative

High-level yet rigorous Measures the project to ATP’s mission!

National Institute of Standards and Technology U.S. Department of Commerce

Example of a gem

National Institute of Standards and Technology U.S. Department of Commerce

How gems are used

To provide ATP with a means of communicating the impact of funded projects to the public For ATP headquarters

• To answer congressional requests For internal staff

• To use as examples at conferences, trade shows

• Feedback to project selection and management

– What are R&D best practices?– Trend analysis

National Institute of Standards and Technology U.S. Department of Commerce

Elements of a “gem”

It answers these three key questions:

What was the technical risk?

What did the project accomplish?

What were the broad economic benefits?

Note the linkage to funding criteria!

National Institute of Standards and Technology U.S. Department of Commerce

What was the technical risk?

What was the problem?

Why was it hard?

Why did it need government support?

National Institute of Standards and Technology U.S. Department of Commerce

Example of Technical Risk Throughout the early 1990’s GE’s central R&D lab, GE Global Research,

began the basic research for the use of amorphous silicon detectors in digital medical imaging with funding from a number of government agencies including DARPA, U.S. Army, U.S. Navy, and Department of Health and Human Services. If the cost of manufacturing amorphous silicon could be reduced enough, it would enable detectors with a wide range of applications, not just from medical imaging but also in industrial uses such as non-destructive testing. While GE scientists conceived of ways to bring the cost of manufacturing down, according to GE Vice President Lewis Edelheit:

“Getting internal support for the low cost manufacturing proposal was hard. At the time it was difficult for some executives to see the advantages of taking digital imaging beyond cardiac applications. Also the combining of process steps in the fabrication of amorphous silicon integrated circuits was considered to be a high risk proposition.”

Thus, in 1995, GE, with its manufacturing partner Perkin Elmer, applied for and received an ATP award due to the high level of risk, innovation, and potential for national benefits that the project proposed. According to Edelheit:

"Given the technical and other risks, had the ATP turned down the proposal, the promising, low-cost manufacturing process initiative would have been shelved."

National Institute of Standards and Technology U.S. Department of Commerce

What did the project accomplish?

Technical achievements Knowledge creation Bibliometric output

• Patents, publications, presentations

Capital attraction or further investment/continuation of project Linking back to ATP’s mission

National Institute of Standards and Technology U.S. Department of Commerce

Example of Project Accomplishments

In the ATP-funded project, GE successfully completed most of the ambitious technical goals. Major accomplishments include: Reducing the total number of process steps from 300 to 200 Reducing the total number of mask steps, the critical

manufacturing challenge, from 11 steps to 7 Reducing the total cost of the process by 25% Filing for and receiving three patents

However, when the ATP project was finished in 2000, there was still a significant amount of research to do before the technology could be ready for commercial application. Two years after project completion, the partners continued investing millions of dollars into the technology with no additional government funds.

In addition, prior to the ATP project, the low cost manufacturing technology was held by GE’s corporate lab. Through the ATP project, GE was able to transfer the technology to Perkin Elmer. Instead of being the manufacturer, GE made the strategic decision to be an end user of the technology, and allowed Perkin Elmer to focus on the manufacturing, where it could then sell the detectors to a wide variety of other users.

National Institute of Standards and Technology U.S. Department of Commerce

What were the broad economic benefits? Describe private benefits

Increased sales, decreased costs, better product/process performance

Describe public benefits (spillovers) How other entities (firms, industries,

customers, consumers, public) benefited Quantify as much as possible

National Institute of Standards and Technology U.S. Department of Commerce

Example of broad economic benefitsIn January of 2004, nearly ten years after the companies began working together on the ATP project, Perkin Elmer signed a $250 million deal to supply GE with amorphous silicon flat panel detectors, where: GE Medical Systems will use these in a variety of medical cost-sensitive medical applications GE Aircraft Engines will use the detectors for non-destructive testing

The medical applications are focused on digital mammography and digital radiography. The major benefits of these uses include: Superior breast cancer detection through lower false positives and fewer unnecessary biopsies Reduced radiation exposure Reduced medical costs through enhanced clinical productivity

The potential economic impact of the digital mammography applications alone is large—a rigorous cost-benefit study 1 estimated economic impacts between $219 million and $339 million, and a benefit-to-cost range of 125:1 to 193:1.

The enabling technology that ATP has funded has created even broader benefits, as Perkin Elmer is pursuing sales of the amorphous silicon detectors to users who will apply this toward: Industrial non-destructive testing (NDT) PCB inspection Pipeline inspection Bone densitometry and veterinary imaging Airport and customs cargo inspection for homeland security

National Institute of Standards and Technology U.S. Department of Commerce

Pre-process for creating gem1. Identify candidates

Review survey data Periodic meetings with project managers Condensed versions of other ATP reports (cost-benefit

studies, status reports)

2. Initial research Web research, project files

3. Prioritize based on “Readiness” – are benefits actual or potential? Technology/geographic area (internal customer need)

4. Now start writing the prioritized gems! Keep track of potential gems

National Institute of Standards and Technology U.S. Department of Commerce

Process for creating gem

1. Write draft based on:• Review of project files, web research

2. Internal review• Project manager, Group leaders, Division

chiefs3. Company review4. ATP Deputy director review5. Put on web:

http://www.atp.nist.gov/gems/listgems.htm

National Institute of Standards and Technology U.S. Department of Commerce

Questions?

How does this apply to your organization?