TEACHING STUDENTS BASIC LAB SKILLS FOR A
REGULATED ENVIRONMENT
BIOMAN 2007Lisa Seidman
Madison Area Technical CollegeMadison, WI
WHY THE BASICS?
• Needs of students
• Needs of employers
MYTH 1
• Basics means simple, easy, obvious
• If this were true, far fewer problems in companies and in research labs
BASIC MEANS:
• Vital
• Essential
• Fundamental
• Primary
• Staple
• Must
MYTH 2
• Most of this does not apply in research labs
MYTH 3
• We all learn the basics in high school, or someone else’s class, or by osmosis
MYTH 4
• Basics are boring
BASICS
• What are basics?
• Different answers, but some common themes
HOW TO TEACH BASICS?
• Consciously
• Systematically – model 1: way teach children music– model 2: way grad students are taught
• Underlying principles
THIS WORKSHOP
• Teaching basics consciously
• Systematically
• Underlying principles
TOPICS FOR THIS WORKSHOP
• Quality
• Basic lab task – making a solution
• Metrology (unifying principles)
STORY OF FRANCES KELSEY
• Case study
KELSEY
• Purpose:– introduce GMP– introduce process of developing drug– most important: idea of quality– Bureaucrat – who understood quality
QUALITY: THE BIG (BUT BRIEF) PICTURE
WHAT IS BIOTECHNOLOGY?
The biotechnology industry transforms scientific knowledge into useful products
OVERVIEW
• Talk about product quality systems
–In broad way
–Apply ideas to the various work places we talked about
QUALITY SYSTEMS
• Broad systems of regulations, standards, or policies that ensure the quality of the final product
• GMP/GLP/GCP are examples of quality systems
WHAT IS PRODUCT QUALITY?
• What is a “good” product in biotechnology?
• That depends…
• Consider biotech:
–Research labs
–Testing labs
–Production facilities
QUALITY PRODUCT: RESEARCH LAB
• Research lab, knowledge is product:
–Knowledge of nature (basic research)
–Understanding of technology (applied research, R&D)
QUALITY SYSTEMS IN RESEARCH LABS
• Quality system in research
• Ensure meaningful data–has been around a long time
• It is called:
• “DOING GOOD SCIENCE”
–Less formalized than other quality systems
–No one book spells it out–No laws to obey–But it exists
INFORMAL SYSTEM
• Consequences of poor quality product not life-threatening so–Government seldom involved in
monitoring research quality
–Oversight not generally by outside inspectors or auditors
BUT THERE IS OVERSIGHT
• Oversight is by peers
–Grant review
–Publications
–Reputation
• Compare and contrast situation in research labs and other work places
PRODUCT QUALITY: TESTING LAB
• Testing lab:
–Information about samples
–Good product = result that can be relied on when making decisions
CONSEQUENCES
• A poor quality product can be life-threatening or have serious effects
QUALITY SYSTEMS IN TESTING LABS
• Include most of what we call “doing good science” plus
• Specific formal requirements– Personnel– Equipment – Training– Facilities– Documentation…
• You can find a book that spells it out for:
–Clinical labs
–Forensic labs
–Environmental labs…
ENFORCEMENT: TESTING LABS
• Since consequences of poor product can be life-threatening
–Is outside oversight
• FBI
• EPA
• Etc.
PRODUCT QUALITY: PRODUCTION FACILITY
–Make tangible items
–Quality product fulfills intended purpose
• Ex.: reagent grade salt vs road salt vs table salt
QUALITY SYSTEMS IN PRODUCTION FACILITIES
• Depends on nature of product
• Poor product may or may not have life-threatening consequences
SO, FOR EXAMPLE
• Products for research use, not generally regulated
• Agricultural products are regulated in one way
• Pharmaceutical products are regulated in another
VOLUNTARY STANDARDS
• Companies that are not regulated may choose to comply with a product quality system for business reasons
ISO 9000
• ISO 9000–Formal product quality system–Extensive–Exists in a series of books–Companies comply voluntarily to
improve the quality of products–…and to make more money
OVERSIGHT: ISO 9000
• Oversight by outside auditors, paid by company
BIOTECH AND MEDICAL PRODUCTS
• Many biotech companies that make money make medical/pharmaceutical products
• Consequences of poor product can be life-threatening
SO…
• These products are highly regulated by the government
• But, it wasn’t always this way…
• history…
• CFR, handout
HOW IS QUALITY BUILT INTO A PRODUCT?
• No single answer• Requires:
– Skilled personnel– Well-designed and maintained facility– Well-constructed processes– Proper raw materials– Documentation– Change control– Validation
ENFORCEMENT
• Compliance is enforced by government
–FDA
QUALITY IS BASIC
• Details may not be essential right now
• Idea of quality is essential
LET’S “GO TO THE LAB”VERY BASIC LAB TASKS
1. Write procedure to make 100 mL of a buffer solution that is:
100 mM Tris, pH 7.5 2% NaCl10 μg/mL of proteinase K
2. QC “your solution” by checking its conductivity
3. Check the pH of a Tris buffer solution
PROCEDURE
• For 100 mL of 100 mM Tris solution (FW 121.1) weigh out 1.211 g of Tris base. Dissolve in about 60 mL of water and adjust pH to 7.5.
• Add 2g of NaCl
• 10 μg/mL of proteinase K X 100 mL = 1000 μg = 1 mg. Weigh and add to Tris.
• Dissolve, BTV, check pH
VARIABILITY IN APPROACHES?
• Value of SOPs in ensuring consistency
• Value of communicating among various lab workers
• Documentation
WHAT DO STUDENTS NEED TO KNOW?
• Conceptual– Why they are making solution, context– How to interpret recipe– Basic calculations
• Instrumentation– How to maintain, use, calibrate balance– How to maintain, use, calibrate pH meter– How to measure volume– How to maintain, use, calibrate conductivity meter
• Quality control– How to ensure that solution is what it should be—– How to document work
TEACHING
• Concrete skills– calculations– using equipment– etc.
• These are activities in the lab manual to systematically build these skills
VARIABILITY
• Mike Fino
UNDERLYING PRINCIPLES
• Quality ideas (e.g. reducing variability and documentation, following directions—SOPs)
• Math calculations/ideas that repeat over and over again
• Safety practices
• Metrology principles
INTRODUCTION TO METROLOGY
Lisa SeidmanBioman 2007
DEFINITIONS
• Metrology is the study of measurements
• Measurements are quantitative observations; numerical descriptions
OVERVIEW
• Begin with general principles
• Next: weight, volume, pH, light transmittance (spectrophotometry)
WE WANT TO MAKE “GOOD” MEASUREMENTS
• Making measurements is woven throughout daily life in a lab.
• Often take measurements for granted, but measurements must be “good”.
• What is a “good” measurement?
EXAMPLE
• A man weighs himself in the morning on his bathroom scale, 172 pounds.
• Later, he weighs himself at the gym,173 pounds.
QUESTIONS
• How much does he really weigh?
• Do you trust one or other scale? Which one? Could both be wrong? Do you think he actually gained a pound?
• Are these “good measurements”?
NOT SURE
• We are not exactly certain of the man’s true weight because:– Maybe his weight really did change – always
sample issues– Maybe one or both scales are wrong – always
instrument issues
DO WE REALLY CARE?
• Do you care if he really gained a pound?
• How many think “give or take” a pound is OK?
ANOTHER EXAMPLE
• Suppose a premature baby is weighed. The weight is recorded as 5 pounds 3 ounces and the baby is sent home.
• Do we care if the scale is off by a pound?
“GOOD” MEASUREMENTS
• A “good” measurement is one that can be trusted when making decisions.
• We just made judgments about scales.
• We make this type of judgment routinely.
IN THE LAB
• Anyone who works in a lab makes judgments about whether measurements are “good enough” – – but often the judgments are made
subconsciously– differently by different people
• Want to make decisions– Conscious– Consistent
QUALITY SYSTEMS
• All laboratory quality systems are concerned with measurements
• All want “good” measurements
NEED
• Awareness of issues so can make “good” measurements.
• Language to discuss measurements.
• Tools to evaluate measurements.
METROLOGY VOCABULARY
• Very precise science with imprecise vocabulary– (word “precise” has several precise meanings
that are, without uncertainty, different)
• Words have multiple meanings, but specific meanings
VOCABULARY
• Units of measurement
• Standards• Calibration• Traceability• Tolerance
• Accuracy • Precision • Errors • Uncertainty
Instrumentation
Measurement itself
UNITS OF MEASUREMENT
• Units define measurements
• Example, gram is the unit for mass
• What is the mass of a gram? How do we know?
DEFINITIONS MADE BY AGREEMENT
• Definitions of units are made by international agreements, SI system– Example, kilogram prototype in France– K10 and K20 at NIST
EXTERNAL AUTHORITY
• Measurements are always made in accordance with external authority
• Early authority was Pharaoh’s arm length
• A standard is an external authority
• Also, standard is a physical embodiment of a unit
STANDARDS ARE:
• Physical objects, the properties of which are known with sufficient accuracy to be used to evaluate other items.
STANDARDS ARE AFFECTED BY THE ENVIRONMENT
• Units are unaffected by the environment, but standards are– Example, Pharaoh’s arm length might change– Example, a ruler is a physical embodiment of
centimeters • Can change with temperature
• But cm doesn’t change
STANDARDS ALSO ARE:
• In chemical and biological assays, substances or solutions used to establish the response of an instrument or assay method to an analyte
• See these in spectrophotometry labs
STANDARDS ALSO ARE:
• Documents established by consensus and approved by a recognized body that establish rules to make a process consistent– Example ISO 9000– ASTM standard method calibrating
micropipettor
CALIBRATION IS:
• Bringing a measuring system into accordance with external authority, using standards
• For example, calibrating a balance– Use standards that have known masses– Relate response of balance to units of kg– Do this in lab
PERFORMANCE VERIFICATION IS:
• Check of the performance of an instrument or method without adjusting it.
• Do this in lab.
TOLERANCE IS:
• Amount of error that is allowed in the calibration of a particular item. National and international standards specify tolerances.
EXAMPLE
• Standards for balance calibration can have slight variation from “true” value– Highest quality 100 g standards have a
tolerance of + 2.5 mg– 99.99975-100.00025 g– Leads to uncertainty in all weight
measurements
TRACEABILITY IS:
• The chain of calibrations, genealogy, that establishes the value of a standard or measurement
• In the U.S. traceability for most physical and some chemical standards goes back to NIST
TRACEABILITY
• Note in this catalog example, “traceable to NIST”
VOCABULARY
• Standards
• Calibration
• Traceability
• Tolerance
• Play with these ideas in labs
MEASUREMENT
• What are the characteristics of good measurement?
• Accuracy
• Precision
ACCURACY AND PRECISION ARE:
• Accuracy is how close an individual value is to the true or accepted value
• Precision is the consistency of a series of measurements
EXPRESS ACCURACY
% error = True value – measured value X 100%
True value
Will calculate this in volume lab
EXPRESS PRECISION
• Standard deviation (p. 187-190)– Expression of variability– Take the mean (average)– Calculate how much each measurement
deviates from mean– Take an average of the deviation, so it is the
average deviation from the mean
• Try this in the volume lab
ERROR IS:
• Error is responsible for the difference between a measured value and the “true” value
CATEGORIES OF ERRORS
• Three types of error:– Gross– Random– Systematic
GROSS ERROR
• Blunders
RANDOM ERROR
• In U.S., weigh particular 10 g standard every day. They see:– 9.999590 g, 9.999601 g, 9.999592 g ….
• What do you think about this?
RANDOM ERROR
• Variability
• No one knows why
• They correct for humidity, barometric pressure, temperature
• Error that cannot be eliminated. Called “random error”
RANDOM ERROR
• Do you think that repeating the measurement over and over would allow us to be more certain of the “true” weight of this standard?
RANDOM ERROR
• Yes, because in the presence of only random error, the mean is more likely to be correct if repeat the measurement many times
• Standard is probably really a bit light
• Average of all the values is a good estimate of its true weight
RANDOM ERROR AND ACCURACY
• In presence of only random error, average value will tend to be correct
• With only one or a few measurements, may or may not be accurate
THERE IS ALWAYS RANDOM ERROR
• If can’t see it, system isn’t sensitive enough
• Less sensitive balance: 10.00 g,
10.00 g, 10.00 g
Versus 9.999600 g…
MeanMedianMode
SO…
• Can we ever be positive of true weight of that standard?
• No
• There is uncertainty in every weight measurement
RELATIONSHIP RANDOM ERROR AND PRECISION
• Random error –– Leads to a loss of precision
SYSTEMATIC ERROR
• Defined as measurements that are consistently too high or too low, bias
• Many causes, contaminated solutions, malfunctioning instruments, temperature fluctuations, etc., etc.
SYSTEMATIC ERROR
• Technician controls sources of systematic error and should try to eliminate them, if possible– Temperature effects– Humidity effects – Calibration of instruments– Etc.
• In the presence of systematic error, does it help to repeat measurements?
SYSTEMATIC ERROR
• Systematic error – – Does impact accuracy
• Repeating measurements with systematic error does not improve the accuracy of the measurements
Match these descriptions with the 4 distributions in the figure:
Good precision, poor accuracy
Good accuracy, poor precision
Good accuracy, good precision
Poor accuracy, poor precision
ANOTHER DEFINTION OF ERROR IS:
• Error = is the difference between the measured value and the “true” value due to any cause
Absolute error = “True” value - measured value
• Percent error is:“True” value - measured value (100 %)
“True” value
ERRORS AND UNCERTAINTY
• Errors lead to uncertainty in measurements
• Can never know the exact, “true” value for any measurement.
• Idea of a “true” value is abstract – never knowable.
• In practice, get close enough
UNCERTAINTY IS:
• Estimate of the inaccuracy of a measurement that includes both the random and systematic components.
UNCERTAINTY ALSO IS:
• An estimate of the range within which the true value for a measurement lies, with a given probability level.
UNCERTAINTY
• Not surprisingly, it is difficult to state, with certainty, how much uncertainty there is in a measurement value.
• But that doesn’t keep metrologists from trying …
METROLOGISTS
• Metrologists try to figure out all the possible sources of uncertainty and estimate their magnitude
• One or another factor may be more significant. For example, when measuring very short lengths with micrometers, care a lot about repeatability. But, with measurements of longer lengths, temperature effects are far more important
REPORT VALUES
• Metrologists come up with a value for uncertainty
• You may see this in catalogues or specifications– Example:
measured value + an estimate of uncertainty
UNCERTAINTY ESTIMATES
• Details are not important to us now
• But principle is: any measurement, need to know where the important sources of error might be
SIGNIFICANT FIGURES
• One cause of uncertainty in all measurements is that the value for the measurement can only read to a certain number of places
• This type of uncertainty. It is called “resolution error”. (It is often evaluated using Type B methods.)
SIIGNIFICANT FIGURE CONVENTIONS
• Significant figure conventions are used to record the values from measurements
• Expression of uncertainty
• Also apply to very large counted values– Do not apply to “exact” values
• Counts where are certain of value• Conversion factors
ROUNDING CONVENTIONS
• Combine numbers in calculations
• Confusing
• Look up rules when they need them
RECORDING MEASURED VALUES
• Record measured values (or large counts) with correct number of significant figures
• Don’t add extra zeros; don’t drop ones that are significant
• With digital reading, record exactly what it says; assume the last value is estimated
• With analog values, record all measured values plus one that is estimated
• Discussed in Laboratory Exercise 1
ROUNDING
• A Biotechnology company specifies that the level of RNA impurities in a certain product must be less than or equal to 0.02%. If the level of RNA in a particular lot is 0.024%, does that lot meet the specifications?
• The specification is set at the hundredth decimal place. Therefore, the result is rounded to that place when it is reported. The result rounded is therefore 0.02%, and it meets the specification.
GOOD WEB SITE FOR SIGNIFICANT FIGURES
• http://antoine.frostburg.edu/cgi-bin/senese/tutorials/sigfig/index.cgi
THERMOMETERS
• Look at the values for the thermometers on the board.
• Significant figure conventions can guide us in how to record the value that we read off any measuring instrument.
• With these thermometers, correct number of sig figs is _______.
THERMOMETERS
• Were they accurate?
• How could we figure out the “true” value for the temperature?
REPEATING MEASUREMENTS
• Would repeating measurements with these thermometers, assuming we did not calibrate them, improve our ability to trust them?
• Is their error an example of random or systematic error?
CALIBRATION
• Calibration of the thermometers could lead to increased accuracy
• This is a type of systematic error
• In the presence of systematic error, repeating the measurement will not improve its accuracy
TOLERANCE
• Here is a catalog description of mercury thermometers.
• Are these thermometers out of the range for which their tolerance is specified?
PRECISION
• Were they precise? How could precision be measured?
• Would calibration help to make them more precise?
CALIBRATION
• Calibration would probably not improve their precision
RETURN TO OUR ORIGINAL TYPE OF QUESTION
• Are our temperature measurements “good” measurements?
• How do you make that judgment?
• Can we trust them?
THERMOMETERS – GOOD ENOUGH?
• Are times that we need to be very close in temperature measurements. For example PCR is fairly picky.
• Other times we can be pretty far off and process will still work.
EXPLORE SOME OF THESE IDEAS
• In lab:– Calibrate instruments– Use standards– Check performance of pipettors– Record measurement values– Calculate per cent errors– Calculate repeatability
ASSAYS
SAME IDEAS APPLY
• A good assay is one can trust when making a decision
• Accuracy and precision
• Linearity
• Limits