ni.com

Avoiding Measurement Errors from Manipulating Data in Software

Speaker/Author: Logan Kunitz

Staff Calibration Engineer

National Instruments R&D

Are you sure your software isn’t adding uncertainty to your measurements???

2ni.com

Sources of Measurement Error

ExternalConditioning and Cabling

Acquire Data

ADC+_

Signal Source Process and

Display Reading

Offset / Biasand Noise Errors

Digitization and Sampling Errors

Software Errors

3ni.com

Sources of Software Errors

• Coding Mistakes / Logic Errors• Often a logic error in software will result in very large errors,

but sometimes these errors can be small enough to go undetected

• Example – units conversion issues (imperial vs. metric)

• Data Processing Errors• i.e. Linear Approximation, Data Extrapolation, Averaging, etc…

• Data Manipulation Errors• Artifacts of how data is handled by the compiler and processor• Quantify to ensure negligible impact to the measurement• Examples:

o Rounding Errorso Numerical Errors

4ni.com

Defining “Negligible” Error Impact

• Software Error Expectation: “Negligible Impact”• Should not need to account for these errors in

uncertainty analysis

• Ok… so, how do I define negligible?• “Really Small”• Just go a ‘couple’ digits past the least significant digit• “10 times smaller is enough”• As long as it doesn’t affect the application

• Unable to find a quantifiable and objective definition

• So… I came up with my own definition

5ni.com

Logan’s Definition of “Negligible” Error Impact• Negligble U(x) Definition

• A number that does not affect reported measurement uncertainty U(y) with >95% confidence

• If Uncertainty U(y) = 2.3• U(x) <0.005 95% chance of not affecting

uncertainty U(y)

6ni.com

• When the number of displayed digits is truncated (and rounded)

• 1.2468642 -> 1.2469 -> 1.2 -> 1

• Error from Rounding• +/- ½ Least Significant Digit or 5x10(-Digits)

• Example: • 2 Digits Displayed• 1.246 -> 1.2• Error = +/- 0.05

• Negligible = Equivalent to rounding to 1 digit past Least Significant Digit of Uncertainty

Rounding Errors

Actual Value Reading = 1.2

1.0

1.1

1.2

1.3

7ni.com

Rounding Error Use Cases

• Numeric -> String Conversions• When numeric data gets converted to a string,

least significant digits will be rounded/truncated based on the application requirements

• Examples:• Displaying measurement results in a user

interface o i.e. DMM # of Digits

• Logging results to a report• Logging data to a file (such as .csv, XML, Excel,

etc..), to be read back by the same or different application

• Converting a number to string for data transfer (i.e. GPIB / Serial communication with and instrument, or transferring to another application over TCP/IP)

8ni.com

Selecting Format for Numeric Representation

Format Type Example Advantages/Disadvantages

Standard Floating Point Representation

1234.56789 Advantage: SimpleDisadvantage: Wasted 0’s displayed for large and small numbers (i.e. 120000000, or 0.000000456)

SI notation 1.23456789k Advantage: Reduce # 0’s reported for large and small numbers

Disadvantage: Limited set of prefixes (i.e. no prefix for 10^-5)

Engineering/Scientific Notation

1.23456789E3 or 1.23456789x103

Advantage: More flexible exponent over SI notation

Advantage: Common format for instrument communication and data storage

BinaryRepresenting the binary equivalent of the number in a string (byte array)

@“JE„ôÆç Advantage: No precision loss

Advantage: Smaller data footprint, lower storage space required

Disadvantage: Implementation is application-specific

9ni.com

Numerical Errors

• Errors resulting from choice of datatype• Floating Point Numbers

o Single Precisiono Double Precisiono Extended Precision

• Integers (i16, i32, i64, u8, etc…)

• Numeric Datatypes are Approximations• Limitation of computers’ representation of the numeric values

|

100 15050001.25

5.00

2.503.75

6.257.508.75

10.00

|||

High Resolution vs. Low Resolution

10

ni.com

Floating Point Numeric Details

• IEEE 754 standardizes the floating point implementation

• Single (32 bit) and Double (64 bit) are most common and consistent implementations

• Extended precision (80 bit)

• Other Numeric Data-types for High Precision in Software• 128 bit Extended Precision• Arbitrary-Precision Arithmetic

Single ->

Double->

11

ni.com

Numerical Error Example

• Error representing Pi using different numeric datatypes

Datatype Value of Pi Aprox. Digits

Error

Actual 3.1415926535897932384626433832795028… - -

Single Precision Float (32 bit)

3.1415927410125732400 7 2.78E-6%

Double Precision Float (64 bit)

3.1415926535897926700 15 1.8E-14%

Extended Double Precision Float (80 bit)

3.1415926535897932380 19 1.4E-17%

Integer 3 - 4.5%

12

ni.com

Machine Epsilon:• Quantified error for a floating point number• Value is the ratio of maximum error to the stored value• Development Environments (ADEs) often have a function to calculate

Datatype Machine Epsilon

Decimal Digits of Precision

Single Precision 2 -23 = 1.19e-07 ~7.2 DigitsDouble Precision 2 -52 = 2.22e-16 ~15.9 digitsExtended Double (80-bit)

2 -63 = 1.08e-19 ~19.2 digits

13

ni.com

Function Examples Error Equations

Error for a Single Value

Addition Operations

Multiplication Operations

Using Machine Epsilon• The following functions describe basic methods for applying machine

epsilon based on the math operations of the components• Note: x and y will only be different if X and Y are different datatypes

14

ni.com

Issues Comparing Floating Point Numbers

• Using “equals” operation with floating numbers can fail to return expected resulted

• Example:• Evaluates to “False”• C version• G version

• Why?• Complicated• Because 0.1 is an approximation

when represented as floatingnumber

15

ni.com

Use Machine Epsilon to Compare Floating Point Numbers• Apply calculated machine epsilon error to the floating point results

• Example:• C version• G version

16

ni.com

Datatype Conversions

• Errors associated with conversion operations on datatypes in an application

• Many conversion handled “automatically” by the ADE

• Example: function z = x + y• C Example:

Single z; Double x; Double y;z = x + y;

• G Example (LabVIEW)

• Double precision numbers get coerced to Single precision after addition operation

• Use machine epsilon of coerced datatype to determine error

17

ni.com

Conclusion• Quantify if an error is ‘negligible’ for your applications• Avoid rounding errors when converting from numeric -> string• Numeric datatype selection should match precision requirements of your data

• Use machine epsilon to quantify error of floating precision numbers

• Never compare two floating point numbers using the ‘equals’ operator

References:

1. "IEEE Floating Point." Wikipedia. <http://en.wikipedia.org/wiki/IEEE_floating_point>.2. "Extended Precision." Wikipedia. <http://en.wikipedia.org/wiki/Extended_precision>.3. "Machine Epsilon." Wikipedia. <http://en.wikipedia.org/wiki/Machine_epsilon>.4. "Floating-point Errors." UAlberta.ca. <http://www.ualberta.ca/~kbeach/comp_phys/fp_err.html>.5. "Arbitrary-precision Arithmetic." Wikipedia. <http://en.wikipedia.org/wiki/Arbitrary-precision_arithmetic>.6. "Digital Multimeter Measurement Fundamentals." NI.com. <http://www.ni.com/white-paper/3295/en/>.7. Schmidt, Darren. "An Introduction to Floating-Point Behavior in LabVIEW." NI.com.

<ftp://ftp.ni.com/pub/devzone/tut/floating_point_in_labview.pdf>.

18

ni.com

Questions?