Designation: C1215 − 92 (Reapproved 2012)´1Standard Guide forPreparing and Interpreting Precision and Bias Statements inTest Method Standards Used in the Nuclear Industry1This standard is issued under the fixed designation C1215; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon (´) indicates an editorial change since the last revision or reapproval.ε1NOTE—Changes were made editorially in June 2012.INTRODUCTIONTest method standards are required to contain precision and bias statements. This guide contains aglossary that explains various terms that often appear in these statements as well as an exampleillustrating such statements for a specific set of data. Precision and bias statements are shown to varyaccording to the conditions under which the data were collected. This guide emphasizes that the errormodel (an algebraic expression that describes how the various sources of variation affect themeasurement) is an important consideration in the formation of precision and bias statements.1. Scope1.1 This guide covers terminology useful for the preparationand interpretation of precision and bias statements. This guidedoes not recommend a specific error model or statisticalmethod. It provides awareness of terminology and approachesand options to use for precision and bias statements.1.2 In formulating precision and bias statements, it isimportant to understand the statistical concepts involved and toidentify the major sources of variation that affect results.Appendix X1 provides a brief summary of these concepts.1.3 To illustrate the statistical concepts and to demonstratesome sources of variation, a hypothetical data set has beenanalyzed in Appendix X2. Reference to this example is madethroughout this guide.1.4 It is difficult and at times impossible to ship nuclearmaterials for interlaboratory testing. Thus, precision statementsfor test methods relating to nuclear materials will ordinarilyreflect only within-laboratory variation.1.5 No units are used in this statistical analysis.1.6 This guide does not involve the use of materials,operations, or equipment and does not address any riskassociated.2. Referenced Documents2.1 ASTM Standards:2E177 Practice for Use of the Terms Precision and Bias inASTM Test MethodsE691 Practice for Conducting an Interlaboratory Study toDetermine the Precision of a Test Method2.2 ANSI Standard:ANSI N15.5 Statistical Terminology and Notation forNuclear Materials Management33. Terminology for Precision and Bias Statements3.1 Definitions:3.1.1 accuracy (seebias) —(1) bias. (2) the closeness of ameasured value to the true value. (3) the closeness of ameasured value to an accepted reference or standard value.3.1.1.1 Discussion—For many investigators, accuracy isattained only if a procedure is both precise and unbiased (seebias). Because this blending of precision into accuracy canresult occasionally in incorrect analyses and unclear statementsof results, ASTM requires statement on bias instead of accu-racy.43.1.2 analysis of variance (ANOVA)—the body of statisticaltheory, methods, and practices in which the variation in a set ofdata is partitioned into identifiable sources of variation.1This guide is under the jurisdiction of ASTM Committee C26 on Nuclear FuelCycle and is the direct responsibility of Subcommittee C26.08 on QualityAssurance, Statistical Applications, and Reference Materials.Current edition approved June 1, 2012. Published June 2012. Originallyapproved in 1992. Last previous edition approved in 2006 as C1215–92(2006). DOI:10.1520/C1215-92R12E01.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at

[email protected] For Annual Book of ASTMStandards volume information, refer to the standard’s Document Summary page onthe ASTM website.3Available from American National Standards Institute (ANSI), 25 W. 43rd St.,4th Floor, New York, NY 10036, http://www.ansi.org.4Refer to Form and Style for ASTM Standards, 8th Ed., 1989, ASTM.Copyright © ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United States1Sources of variation may include analysts, instruments,samples, and laboratories. To use the analysis of variance, thedata collection method must be carefully designed based on amodel that includes all the sources of variation of interest. (SeeExample, X2.1.1)3.1.3 bias (see accuracy)—a constant positive or negativedeviation of the method average from the correct value oraccepted reference value.3.1.3.1 Discussion—Bias represents a constant error as op-posed to a random error.(a) A method bias can be estimated by the difference (orrelative difference) between a measured average and an ac-cepted standard or reference value. The data from which theestimate is obtained should be statistically analyzed to establishbias in the presence of random error. A thorough bias investi-gation of a measurement procedure requires a statisticallydesigned experiment to repeatedly measure, under essentiallythe same conditions, a set of standards or reference materials ofknown value that cover the range of application. Bias oftenvaries with the range of application and should be reportedaccordingly.(b) In statistical terminology, an estimator is said to beunbiased if its expected value is equal to the true value of theparameter being estimated. (See Appendix X1.)(c) The bias of a test method is also commonly indicated byanalytical chemists as percent recovery. A number of repeti-tions of the test method on a reference material are performed,and an average percent recovery is calculated. This averageprovides an estimate of the test method bias, which is multi-plicative in nature, not additive. (See Appendix X2.)(d) Use of a single test result to estimate bias is stronglydiscouraged because, even if there were no bias, random erroralone would produce a nonzero bias estimate.3.1.4 coeffıcient of variation—see relative standard devia-tion.3.1.5 confidence interval—an interval used to bound thevalue of a population parameter with a specified degree ofconfidence (this is an interval that has different values fordifferent random samples).3.1.5.1 Discussion—When providing a confidence interval,analysts should give the number of observations on which theinterval is based. The specified degree of confidence is usually90, 95, or 99 %. The form of a confidence interval depends onunderlying assumptions and intentions. Usually, confidenceintervals are taken to be symmetric, but that is not necessarilyso, as in the case of confidence intervals for variances.Construction of a symmetric confidence interval for a popula-tion mean is discussed in Appendix X3.It is important to realize that a given confidence-intervalestimate either does or does not contain the populationparameter. The degree of confidence is actually in theprocedure. For example, if the interval (9, 13) is a 90 %confidence interval for the mean, we are confident that theprocedure (take a sample, construct an interval) by which theinterval (9, 13) was constructed will 90 % of the timeproduce an interval that does indeed contain the mean.Likewise, we are confident that 10 % of the time the intervalestimate obtained will not contain the mean. Note that theabsence of sample size information detracts from the use-fulness of the confidence interval. If the interval were basedon five observations, a second set of five might produce avery different interval. This would not be the case if 50observations were taken.3.1.6 confidence level—the probability, usually expressed asa percent, that a confidence interval will contain the parameterof interest. (See discussion of confidence interval in AppendixX3.)3.1.7 error model—an algebraic expression that describeshow a measurement is affected by error and other sources ofvariation. The model may or may not include a sampling errorterm.3.1.7.1 Discussion—A measurement error is an error attrib-utable to the measurement process. The error may affect themeasurement in many ways and it is important to correctlymodel the effect of the error on the measurement.(a) Two common models are the additive and the multi-plicative error models. In the additive model, the errors areindependent of the value of the item being measured. Thus,for example, for repeated measurements under identicalconditions, the additive error model might beXi5 µ1b1εi(1)where:Xi= the result of the ithmeasurement,µ = the true value of the item,b = a bias, andεi= a random error usually assumed to have a normaldistribution with mean zero and variance σ2.In the multiplicative model, the error is proportional to thetrue value.Amultiplicative error model for percent recovery(see bias) might be:Xi5 µbεi(2)and a multiplicative model for a neutron counter mea-surement might be:Xi5 µ1µb1µ·εi(3)5µ~11b1εi!(b) Clearly, there are many ways in which errors mayaffect a final measurement. The additive model is fre-quently assumed and is the basis for many common statis-tical procedures. The form of the model influences howthe error components will be estimated and is veryimportant, for example, in the determination of measure-ment uncertainties. Further discussion of models is givenin the Example of Appendix X2 and in Appendix X4.3.1.8 precision—a generic concept used to describe thedispersion of a set of measured values.3.1.8.1 Discussion—It is important that some quantitativemeasure be used to specify precision. A statement such as,“The precision is 1.54 g” is useless. Measures frequently usedto express precision are standard deviation, relative standarddeviation, variance, repeatability, reproducibility, confidenceinterval, and range. In addition to specifying the measure andthe precision, it is important that the number of repeatedC1215 − 92 (2012)´12measurements upon which the precision estimated is based alsobe given. (See Example, Appendix X2.)(a) It is strongly recommended that a statement onprecision of a measurement procedure include the following:(1) A description of the procedure used to obtain the data,(2) The number of repetitions, n, of the measurementprocedure,(3) The sample mean and standard deviation of themeasurements,(4) The measure of precision being reported,(5) The computed value of that measure, and(6) The applicable range or concentration.The importance of items (3) and (4) lies in the fact thatwith these a reader may calculate a confidence interval orrelative standard deviation as desired.(b) Precision is sometimes measured by repeatability andreproducibility (see Practice E177, and Mandel and Laskof(1)). The ANSI and ASTM documents differ slightly in theirusages of these terms. The following is quoted from Kendalland Buckland (2):“In some situations, especially interlaboratorycomparisons, precision is defined by employing two addi-tional concepts: repeatability and reproducibility. The gen-eral situation giving rise to these distinctions comes from theinterest in assessing the variability within several groups ofmeasurements and between those groups of measurements.Repeatability, then, refers to the within-group dispersion ofthe measurements, while reproducibility refers to thebetween-group dispersion. In interlaboratory comparisonstudies, for example, the investigation seeks to determinehow well each laboratory can repeat its measurements(repeatability) and how well the laboratories agree with eachother (reproducibility). Similar discussions can apply to thecomparison of laboratory technicians’ skills, the study ofcompeting types of equipment, and the use of particularprocedures within a laboratory. An essential feature usuallyrequired, however, is that repeatability and reproducibilitybe measured as variances (or standard deviations in certaininstances), so that both within- and between-group disper-sions are modeled as a random variable. The statistical tooluseful for the analysis of such comparisons is the analysis ofvariance.”(c) In Practice E177 it is recommended that the termrepeatability be reserved for the intrinsic variation due solelyto the measurement procedure, excluding all variation fromfactors such as analyst, time and laboratory and reservingreproducibility for the variation due to all factors includinglaboratory. Repeatability can be measured by the standarddeviation, σr,ofn consecutive measurements by the sameoperator on the same instrument. Reproducibility can bemeasured by the standard deviation, σR,ofm measurements,one obtained from each of m independent laboratories. Wheninterlaboratory testing is not practical, the reproducibilityconditions should be described.(d) Two additional terms are recommended in PracticeE177. These are repeatability limit and reproducibility limit.These are intended to give estimates of how different twomeasurements can be. The repeatability limit is defined as1.96=2sr, and the reproducibility limit is defined as 1.96=2sR,where sris the estimated standard deviation associated withrepeatability, and sRis the estimated standard deviation asso-ciated with reproducibility. Thus, if normality can be assumed,these limits represent 95 % limits for the difference betweentwo measurements taken under the respective conditions. In thereproducibility case, this means that “approximately 95 % ofall pairs of test results from laboratories similar to those in thestudy can be expected to differ in absolute value by less than1.96=2sR.” It is important to realize that if a particular sRis apoor estimate of σR, the 95 % figure may be substantially inerror. For this reason, estimates should be based on adequatesample sizes.3.1.9 propagation of variance—a procedure by which themean and variance of a function of one or more randomvariables can be expressed in terms of the mean, variance, andcovariances of the individual random variables themselves(Syn. variance propagation, propagation of error).3.1.9.1 Discussion—There are a number of simple exactformulas and Taylor series approximations which are usefulhere (3, 4).3.1.10 random error—(1) the chance variation encounteredin all measurement work, characterized by the random occur-rence of deviations from the mean value. (2) an error thataffects each member of a set of data (measurements) in adifferent manner.3.1.11 random sample (measurements)—a set of measure-ments taken on a single item or on similar items in such a waythat the measurements are independent and have the sameprobability distribution.3.1.11.1 Discussion—Some authors refer to this as a simplerandom sample. One must then be careful to distinguishbetween a simple random sample from a finite population of Nitems and a simple random sample from an infinite population.In the former case, a simple random sample is a sample chosenin such a way that all samples of the same size have the samechance of being selected. An example of the latter case occurswhen ta