# ASTM E691.13092

Designation E691 19 An American National StandardStandard Practice forConducting an Interlaboratory Study to Determine thePrecision of a Test 1This standard is issued under the fixed designation E691; the number immediately following the designation indicates the year oforiginal adoption or, in the case of revision, the year of last revision. A number in parentheses indicates the year of last reapproval. Asuperscript epsilon indicates an editorial change since the last revision or reapproval.This standard has been approved for use by agencies of the U.S. Department of Defense.1. Scope1.1 This practice describes the techniques for planning,conducting, analyzing, and treating the results of an interlabo-ratory study ILS of a test . The statistical techniquesdescribed in this practice provide adequate ination forulating the precision statement of a test .1.2 This practice does not concern itself with the develop-ment of test s but rather with gathering the inationneeded for a test precision statement after the devel-opment stage has been successfully completed. The dataobtained in the interlaboratory study may indicate, however,that further effort is needed to improve the test .1.3 Since the primary purpose of this practice is the devel-opment of the ination needed for a precision statement, theexperimental design in this practice may not be optimum foruating materials, apparatus, or individual laboratories.1.4 Field of ApplicationThis practice is concerned exclu-sively with test s which yield a single numerical figureas the test result, although the single figure may be the outcomeof a calculation from a set of measurements.1.4.1 This practice does not cover s in which themeasurement is a categorization; however, for many practicalpurposes categorical outcomes can be scored, such as zero-onescoring for binary measurements or as integers, ranks forexample, for well-ordered categories and then the test resultcan be defined as an average, or other summary statistic, ofseveral individual scores.1.5 This standard may involve hazardous materials,operations, and equipment. This standard does not purport toaddress all of the safety concerns, if any, associated with itsuse. It is the responsibility of the user of this standard toestablish appropriate safety, health, and environmental prac-tices and determine the applicability of regulatory limitationsprior to use.1.6 This international standard was developed in accor-dance with internationally recognized principles on standard-ization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recom-mendations issued by the World Trade Organization TechnicalBarriers to Trade TBT Committee.2. Referenced Documents2.1 ASTM Standards2E29 Practice for Using Significant Digits in Test Data toDetermine Conance with SpecificationsE177 Practice for Use of the Terms Precision and Bias inASTM Test sE456 Terminology Relating to Quality and StatisticsE1169 Practice for Conducting Ruggedness TestsE1402 Guide for Sampling DesignE2282 Guide for Defining the Test Result of a Test 3. Terminology3.1 DefinitionsTerminology E456 provides a more exten-sive list of terms in E11 standards.3.1.1 accuracy, nthe closeness of agreement between atest result and an accepted reference value. E1773.1.2 bias, nthe difference between the expectation of thetest results and an accepted reference value. E1773.1.3 interlaboratory study, ILS in ASTM, na designedprocedure for obtaining a precision statement for a test ,involving multiple laboratories, each generating replicate testresults on one or more materials.3.1.4 observation, nthe process of obtaining inationregarding the presence or absence of an attribute of a testspecimen, or of making a reading on a characteristic ordimension of a test specimen. E22823.1.5 precision, nthe closeness of agreements betweenindependent test results obtained under stipulated conditions.E1771This practice is under the jurisdiction ofASTM Committee E11 on Quality andStatistics and is the direct responsibility of Subcommittee E11.20 on Test uation and Quality Control.Current edition approved Sept. 1, 2019. Published October 2019. Originallyapproved in 1979. Last previous edition approved in 2018 as E691 18. DOI10.1520/E0691-19.2For referenced ASTM standards, visit the ASTM website, www.astm.org, orcontact ASTM Customer Service at serviceastm.org. For Annual Book of ASTMStandards volume ination, refer to the standards Document Summary page onthe ASTM website.Copyright ASTM International, 100 Barr Harbor Drive, PO Box C700, West Conshohocken, PA 19428-2959. United StatesThis international standard was developed in accordance with internationally recognized principles on standardization established in the Decision on Principles for theDevelopment of International Standards, Guides and Recommendations issued by the World Trade Organization Technical Barriers to Trade TBT Committee.1Copyright by ASTM Intl all rights reserved; Wed Nov 6 004220 EST 2019Downloaded/printed byUniversity of Michigan University of Michigan pursuant to License Agreement. No further reproductions authorized.3.1.6 repeatability, nprecision under repeatabilityconditions. E1773.1.7 repeatability conditions, nconditions where inde-pendent test results are obtained with the same onidentical test items in the same laboratory by the same operatorusing the same equipment within short intervals of time. E1773.1.8 repeatability limit r, nthe value below which theabsolute difference between two individual test results obtainedunder repeatability conditions may be expected to occur with aprobability of approximately 0.95 95 . E1773.1.9 repeatability standard deviation, sr, nthe standarddeviation of test result obtained under repeatability conditions.E1773.1.10 reproducibility, nprecision under reproducibilityconditions. E1773.1.11 reproducibility conditions, nconditions where testresults are obtained with the same on identical testitems in different laboratories with different operators usingdifferent equipment. E1773.1.12 reproducibility limit R, nthe value below whichthe absolute difference between two test results obtained underreproducibility conditions may be expected to occur with aprobability of approximately 0.95 95 . E1773.1.13 reproducibility standard deviation sR, nthe stan-dard deviation of test results obtained under reproducibilityconditions. E1773.1.14 ruggedness test, na planned experiment in whichenvironmental factors or test conditions are deliberately variedin order to uate the effects of such variation. E11693.1.15 test determination, nthe value of a characteristic ordimension of a single test specimen derived from one or moreobserved values. E22823.1.16 test , na definitive procedure that producesa test result. E22823.1.17 test observation, nsee observation. E22823.1.18 test result, nthe value of a characteristic obtainedby carrying out a specified test . E22823.1.19 test specimen, nthe portion of a test unit needed toobtain a single test determination. E22823.1.20 test unit, nthe total quantity of material containingone or more test specimens needed to obtain a test result asspecified in the test ; see test result. E22823.2 Definitions of Terms Specific to This Standard3.2.1 average of the cell averages, x,nthe average of thecell averages for a particular material.3.2.2 between-laboratory consistency statistic, h, ntheratio of the cell deviation to the standard deviation of the cellaverages.3.2.2.1 DiscussionThis statistic is an indicator of how onelaboratorys cell average compares with the average of theother laboratories for a particular material see A1.2.2.3.2.3 between-laboratory standard deviation, sL,nthesample standard deviation attributable to differences of testresult means among laboratories.3.2.4 between-laboratory variance, sL2,nthe sample vari-ance component attributable to differences of test result meansamong laboratories.3.2.4.1 DiscussionThis statistic is estimated indirectlyfrom the variance of cell averages and the repeatabilityvariance. In situations where there is good agreement amonglaboratories the estimate of this variance component may beclose to zero or be negative. In the latter case, the estimate isset to zero. See Note 2 and A1.1.2.3.2.5 cell, nthe intersection of a row and column in atwo-way classification table, in which the rows represent thelaboratories and the columns represent the materials.3.2.5.1 DiscussionThe table holds the test results from aninterlaboratory study, and each cell contains the test resultsfrom a particular laboratory on a particular material seeSection 7 and Table 1.3.2.6 cell average, x,nthe average of the test results in aparticular cell.3.2.7 cell deviation, d, nthe cell average minus the aver-age of the cell averages.3.2.8 cell standard deviation, s, nthe standard deviation ofthe test results in a particular cell.3.2.9 repeatability variance, sr2,nthe sample variance oftest results obtained under repeatability conditions.3.2.9.1 DiscussionThis statistic is estimated for a materialas the pooled within-laboratory variances over all of thelaboratories in the ILS.3.2.10 reproducibility variance, sR2,nthe sample varianceof test results obtained under reproducibility conditions.3.2.10.1 DiscussionThis statistic is estimated as the sumof the two variance components due to between-laboratories,sL2, and within-laboratories, sr2.3.2.11 standard deviation of the cell averages, sx,nthestandard deviation of the cell averages for a particular material.3.2.12 variance of the cell averages, sx2,nthe samplevariance of the cell averages for a particular material.3.2.13 within-laboratory consistency statistic, k, nthe ra-tio of the cell standard deviation to the repeatability standarddeviation.3.2.13.1 DiscussionThis statistic is an indicator of howone laboratorys cell standard deviation under repeatabilityconditions compares with the repeatability standard deviationestimated from all laboratories for a particular material seeA1.2.3.4. Significance and Use4.1 ASTM regulations require precision statements in alltest s in terms of repeatability and reproducibility. Thispractice may be used in obtaining the needed ination assimply as possible. This ination may then be used toprepare a precision statement in accordance with PracticeE177. Knowledge of the test precision is useful incommerce and in technical work when comparing test resultsagainst standard values such as specification limits or be-tween data sources different laboratories, instruments, etc..E691 192Copyright by ASTM Intl all rights reserved; Wed Nov 6 004220 EST 2019Downloaded/printed byUniversity of Michigan University of Michigan pursuant to License Agreement. No further reproductions authorized.4.1.1 When a test is applied to a large number ofportions of a material that are as nearly alike as possible, thetest results obtained will not all have the same value. Ameasure of the degree of agreement among these test resultsdescribes the precision of the test for that material.Numerical measures of the variability between such test resultsprovide inverse measures of the precision of the test .Greater variability implies smaller that is, poorer precisionand larger imprecision.4.1.2 Precision is reported as a standard deviation, coeffi-cient of variation relative standard deviation, variance, or aprecision limit a data range indicating no statistically signifi-cant difference between test results.4.1.3 This practice is designed only to estimate the precisionof a test . However, when accepted reference values areavailable for the property levels, the test result data obtainedaccording to this practice may be used in estimating the bias ofthe test . For a discussion of bias estimation and therelationships between precision, bias, and accuracy, see Prac-tice E177.4.2 The procedures presented in this practice consist ofthree basic steps planning the interlaboratory study, guidingthe testing phase of the study, and analyzing the test result data.4.2.1 The planning phase includes ing the ILS taskgroup, the study design, selection and number of participatinglaboratories, selection of test materials, and writing the ILSprotocol. A well-developed test , including a rugged-ness test to determine control of test conditions, isessential.NOTE 1In this practice, the term test is used both for the actualmeasurement process and for the written description of the process, whilethe term protocol is used for the directions given to the laboratories forconducting the ILS.4.2.2 The testing phase includes material preparation anddistribution, liaison with the participating laboratories, andhandling of test result data received from the laboratories.4.2.3 The data analysis utilizes tabular, graphical, and sta-tistical diagnostic tools for uating the consistency of thedata so that unusual values may be detected and investigated,and also includes the calculation of the numerical measures ofprecision of the test pertaining to repeatability andreproducibility.4.3 The ination in this practice is arranged as followsSectionScope 1Referenced Documents 2Terminology 3Significance and Use 4Concepts of Test Precision 5Planning the Interlaboratory Study ILS SectionILS Membership 6Basic Design 7Test 8Laboratories 9Materials 10Number of Test Results per Material 11Protocol 12Conducting the Testing Phase of the ILS SectionPilot Run 13Full Scale Run 14Calculation and Display of Statistics SectionCalculation of the Statistics 15Tabular and Graphical Display of Statistics 16Data Consistency SectionFlagging Inconsistent Results 17Investigation 18Task Group Actions 19Glucose ILS Consistency 20Precision Statement Ination SectionRepeatability and Reproducibility 21SectionKeywords 22Tables TableGlucose in Serum Example 14, 68Critical Values of Consistency Statistics, h and k 5Figures FigureGlucose in Serum Example 13Anns AnnexTheoretical Considerations Annex A1Calculation of the ILS Statistics for Unbalanced Data Sets Annex A2Appendixes AppendixPentosans in Pulp Example Appendix X1Spreadsheet for E691 Calculations Appendix X25. Concepts of Test Precision5.1 Repeatability and ReproducibilityThese two termsdeal with the variability of test results obtained under specifiedlaboratory conditions and represent the two extremes of test precision. Repeatability concerns the variability be-tween independent test results obtained within a single labo-ratory in the shortest practical period of time by a singleoperator with a specific set of test apparatus using testspecimens or test units taken at random from a single quant