Overview of Gauge Tool Studies and the Agilent 5DX

Agilent Technologies

Purpose: In order to understand the variability of a manufacturing process it is necessary to monitor the salient characteristics of the end product such as ball diameter for a roller bearing, or solder joint thickness for a printed circuit board assembly. The variability of these characteristic values will contain variability produced by the process, by the individual conducting the measurements, and by the measurement tool. If the uncertainty introduced by the measurement activity is sufficiently large, the manufacturing organization may inappropriately respond to apparent trends that are not the result of the process itself. The purpose of the gauge tool study (GR&R) is to ascertain the contributions of the measurement tool and the measurement tool operator (Repeatability and Reproducibility.)

Contents: Without creating a detailed list, this document contains a discussion regarding the scope of the study along with advice to facilitate preparation. Another section explains features of the 5DX that are needed to perform the study, as well as step-by-step instructions regarding the analysis spreadsheet. Having completed the analysis, a brief presentation expands on the two analytical techniques used, and assists in the interpretation of the results. To be complete, a glossary and collection of references are provided at the conclusion.

Anticipated Results: The user is generally seeking a “Precision-to-Tolerance” capability for the measurement system. This P/T ratio is expected to satisfy a “Gauge Rule” in that the variability of the measurement technique should not exceed about 30% of the total available specification range.

In this P/T ratio, both the numerator and the denominator will be influenced by decisions that are made as the tests are defined. It is recommended that the testing and specifications be defined such that they are directly relevant to the process that is to be monitored. For example, if the instrument is to be used on multiple shifts, by multiple operators, then these situations should be included. Similarly, the tolerance used in the denominator should be based on the realistic needs of the process.

Caveats: It is important to note that although the nature of gauge tool studies is universal, the procedures for conducting them are not. This inconsistency is brought about by the specific application, the immediate needs of the organization, and the behavior of the instrument itself. At one extreme, a metrology engineer may be seeking to characterize each of the primary variables that contribute to machine precision. At the other, the process engineer may be seeking a quick affirmation of the instrument performance in the context of total process variability. When conducting the study the measurements need to be rigorously collected in an unbiased manner, and the data should be accurately analyzed. Any conclusions that are subsequently extracted should not exceed the scope of the testing.

In the segments that follow recommendations regarding sampling and procedure will be offered, but the user should not interpret this as the “approved cook book.” There will be omissions of detail, and the strategy may not match the local objectives. The users own statistical resource and quality assurance staff should be consulted before executing Agilent Technologies Overview of Gauge Tool Studies, Rev. F, April 2, 2001 Page 1 of 12 Agilent Technologies the study. Having said that, the format for study that is delineated, is statistically rigorous using both a range technique and an ANOVA technique. The two techniques are provided to assist the user by providing a methodology with which they are likely familiar. The sampling plan is tightly controlled in order to emphasize instrument and operator effects as well as to simplify implementation of the study.

Preparation: This section will highlight a number of details that should be addressed prior to testing.  Sample Types: Select challenging samples that are representative of the process. Fine-pitch gull wing joints (FPGW) and connectors are frequently chosen at the minimum anticipated geometry. J-Lead parts, resistors, and capacitors can be informative but they are less remarkable. BGA and SOTs can also present challenges. This analysis software package will only perform studies for joints inspected using, Gullwing, Fine Pitch Gullwing, J-lead, or BGA2 algorithms.

 Parameters: Joint parameters such as heel, toe, and center thickness are of particular interest because they are directly measured. Other parameters such as fillet length are inferred from thickness measurements and are perhaps not as interesting unless the user is particularly inquisitive. The user will employ filters during the Review Measurements process to select specific joints but the analysis package will provide results related to only the following parameters. o FPGW: . Heel thickness . Center thickness . Toe thickness . Full Pad Volume o J-Lead or Gullwing . Average Thickness . Heel Thickness . Center Thickness . Toe Thickness o BGA2 . Diameter . Thickness . Circularity . Flattening

 Test Machine: The user likely has a particular set of reasons for selecting one or more machines for this test. However, a separate study must be conducted on each machine in order to preserve the simplicity of this test package. Although several machines can be tested concurrently, it is recommended that the first study include a single machine. This will allow the process of testing and analysis to be exercised, evaluated, and modified. There is generally a wealth of insight obtained from the first event that will impact subsequent studies.

Agilent Technologies Overview of Gauge Tool Studies, Rev. F, April 2, 2001 Page 2 of 12 Agilent Technologies

 Sample Boards: The test samples need not be functional, but the 30 joints measured for the test should be representative of those provided by the production line. The same samples must be used throughout the testing. Only one board will be used for each joint type in this study, but if more than one joint type is available on the board it may be used for each type.

The single board restriction is imposed to avoid a situation where the process variation could mask the measurement variation.

It is recommended that the boards used in this study be set-aside as golden- boards for future verification testing, and matching of instruments.

 Program Setup: The 5DX programs for the test samples should be carefully setup and verified to insure proper operation without operator intervention. The programming techniques used on the samples should be consistent with those used on other products. Typical areas of difficulty include slice selection, surface map setup, algorithm tuning, and focus.

 Machine Confirmation and Adjustment: It is recommended that a full C&A be performed prior to the GR&R test sequence. Other C&A procedures such as Photocal and Selftest should occur at normal frequencies. Any maintenance event that transpires should be documented and the data should be evaluated for any adverse impact.

 Instrument Operation: All normal production evolutions should be conducted during the term of the test. Again, the intent is to evaluate the instrument’s performance in a manner that represents the instrument’s normal use. So, if the machine runs continuously it should be properly warmed prior to each test sequence. If the machine has a modest duty cycle, attempt to replicate that activity level throughout the course of the study. If there is no preference on the part of the user, it is recommended that the 5DX be exercised continuously for 1 hour prior to each test session.

 Operator Selection: Again, the five individuals chosen to operate the instrument during each test sequence should be sufficiently skilled to prevent adverse results. Similarly, the operators selected should be representative of the entire staff of operators. If the facility has less than five qualified operators then operators may conduct the test more than once, but the sequence should be randomized and a delay of not less than four hours should occur between any given operator’s measurements. (The analysis spreadsheet will call for an operator designation. This can be threatening to a worker and an actual name is not necessary.)

Because 5DX is largely automated, the influence of multiple operators may be modest. If a noticeable value is reported it may be attributable to the operator, a shift dependence, or a variation of the instrument over time.

 Randomization: Statistical studies generally benefit from randomization during the data collection. This is because it minimizes the adverse effects of uncontrolled variables which may be changing during the course of the experiment. In the execution of this study, measurement sets should transpire over several days, and on different shifts. Each operator’s activity should be separated from that of the other operators by several hours. If multiple sample boards are required to test the differing joint types, then alter the sequence in which these are tested as well.

 Dry Runs: A dry run is useful to exercise the machine, the data collection methodology, and the data analysis processes. Inevitably all will not go as planned. Also, early preparation of the analytical spreadsheets will permit rapid response to those tough questions that will come immediately at the conclusion of the study.

5DX Features: Several tips are provided below which will facilitate the test sequences and data collection.  Auto Demo: Normally when the 5DX starts the user interface is provided automatically. This interface facilitates the inspection of one board at a time, but by exploiting the Auto –demo mode a single board can be loaded and then repeatedly measured without assistance. This repeated measurement sequence includes alignment, surface mapping, and inspection so it simulates a normal production run with the exception of board load, clamp, unclamp, unload, sequence. To take advantage of this mode simply exit the user interface, System Access/Quit from 5DX, and then enter Auto –demo at the prompt. Load the test board using the Utilities menu, and execute the program from Test/Board. The data for each subsequent run is saved as ‘demo1’, ‘demo2’ etc. and can be accessed through Review Results as explained below. The Auto -demo technique can also be used to warm-up the instrument should the user choose to, prior to each test.

After the desired number of inspection trials has been completed press Escape and respond with a ‘Yes’ to the query. To return to normal operation simply exit the user interface as before, and enter Auto at the prompt.

 Data Filter and Report Generation: The accompanying analysis spreadsheets will anticipate a data file in a specific format for each joint type. This section will offer the details necessary to create those reports and output the data in the needed ASCII text file format.

Data Filters are most easily set up after a series of three trial inspections has been conducted on the test panel. If the test panel contains more than one of the joint types of interest then filters can be created for each. o From the main menu in the user interface select Panel Programming followed by Algorithm Setup, and Review Measurements. o A new window will open with a menu at the top. Select File or use Alt-F. o Select Tests, Ctrl-T

o Mouse click on the three files of interest. If the test was run by executing the inspections one at a time the user will recognize the names entered at the time of inspection. If Auto –demo mode was used, the inspections will be labeled as Demo_. Because only a partial data set will be available for the inspection in progress when the user escapes from the Auto –demo sequence, do not use the last demo file listed. o Select OK, and the ‘Select Tests’ window will close. o Select Alt-F, and Create Data Sheet, F4 o In the field indicating a Filter Name, enter a name that reflects the part or joint type. For example: U28FPGW. (Once this filter is established, simply use the down arrow adjacent to the Filter Name field and select it from the list. No additional editing will be required.) . Select Edit Filter. (Take note of the commands available in this mode that are presented at the bottom of the display window.) . Each of the items present can be altered to narrow the data search. Highlight the All under the heading Components and then press ALT-I for insert. Select the desired component from the menu. . Next highlight the All term under Pins. As before use ALT-I to insert each of the 30 joints for this particular component. This is tedious in that each pin must be entered with its own ALT-I. There are no combinations or shortcuts. For the situation where one component does not have 30 joints, specify more than one component. In this case, ALL pins can be used. If the total pin count for two or more components exceeds 30 the extra pins will simply not be evaluated. . Select ALL under Measurements next. Then select SPC. This will provide a number of parameters that are not evaluated by the analysis tool but the data will be provided in the anticipated format.  The BGA2 setup is slightly different, and we will be somewhat restrictive in this application. Under the Conditions heading, highlight Slice Range, and press Alt-M to permit modification.  In the window provided, select the Slice Range mode.  In the Slice Range field enter the slice of interest, [2,2]. Only one slice range can be accommodated by the analysis package. A slice near the ball center is likely to provide a more reliable evaluation. . Select FILE and SAVE to protect your work. Then FILE and EXIT. o This will return to the window labeled ‘Generate New Raw Data Sheet’. o At this juncture the files to be sorted have been identified and the filters are in place. Select OK to create the data sheet. o A blue background ‘RAW DATA SHEET’ will open. Check for several details: . All three, and only three, data files should be referenced. If Auto – demo mode was used then they will be named Demo_.

. Check to see the pin numbers of interest were collected. At a minimum there must be 90 measurement entries in the file grouped by three measurements of each pin. This figure of 90 joints may be exceeded if multiple components had been designated in order to reach the needed 30 joints. As mentioned, data beyond the first 90 entries will not be used. o Save the file as an ASCII text file. Select FILE, followed by Write to ASCII CTRL-W. o Enter a path and a name for the file. It is possible to write directly to the floppy disk in which case the entry might look like: A:\U85Oper1.txt If only the file name is specified and no path, then look for the file in the master directory. C:\5DX\Rnn\master o Exit the menus and close the raw data files to return to the main menu. The filters which have been created will be preserved for later use.

Executing the Study: With all the preparation in place, executing the study is relatively straightforward.

 In the case of a single joint type, there will be a single board to be tested three times, by each of five different operators. Total test time will likely be less than two hours. Should you choose to warm the machine for an hour before each test set, an additional 5 hours of system time will be required.  Verify operation and maintenance logs to discover any maintenance, calibration or failure events, which have occurred since the full C&A that was performed in anticipation of this study. Check again before each of the five operators conducts a test.  Self-test is typically set to run at 15 or 30-minute intervals. This should remain in place as the testing is conducted. Again, we wish to accurately replicate the instrument’s normal mode of operation.  Either Auto –demo may be used to collect the data, or the panel may simply be loaded repeatedly.  After inspection use Review Measurements as described to create the ASCII text files for use by the analysis tool, and return the machine to production. Be certain that the filters are properly setup per the section above. If the files are not properly formatted, the analysis package won’t be able to import the data correctly.  There are no special setups or adjustments required for the gauge tool study.  Tolerances must be specified by the user within the analysis package for each parameter of each joint type. The value of the P/T ratio is strongly dependant on the tolerance entered. The user should carefully consider what range is reasonable for the solder process being inspected.

Step-by-Step Procedure: What follows is a copy of the specific instructions provided within the analysis package.

Instructions for Using the Spreadsheet

Note: This application requires the use of the Add-in Analysis Toolpak - VBA. To verify its availability, select Tools/Add-Ins. Make sure the box for this add-in is checked. Also refer to the accompanying instructional document. "Overview of Gauge Tool Studies and the Agilent 5DX".

1. Click the tab Data 1 in the lower right corner of the screen. 2. Hit Ctrl-Home to go to the top of the page. 3. Click the button 4. Open the first data file in Notepad (Start --> Programs --> Accessories --> Notepad). 5. Retrieve the data file with File --> Open. 6. Select all the text in the data file and choose Edit -->Select All. 7. Next prepare to copy with Edit --> Copy from the same menu. 8. Switch back to the Excel spreadsheet choose Edit --> Paste. 9. Repeat steps 1 through 6 for the remaining data files on the other four data tabs. 10. Click the tab named Summary in the lower left corner of the screen. 11. Fill in the information related to the study. (Light blue backgrounds are for data entry.) 12. Click the button. 13. At this point the calculations are completed but Precison-to-Tolernace (P/T) ratios are not valid until viable tolerances are appropriately set. Review the measurement Means and the process specifications, then adjust the tolerances on the Summary tab as required. For convenience, the Summary tab contains the respective grand means from the Range tabs. Additionally two possible tolerance values are presented. The first is simply +/- 30% from the grand mean. The second is an estimate based on the joint Variance Component. (See the explanation notes.)

Evaluating The Results: The analysis package will have many worksheet tabs but the first two are summaries of the Gauge Repeatability and Reproducibility Study as performed using an Analysis of Variance (ANOVA) technique and a Range technique. The range technique is the most popular in industry because it requires a minimum of measurements and because the calculations are straightforward. The disadvantage of the range technique is that it attempts to estimate the standard deviation of the measurements based on the range (i.e. the highest measurement minus the lowest measurement). While viable, this methodology unfortunately neglects the information provided by the other measurements. As a result, it is relatively inefficient when compared to estimating the standard deviation directly.

In contrast, the ANOVA technique is statistically more rigorous, but it requires more sampling, and the calculations are more complex. The advantages of the ANOVA method include being able to estimate any existing interaction between joints and operators, and it provides confidence intervals to reflect the uncertainty of the results. This study has been rigidly defined such that there is sufficient sampling to provide Agilent Technologies Overview of Gauge Tool Studies, Rev. F, April 2, 2001 Page 7 of 12 Agilent Technologies reasonably precise results, and yet permit an analysis package that can automatically perform the calculations. Both methodologies are included so that the user may seek a familiar solution, and also enjoy the benefits of the ANOVA analysis.

The two techniques will not provide identical results because methodologies differ. Our preference is the ANOVA technique because of its statistical rigor.

As the user reviews the summary pages of the analysis sheets there are some key points to check. ANOVA: . The ANOVA method for performing measurement system analysis (also called gauge R&R) fundamentally partitions the total variation in the data set into pieces called variance components (VC). The variance component associated with solder joints, joint-VC, is an estimate of the variance among the 30 joints used in the study. The operator-VC is an estimate of the variance among the 5 operators. This corresponds to the AV in the range method. The repeatability-VC is an estimate of the variation among repeated measurements by the same operator on the same joint. This corresponds to the EV in the range method. The measurement-error-VC is an estimate of the total variation in the measurement system and is the sum of the operator, repeatability, and interaction-VC’s. This corresponds to the R&R in the range method. Additionally an interaction-VC is estimated which represents any interaction between the joints and operator. . Confidence intervals are provided for each of the estimated VC’s. These interval estimates give an indication of the uncertainty associated with each estimated VC due to the selected sample sizes. The interpretation of a 90% confidence interval is that the true, unknown variance is contained within the interval with 90% likelihood. When the lower confidence limit is greater than zero, there is at least 90% confidence that the true, unknown variance is greater than zero. . Interpretation of the interaction-VC deserves additional explanation. If the lower confidence limit is greater than zero, the interaction is statistically significant at the 90% confidence level or greater. A significant interaction is often an indication of a data integrity problem and should be investigated. An interaction implies that the variation among operators is different depending on which joint is measured. Alternatively, it says that the variation among joints is different depending on which operator is doing the measurement. Neither situation is expected in this type of analysis. So, when a significant interaction exists, it can often mean a data entry error or some other anomaly in the data. When the true interaction is near zero, it is possible to estimate a negative VC due to sampling uncertainty. It is common practice to simply interpret negative VC’s as zero. . Two different metrics are provided for judging the adequacy of the measurement system: Signal-to-Noise ratio and Precision-to-Tolerance ratio. The S/N ratio is defined as the square root of the joint-VC Agilent Technologies Overview of Gauge Tool Studies, Rev. F, April 2, 2001 Page 8 of 12 Agilent Technologies

divided by the square root of the measurement-error-VC. Recall that the standard deviation is simply the square root of the variance, so the ‘Signal’ is represented by one standard deviation of the variation provided by the joints. While the ‘Noise’ is a one sigma estimate of the measurement error.

The S/N is most meaningful when the 30 joints selected for analysis are representative of all joints of a given type produced by the production process. For the 5DX to provide adequate discrimination among joints, the S/N should be greater than 5. A lower confidence limit which is greater than 5 implies at least 90% confidence that the true S/N exceeds 5.

. The other metric, P/T, is defined as 5.15 times the square root of the measurement-error-VC divided by the total tolerance width (upper limit minus lower limit). This is analogous to the P/T in the range method. An upper confidence limit less than 30% would provide at least 90% confidence that the true P/T is below 30%. . The “percent” column in the ANOVA analysis simply gives the percent associated with each VC. The joint-VC and measurement- error-VC sum to 100% so that it is obvious whether most of the variation comes from the measurement system or the joints being measured. Within the measurement system, the percentages associated with operator, repeatability, and interaction sum to 100%. This makes it apparent where most of the variation within the measurement system is coming from. The extent to which the estimated VC’s are negative can cause individual percentages to exceed 100%. . It is possible to obtain a negative estimate of any variance component which is truly at or near zero. This implies that the measurement- error-VC, previously defined as the sum of the operator, repeatability and interaction VC's, could be less than the repeatability-VC alone. While not an especially intuitive result, the ANOVA measurement- error-VC is still the "best" estimate of the total variation in the measurement system. And the resulting ANOVA estimate of the R&R remains directly comparable to the R&R from the range method which makes no attempt to estimate the interaction.

RANGE: . The Upper control limit (UCL) for range is estimated using the Grand Range, which is the average of the ranges resulting from the three trials for each sample and each operator. Subgroup ranges should generally not exceed the UCL. This is easily checked by the color of the tattletale bar…green means all subgroup ranges are below the UCL. For a range that exceeds the UCL we would expect to find an assignable cause, which should be corrected. If an assignable cause is found, and if that cause is not consistent with the normal operation of the instrument then the data may be discarded.

. The Repeatability or Equipment Variation is the estimate of the uncertainty provided by the instrument itself, independent of sample and operator. No units are provided on the spreadsheet but for solder thickness the units are mils. Notice that the EV is reported as a multiple of the standard deviation. Specifically (5.15 * sigma) is used because it represents 99% of the population. . The Reproducibility or Appraiser Variation is the uncertainty generally attributed to the operator. One must be very careful with this conclusion because, as mentioned earlier, there can be confounding effects of time, work shift, environment, or instrument duty cycle. Here again the value represents 99% of the population. . R&R or Precision is the combination of the two variation components previously identified so it is a measure of the uncertainty provided by the entire measurement process. . The last term presented on this sheet is the Precision-to-Tolerance ratio. Generally, the P/T ratio should be 30% or less to control a manufacturing process. Because the magnitude of this ratio is directly controlled by the tolerance one must be very judicious in defining the specifications.

Glossary:

Precision --The ability of an instrument to routinely produce the same result when operated by a variety of operators under consistent measurement conditions. Precision, or total Gauge R&R, in this discussion will include the repeatability component of the measurement tool as well as the reproducibility component resulting from multiple operators. Precision= GR&R = [(Repeatability^2) + (Reproducibility^2)]^1/2

Repeatability -- The ability of a measurement device to routinely produce the same result under consistent measurement conditions, by a single operator. Repeatability is commonly used interchangeable with precision but in this presentation, repeatability specifically means Equipment Variation (EV) and it does not include the reproducibility content produced by multiple operators.

Reproducibility -- That component of measurement precision variability attributable to the employment of multiple measurement tool operators in the measurement process.

Accuracy --The ability of a measurement device to provide results identical to a known, or NIST traceable, standard. For a measurement to be accurate both the measurement variability and the measurement bias must be small.

References: Automotive Industry Action Group (AIAG), Measurement System Analysis. Southfield, MI: AIAG Press. http://www.aiag.org/publications/quality/msa2.html

Burdick, Richard K., and Larsen, Greg A.” Confidence Intervals on Measures of Variability in R&R Studies.” Journal of Quality Technology, Vol. 29, No.3, July 1997.

“Measurement System Evaluation.” General Requirements for Implementation for Statistical Process Control, IPC-PC-90, October 1990, pp. 24-36. http://www.ipc.org/.

Crossley, Mark L., Statistical Quality Methods. Milwaukee, WI: American Society for Quality Press. http://www.qualman.com/

NIST, “Engineering Statistics Handbook: 2.4 Gauge R & R Studies”, http://www.nist.gov/itl/div898/handbook/mpc/section4/mpc4.htm

Acknowledgements: *Excel, and Notepad are registered applications of Microsoft Corporation. *This analysis package and accompany spreadsheet, "Full Gauge R&R Worksheet," are the property of Agilent Technologies. All rights are reserved. (March 1, 2001)

Agilent Technologies Overview of Gauge Tool Studies, Rev. F, April 2, 2001 Page 12 of 12