Reporting and Analysis of Results from Rechecking of Smears for AFB

Reporting and analysis of AFB-smear laboratories performance

The following documents are used for this purpose: - Quarterly Report “AFB laboratory performance and stock of consumables”: for reporting by the peripheral laboratories (file: microscreport.xls) - Quarterly Report “AFB laboratory performance”: for the intermediary level coordinator, to report consolidated performance of the peripheral laboratories to higher up (i.e. national) (file: AFBLABform4RegionalReport.xls) - Form 1 “Blinded Rechecking of Sputum Smear Examinations for Acid-Fast Bacilli”: for the visiting supervisor, to register slides sampled for rechecking, and can also be used for feedback on the same (EQAform1_sampling.xls) - Form 2 “Rechecking of Sputum Smears for AFB, List of Discordants”: for use by the rechecking coordinator at intermediary level, to list discordant smears and their results to be sent to the second controller + for second controller’s results (EQAform2_discordants.xls) - Form 3 “Rechecking of Sputum Smears for AFB, Quarterly / Annual Report Form”: for use by the rechecking coordinator at intermediary level, to compile manually the rechecking data and forward these to higher up (i.e. national) (EQAform3_reporting.xls) - Workbook “Evaluation of AFB-microscopy laboratories”. There are two versions of this book: a smaller one for use at national in smaller countries, or at intermediary level in larger countries (EQA&PerformanceLabsMaster.xls). It can hold the names of max. 250 laboratories and 31 administrative divisions one level below the one used for the full workbook (i.e. the smaller country). The larger one is laid out in the same way, but it can be used for a complete larger country, since it can hold up to 1500 laboratories and 3 administrative divisions below country level (with max. 50 names for the first level, 100 for the second and 500 for the third) (CountryEQA&PerformanceLabsMaster.xls). These workbooks are used to compile and analyse data received on performance of the labs and on rechecking results (from Form 3 and from the “AFB laboratory performance” reports combined).

How to use the performance documents? The Quarterly Report “AFB laboratory performance and stock of consumables” is in principle filled quarterly by each peripheral laboratory and sent to the district / region / province. It can also be filled by a visiting supervisor with help from the laboratory staff. Laboratory staff should at least themselves count numbers of smears, positives… each month, and their remaining stock quarterly. The higher level will use the information on smears performed to compile a consolidated performance report for the district/ region / province… for reporting to the higher level. For this purpose the individual reports are either summarized on the quarterly report form “AFB laboratory performance”, either directly in the Workbook. The following information needs to be filled: - name and sequence number of the laboratory - technique used: Ziehl-Neelsen or Fluorescence Microscopy - totals of AFB smears examined during the quarter / year: from the performance reports of the peripheral labs, totals of positive, scanty 1-9 and negative smears reported, by type of examination (diagnostic or follow-up) - stocks left for the main products The stock situation is not reported to higher up, but used to estimate the needs of essential supplies at the intermediary level, for automatic supply to the peripheral laboratories. For this purpose a simple spreadsheet should be used. These stocks are NOT used to calculate the supplies that need to be ordered from higher up, but the number of smears examined and the number of functional laboratories will be needed for this purpose.

How to use the rechecking documents? Forms: - Form 1 is used by the supervisor taking the sample to list the slides selected and their results on one copy (kept by the EQA coordinator); only slide numbers are listed on another copy (goes to the first controller together with the slides) - the coordinator will fill the column of missing results on both copies, upon receipt of the second copy with first controller results - the coordinator then identifies discordant results, and lists them on Form 2 together with both results (peripheral and first controller); discordants from several peripheral microscopy labs can be filled on the same form, identifying also the laboratory. It is imperative that the columns will be switched regularly, i.e. for the first peripheral laboratory results using column 1 (and column 2 for the first controller results), but listing results of the second peripheral laboratory in column 2 (and first controller results in column 1), etc.. - Form 2 goes with these discordant slides to the second controller, who will add her/his results in the third column, and return the form to the coordinator - the coordinator then adds the second controller results to both copies of Form 1, and fills the boxes below the slide listings: o numbers of slides checked by result, for both peripheral lab and first controller: these numbers are obtained by counts made in the respective columns (“Peripheral lab result” ; “Result first control”), and will regularly be slightly different between peripheral lab and first controller, but their totals should be the same. o numbers of errors made for both peripheral laboratory and first controller in the boxes below the totals rechecked (classified in various ways). - Form 1 is then completed by adding recommendations at the bottom, depending on the findings (errors; qualitative remarks on smear and staining quality) - one copy of Form 1 is used to give feedback to the peripheral laboratory, the second one is filed by the coordinator and will be used to compile the quarterly report (Form 3) - Form 3 compiles rechecking results for an area (district, region, province..), for all the laboratories controlled. Only the summary counts are listed, not individual slides. Form 3 can also be omitted, filling the Workbook directly from Form 1. The workbook is then used to report to the higher level. The following information is entered on Form 3: o names of the peripheral laboratories and a sequential number (left-hand columns) o peripheral labs: totals of positive, scanty 1-9 and negative smears rechecked (from Form 1, totals by result for the peripheral lab) and errors found (from Form 1, summary of errors detected for the peripheral lab) o names or codes of the first controllers, by lab rechecked o first controller(s) summary counts per lab rechecked: totals of positive, scanty 1-9 and negative results reported for smears rechecked and errors committed by the first controller (from Form 1, as for the peripheral labs) o totals are filled at the bottom by adding up each of the columns Form 3 is then sent to the higher level quarterly (an annual report is optional, only required if the compilation / analysis workbook is not used).

Compilation and analysis workbook: This workbook is for use at national and eventually also at intermediate level. It will automatically make the additions quarterly and annually, by peripheral microscopy laboratory, by one or more higher administrative division(s), and by first controller. It will also automatically analyse the results, provided that all information needed is entered, including the performance of each laboratory as numbers of smears examined. The analysis of the performance calculates total workload, besides prevalence of positives among diagnostic versus follow-up smears and the total prevalence for positives or scanty positive smears. These are all indicators which should be checked when probing for problems. The rechecking results analysis takes into account differences in positives prevalence between laboratories, and between the samples presented to the first controllers and the routine slides. This is done by calculating the percentage of true positive / scanty detected by the controllers (= the relative sensitivity of their work), overall and by controller; the quality of the work of each peripheral lab is compared to that of the controllers calculating a ratio, which shows the detection by the peripheral lab compared to the average for the controller. If this value is about 1.00 (or 100%, the format can be selected), the lab detects positives equally well as the controllers; a value clearly below 1.00 means that the lab is less good at detecting true positives, while clearly above 1.00 it is better than the controllers. Besides, the HFP detected are taken into account by showing the percent true positives out of all positives, so the accuracy of a positive result. Scanty results are not included here, since they cannot always be confirmed by rechecking. However, this calculation will only appear if at least 5 positives were rechecked already, since otherwise it would be highly inaccurate. The workbook contains several sheets, which can be assessed through links in the opening Menu sheet. They are also labeled at the bottom.

Menu display for large version

Menu display for smaller version Besides the Menu sheet, these are from left to right:

- Set up (the larger version example is shown): here the designation and name of the administrative reporting unit (smaller version) or that of the country plus the designation of up to three administrative divisions have to be entered on top (once for the whole book and year). Lower down, fill also the year. In the columns at the bottom, the names of the administrative divisions (one level in the smaller version, up to three in the large version), and the names of the different laboratories covered in these divisions should be entered, also only once. They will then appear in the correct places throughout the workbook sheets, wherein they cannot be changed nor added Set up (with smaller version example) There is also a list of codes of first controllers, which should be replaced by the real code or name. These will then be available as a in the appropriate place when filling the quarterly results of rechecking (Q1 to Q4). Only as many names or controller codes as participated should be specified, one below the other, and the remaining pre-entered codes should be left intact for the list to function correctly. In this SETUP, the controllers’ names or codes DO NOT have to correspond to the laboratories / administrative areas they work for, list them all on top.

drop-down list Laboratories and the administrative divisions to which they belong can be entered in any sequence. So, new laboratories that are added later during the year should be added in the first empty line. DO NOT ATTEMPT TO INSERT OR SORT this table, it will result in corrupting the workbook entirely, and it is unnecessary. Q1 to Q4 (with larger version example) These sheets will be filled with quarterly results of rechecking performed. The columns on the left under “Evaluation of the microscopy labs” automatically show the names of the laboratories covered in the report, and their administrative sub-unit (all the different ones specified in the set- up for the larger workbook version). In the columns to the right the numbers of smears rechecked (by results) and the numbers or errors must be filled, taking them from the quarterly report Form 3 (or directly from Form 1). The same is done for the first controllers, in the right part. Note: only fill NUMBERS in those boxes, nothing else, otherwise the calculations will not work!! The identification code or name of the first controllers must be filled in each sheet and on each line, clicking on the arrow of the box and choosing the correct one from the drop-down list that will appear.

- Q1P to Q4P (with larger version example) These sheets will be filled from quarterly performance (“output”) reports of the laboratories (individual or already consolidated). The columns on the left automatically show the names and administrative sub-units of the laboratories covered in the report. In the columns to the right the numbers of smears routinely read (by result and by type, diagnostic or follow-up) should be entered. Totals are NOT entered, they will be added up in the annual sheet by the workbook. Note: only fill NUMBERS in those boxes, nothing else, otherwise the calculations will not work!! - YR EQA (larger version example) It shows the totals of the quarterly rechecking data already entered. Nothing in this sheet needs to be filled, it will automatically add up the quarters. - YR P (larger version example) It shows the totals and analysis of the routine lab performance data entered in Q1P to Q4P, as far as filled in. The part on the right is an analysis, showing totals and percentage of positive / scanty results for diagnostic and follow-up smears separately. Nothing in this sheet needs to be filled, it will automatically add up the quarters. - YR CONTR (any version)

It shows the totals of the quarterly rechecking data already entered, right part on first controller performance. Each controller is shown individually, adding up all rechecking results as far as entered for the year. Percentage true positive detected (= relative sensitivity) and percentage true positive / all positives analyses their performance in rechecking. Nothing in this sheet needs to be filled, it will automatically add up from the quarters, but newly added data will be calculated only after saving / closing and reopening the workbook. Analysis_Labs sheet (larger version example)

Also here nothing has to be filled. The sheet shows analysis of routine AFB-microscopy as well as the findings from rechecking by peripheral laboratory, for all quarters filled together.

To the right of the name appear the total number, percent positive and percent scanty results for “Smears examined during the year”, this time without distinction “diagnostic” versus “follow- up”. The next columns show the totals of smears rechecked, by result. The right hand part shows selected results of rechecking: number of HFP, HFN and LFN, besides their analysis (“detection proportional to the controllers”; “percent true positives out of all positives”). In the top right corner appears one average estimate of controllers’ performance (“Percentage of true positive / scanty overall detected by the controllers”). These analysis columns have been given colour codes to automatically highlight unsatisfactory performance. Different levels have a different colour, which can be changed via the “Format” menu, “Conditional formatting”. Changing the conditional formatting These columns also have autofilters as shown by the downward arrows in their heading boxes. Clicking one of these shows the list of values in that column, besides a “Custom” option. Clicking this option brings up filtering criteria boxes, which allow obtaining a list of labs with for instance less than 0.75 detection relative to the controllers. This list can then be printed or copied to another location. To undo the filter click again on the arrow, and select “all”. - Summary_Labs sheet: Nothing needs to be filled. The sheet automatically recapitulates a summary of performance and quality of all laboratories, as far as they were included in the performance and rechecking reports. This includes a classification of laboratories based on quality of positive (“true positives”) and negative (“detection”) results. - District_totals:

This sheet shows totals for the administrative subunits when these were specified. In the title, the designation of the division is automatically adapted to how it is called in the set-up sheet, but the tab of this sheet remains “district”. Changing this tab name is possible but will disturb the hyperlink from the menu. This sheet adds up automatically and cumulatively during the year, and is automatically refreshed each time the workbook is opened. It will not be updated during data entry though, until the workbook is saved, closed and opened again. It contains the same information as the summary report for the first controllers. In the large version, the sheet “District totals” is now called “Admin_Div_III_Totals”, and there are two more of these sheets to make the summaries by higher administrative division, “Admin_Div_II_Totals” and “Admin_Div_I_Totals”. They function in the same way, updates are made only after saving, closing and reopening the workbook. “ Calculations”, “Pivot_regions” and “Pivot_contr” There are three more hidden sheets: “Calculations”, “Pivot_regions” and “Pivot_contr”. These sheets contain intermediary calculations needed for some of the analyses. The two last ones cannot be protected for automatic functioning and are complicated, so they would better never be touched and remain hidden.

How to unhide: All sheets can be printed, or their results copied, for summary reporting to the higher level, or for feedback to the lower level, depending on where they are used.

The large workbook can be used to compile a national report, filling it directly manually, or by copying and pasting from several smaller workbooks filled at intermediate level and send to national. This is done always by using “PASTE SPECIAL” from the EDIT menu, not a simple PASTE, to avoid disturbing formats etc..

Clicking PASTE SPECIAL after selecting, copying and placing the cursor in the left hand uppermost cell will bring up a sub-menu, from where “Values” must be chosen. If done this way, first copy the set-up sheet with names of lowest administrative division, names of labs. Then add the name(s) of the second level administrative division in the column to the left of the lowest division names, and copy downwards as far as there are lab names. Do the same for the names of the highest level divisions. Finally, copy the names of the controllers in the right-hand column. Do the same for each of the smaller workbooks comprising an administrative division, each time starting on the first empty line (note: so the controllers should not follow the lines of the labs but be all together, not leaving any codes or spaces between them). After copying the names to the set-up sheet, they will appear in all data sheets. Now copy the data to these sheets quarter by quarter, making sure to select only the area containing numbers and controller names (all at once, comprising peripheral lab numbers, controller names and controller numbers from left to right and all labs from top to bottom); make sure the paste special is done also in the correct place, just to the right of the name of the first lab, and check that after the paste the correct area is covered (grey). Do the same sheet after sheet for the Q1 to Q4 and Q1P to Q4P. And repeat this for workbook after workbook, making sure to paste in the correct sheet and on the correct line.

Protection of the workbook.

All sheets except those mentioned as such are protected, but there is no password. Formatting remains possible. It is strongly recommended not to remove the protection, since this would allow erasing formulas by accident during data entry, which is impossible with the protection on. Once proper functioning is assured, it is essential for the EQA coordinator or data specialist to add a password to the protection, which must remain unknown to those entering the data. IF THIS IS NOT DONE, IT WILL NOT TAKE LONG BEFORE THESE WORKBOOKS ARE CORRUPTED inadvertently by the users. The sheets are very large, because it is very difficult to add more lines without disturbing the formulas and calculations. If less lines are needed these should simply be hidden (i.e. all the empty lines between the last lab and the totals at the bottom of each sheet). To do this, select these lines, click the right mouse button, and click “hide”. They can easily be unhidden again selecting the lines above and beneath and clicking the right mouse button, “unhide”. NEVER DELETE LINES THAT ARE NOT NEEDED, which should be impossible anyway if the protection is left on.