Meta-Test version 0.6 April 30, 1997 Copyright 1992-1997 Joseph Lau, MD This is the first "beta" release version of Meta-Test, a program to conduct statistical analysis of diagnostic test data. This program shares similar interface and file format as the Meta-Analyst family of meta-analysis software. This program is written in Turbo Pascal version 5.5/7.0 and runs under DOS. This program is also available for download via the following FTP site: healthinfo.fmc.flinders.edu.au login: cochrane password: australia and look for in the accuracy.tst directory for the files: mt.exe mt-intro.txt Statistical Methods: The 95% CI of the sensitivity and specificity for individual studies are calculated according to Diamond's formula, this formula provides a very good approximation to the exact method. Pooling of sensitivity and specificity are performed independent of each other. The logits of the sensitivity of specificity values are taken and pooled. A random effects model (DerSimonian & Laird) was used for pooling weighted by the inverse of the variance. The pooled logit sensitivity or specificity are then re-transformed back to the standard representation. Known limitations: The 95% CI for the weighted regression parameters have not yet been incorporated into the results. A 95% CI (band) around the regression lines will be incorporated in the future. The results has been compared against a SAS implementation on several data set and show similar results. An occasional third or fourth decimal difference occurs, this may be due to different methods of calculation. Summary ROC are performed as per Lincoln Mosses's paper. Areas under the SROC curves are calculated for both directly under the curve where there are data as well as extrapolated curve from false positive rates from 0 to 1 (actually 0.0000000001 to 0.999999999 due to divide by zero problem at the extremes). The Simpson's method of numerical integration method is applied using an error tolerance of 10E-6. Using the Program: The interface of the program is very similar to Meta-Analyst. At this time, most of the interactions occur with the SROC graph where many options are available. Printing of the graph is possible with HP Lasejet (standard PCL mode), dot matrix (Epson compatible) and PostScript capable printers. Color printing is not implemented. Graphics: There are currently 3 graphical display of the data, independent sensitivity and specificity, logits of TPR and FPR and their sums and differences, and the SROC graph. Features of the SROC graph: A yellow shaded region of the SROC graph marks the zone of 95% CI of the pooled sensitivity and 95% CI of pooled specificity. A 'X' marked the exact estimate. This zone could be turned on and off with the key 'Z' for zone. The dot/patch size of the study on the graph is proportional to the square root of the size in the disease and non-disease arms of the study. You could use the 'S' key to control which symbol (fixed sized dot, variable sized dot, cross, none) to use and use the '+' or '-' keys to control the size of the variable dot. The 'E' key extrapolates the curve and display of areas under the ROC curve. It toggles between "un-extrapolated" and "extrapolated". Other option keys are: 'C' - toggles between color and black-only lines 'D' - returns all options back to default display 'G' - toggles grid on-off 'R' - which regression line(s) to display, weighted, unweighted, both, none 'I' - 95% confidence intervals of the studies, sensitivity, specificity, both, or none 'P' - screen dumps the graph as it appears (HP Laserjet - PCL mode only currently) 'L' - pages through information on the right of graph 'M' - rotates mapping of 1 of 6 covariate using colors '1', '2', '3', '4', '5' - selectively turn each category in the selected covariate on and off. 'A' - turns ALL categories on/off Mapping of Covariates to the studies displayed on the graph: This is a new feature to be developed more. Currently, 6 covariates are available: Publication year (this could be used as a interger variable as well), prevalence of disease, 2 real number covariates #1 and #2, 2 categorical covariates #3 and #4. When fully implemented, up to five different colors can be mapped to each covariate. This is accomplished either automatically determining the lowest value and the highest value and dividing into five categories. The color blue is then mapped to category 1, red is mapped to category 2, white mapped to 3, yellow mapped to 4, and green mapped to 5. In the future, the user will have control of the number of categories to create and the mapping range. Mapping is available to all three graph display methods. PRINTING GRAPHS Graph output to Dot-Matrix and LaserJet (or HP compatible) is a screen dump (lower quality), Postscript output is of publication quality and could be customized (for those familiar with the PostScript language). Steps to print graph: 0. You must analyze the data before viewing and printing graphs. 1. from the main menu select the output device: "OUTPUT" item in the menu. 2. from the OUTPUT submenu, select PRINTER from the choices of: SCREEN, PRINTER, DISKFILE 3. from the PRINTER sub-submenu, select either Dot-Matrix, PostScript, or LaserJet 4. you could ignore the next question about text size, just hit . 5. Goto the graph you would like to print (F10), configure the desired output using the various options, strike the key < P > when you are ready to print. 6. Be sure to turn off printing after you are done to avoid inadvertent printing of many pages of calculations that you may not want. < F8 > key from the main menu allows you to toggle back and forth without going through all the questions. If you do not have PostScript printer, I suggest that you download the freeware "GSView" from: http://www.cs.wisc.edu/~ghost/ GSView allows you to visualize the encapsulated PostScript (EPS) file on screen and sending the graph to a non-PostScript printer. I recommend that you save the graphs as Postcript diskfile and use GSView to view and print. Some References: Diamond GA. Limited assurance. Am J Cardiol 1989; 63:99-100. (approximation of 95% CI for proportions) Hasselblad V, Hedges LV. Meta-analysis of screening and diagnostic tests. Psychological Bulletin 1995; 117:167-178. Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC surve: Data-analytic approaches and some additional considerations. Stat Med 1993;12:1293-1316. Irwig L, Tosteson A, Gatsonis C, Lau J, Colditz G, Chalmers TC, Mosteller F. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med 1994; 120:667-76. Alan R. Miller. Borland Pascal Programs for Scientists & Engineer. Sybex Inc, Alameda, 1993. (Algorithm for Simpson method of integration) Acknowledgement: It is not necessary for you to acknowledge the use of my software in your work. But if you wish to or it is required for the journal you are submitting to, then: Lau J. Meta-Test version 0.6. New England Medical Center, Boston, 1997. As a first release, I guaranty that there will be many bugs. Please send comments and bug reports to: Joseph Lau, MD Division of Clinical Care Research New England Medical Center 750 Washington Street, Box 63 Boston, MA 02111 USA voice: (617) 636-7670 fax : (617) 636-8023 email: joseph.lau@es.nemc.org