A Critical Appraisal of Diagnostic Accuracy of Clinical Tests

A critical appraisal examines research to recommend its worthiness, validity and significance in a particular context. Research findings published in scientific journals can have a great impact on medical practice. The impact is amplified when popular press coverage of the findings gets involved. Moreover, it is evident that new discoveries are made and published in noticeable journals whereby the information is more relevant posing costly changes in the medical practice.  However, such cases have the prospective to negatively affect the quality, cost, and delivery of healthcare services, especially when the changes proposed by the findings lack significant support from the study’s data. 

On the other hand, in the majority of situations, the printed journals do not converse or recognise the flaws of the study. Thus, it forces the reader to spend much time in identifying the weaknesses. However, it has been seen creating problems to the physicians since majority of them are busy and lack time for systematic assessment of the research’s findings. The problems created by these reports on new research findings have created a need for critical appraisals of studies that are published in medical journals. The critical appraisals are instrumental in evaluating study evidence in an aim to help physicians and other professionals in healthcare to relate the findings to medical practice. This paper is a critical appraisal of a research paper aimed at assessing the diagnostic accuracy of clinical tests, for ACL injuries versus golden standard (MRI) using the PEDRO-scale.

Critical Appraisal: MRI versus Clinical Examination Test

This is a critical appraisal of the research study “Comparison of the accuracy of using MRI or clinical examination as primary diagnostic methods on ACL patients with arthroscopy as the diagnostic reference. The study aims at assessing the diagnostic accuracy of two methods used by orthopaedic surgeons to determine the presence of Anterior Cruciate Ligament (ACL) injuries. The study explores available research work on Magnetic Resonance Imaging (MRI) and clinical examination, which are the two methods of diagnosis. The objective was to determine whether the two methods were sufficient enough for surgeons to use when assessing the presence of an ACL injury before proceeding to surgical treatment.
 The process is important as it helps doctors in getting certainty that a perceived patient is eligible for surgery. The effectiveness of these methods is of huge significance, as an analysis through MRI and clinical examination can help the healthcare provider to avoid administering wrong treatment. Stopping an unnecessary arthroscopic surgery can not only save the patient from undergoing risky treatment, but also save costs associated with conducting the surgery (Kocabey et al 2004). The study focusses on the comparison of information, data, and analyses of five research papers that have already been published. The examined research papers entail studies that have been conducted with an aim of determining how accurate diagnosis through clinical examination of ACL injuries is compared to the diagnosis carried out using MRI. 

To address the validity of the study, an effective critical appraisal tool like the PEDro Scale can be used. To ensure validity, the appraisal must address several important issues. Firstly, the study has to have a clear and focused clinical research question. This involves a clear definition of the research area, the population of interest, the proposed intervention, the existing standard of care under scrutiny, and the expected outcomes. Identifying these issues enables the research study to be matched with the right patients, the proposed changes to practice are also identified and analysed for compatibility, and also the effects of the changes to both the healthcare professionals and the patients are identified (Thomas et al. 2007). In this study, the research area is a comparison of the accuracy of using clinical examination to MRI when assessing the presence of an ACL injury before proceeding to surgical treatment. The population of interest includes all patients with ACL injuries. 

Secondly, the appraisal must determine whether the implemented study design is relevant for the clinical research question. The study design varies depending on the nature of the study itself. The study being appraised in this case entails an analysis of other published research studies. A third issue that should be taken into consideration to ensure validity of the study results, is the methodological of the study, this entails discussions on various biases affecting the study, comparison of the study groups to the baseline, intent-to-treat cases, among others. A PEDro scale is an important and effective tool that can be used to determine the validity of a research study. This critical appraisal will involve the use of the PEDro scale.  

PEDro Scale Analysis

The critical analysis leverages the use of PEDro scale analysis of the research paper based on criteria. In a PEDRO scale points are awarded to a research study based on the study’s satisfaction of various criterions.

Criterion 1

The first criterion on the PEDro scale is concerned on whether the criteria for eligibility were specified during the study. The study in question does not meet or satisfy the criterion. Firstly, the study does not involve any participants as it is only an analysis of existing published research studies. Secondly, assuming the extensive literature review represents the subjects, there is no clear definition on the method used to determine which material to include in the study.

Criterion 2

Random allocation of subjects to control groups ensures the comparability of various control groups in the study. The criterion is satisfied if a random allocation procedure is used in the study. The study entails an analysis of five different published research studies, which involve different settings, different participants, and different conditions. Therefore, the study does also not conform to the second criterion in the PEDro scale. However, the paper analyses the methods used in research in all the five research studies. After rating them, the findings of the different papers are compared and then conclusions are drawn.

Criterion 3

This criterion is concerned with the allocation of subjects to a study group. The criterion is satisfied if the individual, who made the decision to include subjects for a trial, was unaware at the time of decision making. In cases where allocation is not concealed, the knowledge of an individual’s current health status could influence the decision of whether the individual is to be included in a trial or not. Since the paper is centred on the analysis of already published works, there was no allocation of subjects, let alone concealing of the allocation. However, the different studies that have been analysed in the research entail different types of allocations. For instance, the study by Rayan et al. (2009) involved patients who had been identified through some unidentified selection process as subjects. Another study by Behairy et al. (2009), involved a total of 70 patients as subjects, who had been identified through history taking, clinical examination, MRI and X-ray imaging, arthroscopy and results on accuracy recorded. In this case, the study did not satisfy the allocation criterion.

Criterion 4

The criterion is satisfied if the research study mentions one measure of the severity of the area under study, at baseline, and a measure of the effects of treatment. The outcomes of the groups in this case are not expected to differ by clinically significant values, on the basis of different baseline prognostic variables. The criterion is used to pinpoint any bias that may arise as a result of the presence of gross discrepancies among the groups at baseline. The current study entails an analysis of different cases involved in different studies. In most cases, there was a difference between the outcomes and the baseline. The differences were as a result of the different in results recorded between the use of clinical methods for examination and the MRI.

Criterion 5, 6, and 7

The criterion is concerned with the blinding of subjects, assessors, and therapists involved in the study. Blinding in this case refers to the lack of knowledge of the subject, assessor, or therapist, of the particular group the subject is allocated to. Blinding helps in eliminating placebo or Hawthorne effects; where the responses of the subjects are distorted or influenced by how they think they are expected to respond by the experimenters. When the assessors are blinded, a reader can be confident that the recorded results of treatment or diagnosis in this case are free from any biases. Blinding the therapists also helps in ensuring that the effects of the treatment or diagnosis are not influenced by the therapists’ enthusiasm levels for the exercise. This study entailed an analysis of the accuracy of methods used for diagnosis. The subjects, assessors, and therapists were blinded to ensure that there were no external influences on the observed results. The findings of these studies revealed results that were almost similar, which was an indication of the lack of biases.

Criterion 8

This criterion is concerned with the outcome of the study, which should be obtained from at least 85% of the total subjects. When conducting a study, not following up with the subjects may result in differences in terms of outcomes, which may result in bias. Not following up more subjects increases the magnitude of the bias. Satisfaction of this criterion is achieved when the research study states the actual number of subjects who participated in the study, and the actual number of participants that were considered when obtaining outcome measures. Some of the research papers analysed in the study satisfy this criterion while others don’t. For instance, the study by Rayan et al (2009) does not specify the number of subjects involved in the study. On the other hand, Behairy et al. (2009) involved a total of 70 patients as subjects in the study. The total 70 subjects were considered in obtaining the results. Also, the study by Siddiqui et al. (2013) also highlights the number of subjects involved in the study.

Criterion 9

Clinical trials mostly involve protocol violations, such as a mix up in the treatment administration to the subjects. A case where a subject does not receive treatment according to the set out plan usually amounts to violation of protocol. These violations may amount to bias that may have an impact on the results of the study. The intention-to-treat analysis refers to the situation where a subject did not receive the intervention intended for them, but their data was still analysed as if they received the intervention. The study satisfies the criterion, as all the involved research papers explicitly state that all subjects were diagnosed using the two methods under assessment.

Criterion 10

The criterion is concerned with the reporting of statistical outcomes of the groups participating in the study. The between-group statistical comparison involves a statistical contrast of two different groups. The comparison may vary depending on the study design, for instance, comparison between different treatments, diagnosis, or between a control condition and treatment. The involved analysis may entail a comparison of the results after the administration of a treatment, or a group-based change comparison. The current study under appraisal entails a comparison of the accuracy of using MRI or Clinical Examination as Primary Diagnostic methods on ACL patients with arthroscopy as the diagnostic reference. Therefore, the study satisfies the criterion.

Criterion 11

The criterion is concerned with point and variability measures of the outcomes. A point measure is used to determine the size of the effects of the treatment. The outcomes in all the groups or the differences among the group outcomes constitute the treatment effect. The measure of variability entails standard deviations, confidence intervals, standard errors, ranges, and interquartile ranges, among other applicable measures. In most cases, point and variability measures are represented through graphical methods. The study entails an analysis involving multiple sets of data. The data reveals minimal differences between the accuracy of using MRI and clinical examination. For instance, the research study by Siddiqui revealed an accuracy rate of 88.5% for the MRI, against a 96.9% accuracy rate for clinical examination. Junior, Leao and Oliveira (2015) also conducted a study that revealed 90.27% for clinical examination, against the MRI’s 83.33%. The study by Rayan et al. (2009) also showed comparison of results in percentage form.  The accuracy of clinical examination was 96%, as compared to that of MRI’s 93%. The study therefore satisfies the criterion. In summary, the study satisfies 8 out of 11 criterions in the PEDro scale. This means that the study gets a score of 8 out of 11. 

Discussion

The study aimed at assessing the diagnostic accuracy of two methods used by orthopaedic surgeons to determine the presence of Anterior Cruciate Ligament (ACL) Injuries in patients. The assessment has been carried out through the analysis of already published research papers addressing the topic. According to study findings, Clinical examination seems to be more accurate than using MRI with a small difference (Ben-Galim 2006). This makes the two methods suitable for use during primary diagnosis of ACL injuries. The author has covered all the analysed research studies efficiently, detailing the methods used by the researchers in analysis of the accuracy. The final part of the paper is the “limitations of the study” section, which highlights the main challenges affecting the study. A healthcare professional willing to adopt the research in practice will be fully informed.

References

Ben-Galim, P., Steinberg, E. L., Amir, H., Ash, N., Dekel, S., & Arbel, R. (2006). Accuracy of magnetic resonance imaging of the knee and unjustified surgery. Clinical orthopaedics and related research, 447, 100-104.

Kocabey, Y., Tetik, O., Isbell, W. M., Atay, Ö. A., & Johnson, D. L. (2004). The value of clinical examination versus magnetic resonance imaging in the diagnosis of meniscal tears and anterior cruciate ligament rupture. Arthroscopy: The Journal of Arthroscopic & Related Surgery, 20(7), 696-700. 

Rayan, F., Bhonsle, S., & Shukla, D. D. (2009). Clinical, MRI, and arthroscopic correlation in meniscal and anterior cruciate ligament injuries. International orthopaedics, 33(1), 129-132.

Siddiqui, M. A., Ahmad, I., Sabir, A. B., Ullah, E., Rizvi, S. A., & Rizvi, S. W. (2013). Clinical examination vs. MRI: evaluation of diagnostic accuracy in detecting ACL and meniscal injuries in comparison to arthroscopy. Polish orthopedics and traumatology, 78, 59-63.

Thomas, S., Pullagura, M., Robinson, E., Cohen, A., & Banaszkiewicz, P. (2007). The value of magnetic resonance imaging in our current management of ACL and meniscal injuries. Knee Surgery, Sports Traumatology, Arthroscopy, 15(5), 533-536.