Analysis of rare binary events is an important problem for biomedical researchers. Due to the sparsity of events in such problems, meta-analysis that integrates information across multiple studies can be applied to increase the efficiency of statistical inference. Although it is critical to examine whether the effect sizes are homogeneous across all studies, a comprehensive review of homogeneity tests has been lacking, and in particular, no attention has been paid to infrequent dichotomous outcomes. We systematically review statistical methods for homogeneity testing. By conducting an extensive simulation analysis and two case studies, we examine the performance of 30 tests in meta-analysis of rare binary outcomes. When using log-odds ratio as the association measure, our simulation results suggest that there is no uniform winner. However, we recommend the test proposed by Kulinskaya and Dollinger (BMC Med Res Methodol, 2015, 15), which uses a gamma distribution to approximate the null distribution, for its generally good performance; for very rare events coupled with small within-study sample sizes, in addition to the Kulinskaya-Dollinger test, we further recommend the conditional score test based on the random-effects hypergeometric model proposed by Liang and Self (Biometrika, 1985, 72:353-358). One should be cautious about the use of the Wald tests, the Lipsitz tests (Biometrics, 1998, 54:148-160), and tests proposed by Bhaumik et al (J Am Stat Assoc, 2012, 107:555-567).

A comparison of hypothesis tests for homogeneity in meta-analysis with focus on rare binary events.

Irritable bowel syndrome (IBS) is a functional bowel disease characterized by abdominal pain or discomfort associated with altered bowel habits. Several clinical studies have demonstrated the effectiveness of acupuncture and moxibustion for IBS. Many systematic reviews of acupuncture and moxibustion for IBS have been published in recent years, but their results are not entirely consistent.
To evaluate the methodological, reporting, and evidence quality of systematic reviews of acupuncture and moxibustion for IBS.
Systematic reviews of acupuncture and moxibustion for IBS published before February 20, 2023 were searched in eight databases: PubMed, Embase, Cochrane Library, Web of Science, China National Knowledge Infrastructure, Wanfang Data, VIP Database for Chinese Technical Periodicals, and China Biology Medicine. The keywords used for literature search were acupuncture, moxibustion, systematic review, meta-analysis, and irritable bowel syndrome.
Systematic reviews and meta-analyses of randomized controlled trials of acupuncture and moxibustion for IBS were included.
Relevant information was independently extracted by two investigators. The A MeaSurement Tool to Assess systematic Reviews 2 (AMSTAR 2), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020), and Grading of Recommendations Assessment, Development and Evaluation (GRADE) were used to evaluate the methodological quality, reporting quality and evidence quality, respectively.
A total of 342 studies were retrieved and 15 systematic reviews were included. The results of AMSTAR 2 showed low methodological quality in 2 studies and very low methodological quality in the remaining 13 studies, with main issues being failure to register a protocol, incomplete search strategy, not providing a list of excluded studies, incomplete consideration of the risk of bias in the included studies, and a failure to assess the publication bias. The results of PRISMA 2020 showed seriously deficient reporting quality of 2 studies, somewhat deficient reporting quality of 12 studies, and relatively complete reporting quality of 1 study, with the main problems being lack of a complete search strategy, non-availability of a list of excluded studies with justification for their exclusion, not conducting heterogeneity and sensitivity analyses, not evaluating the credibility of the evidence, and not registering the protocol. The results of GRADE showed that the quality of the evidence is low or very low.
Most included systematic reviews interpreted findings to suggest that acupuncture and moxibustion have benefits for IBS. However, there is a need to improve the methodological, reporting and evidence quality of the systematic reviews. Larger, multicenter, rigorously designed randomized controlled trials and high-quality systematic reviews are required to obtain more robust evidence.
Ma YY, Hao Z, Chen ZY, Shen YX, Liu HR, Wu HG, Bao CH. Acupuncture and moxibustion for irritable bowel syndrome: An umbrella systematic review. J Integr Med. 2024; 22(1): 22-31.

Papers