Challenges In Performing An Individual Participant-level Data Meta-analysis

Individual participant–data meta-analysis (IPDMA) uses data at the participant level to synthesize evidence on clinical decision-making. We discuss the differences from aggregate-data meta-analysis and the advantages and challenges of IPDMA. IPDMA provides an opportunity to study effect modiﬁers at the participant level and can reduce bias in comparison to aggregate-data meta-analysis. A challenge with IPDMA is that it often requires considerable effort to obtain all the data, and an inability to obtain data for all the studies can lead to availability bias. Patient summary: Systematic reviews of the literature are performed to summarize available evidence regarding clinical decisions. Here we explain the differences between reviews that use aggregated data from published studies and reviews that use the participant-level data from those studies. (cid:1) 2023 The Author(s). Published by Elsevier B.V. on behalf of European Association of Urology. This is an open access article under the CC BY license (http://creativecommons. org/licenses/by/4.0/).


Introduction
Traditionally, systematic reviews and meta-analyses have been used to synthesize evidence regarding clinical decision-making using aggregate data from published studies.More recently, individual participant-data metaanalysis (IPDMA) has been introduced, in which data for each participant from multiple studies are used to answer particular research questions.This approach has a number of advantages in comparison to aggregate-data metaanalysis (ADMA), but it also poses a number of challenges.
In the first phase, IPDMA follows the normal steps of a systematic review with meta-analysis.This includes a systematic literature search, selection of eligible studies, and assessment of the risk of bias and evidence certainty.Instead of data extraction from publications, applied in ADMA, original data are retrieved to perform IPDMA.This results in a data set with one line per subject, in comparison to one line per study in ADMA.Here we discuss the advantages, challenges, and methodological issues associated with IPDMA.We focus on IPDMA of the effectiveness of interventions, but IPDMA can also be applied to other domains such as the accuracy of diagnostic tests, prognostic factors, and the performance of prediction models.

Advantages of IPDMA
The main advantage of IPDMA is that it is possible to study effect modifiers (ie, predictors of a treatment effect) at the participant level.Take the example of a hypothesis that the treatment effects of a certain intervention are greater among older patients.An ADMA could compare studies involving older patients on average to studies involving younger patients on average.As these studies will have a mix of older and younger patients, this can lead to two problems:(1) there is no clear separation between old and young patients; and (2) the studies with older patients may have differences other than patient age in comparison to the studies including (on average) younger patients.By contrast, in an IPDMA, younger patients can be separated from the older patients in every study to categorize all patients in two distinct groups.Next, within-study comparisons between the old and young groups are conducted in IPDMA, with control for other possible differences between studies.In this way it is possible to avoid ecological or aggregation bias; relationships seen at the aggregated level do not always apply to the individual level (Fig. 1).In addition, an IPDMA has higher statistical power for identifying effect modifiers in comparison to individual studies.
Another advantage of IPDMA is the potential to improve the quality of the data and provide more flexibility for any analyses.With IPDMA, a uniform method of analysis can be applied to the data for all studies, using similar approaches for missing data, outcome selection, and statistical analyses.In addition, the data can be checked and unpublished data from the original studies can be included [1,2].Together, these steps may reduce bias.

Challenges of IPD
Performing an IPDMA also comes with several challenges.First, obtaining the data from the studies identified via review will require time and effort.Each study team has to be approached and asked for their data, and this may involve addressing legal issues [1].If a study team does not want to share their data, or when data are simply no longer available, this can lead to bias if any such rejection is related to the trial results (availability bias) [2].Initiatives such as ClinicalStudyDataRequest.com may help in reducing the burden of data collection for individual studies.As IPDMA and ADMA often lead to similar results and conclusions [3], researchers should have good reasons for choosing IPDMA over ADMA.Such reasons include concerns regarding the analysis in individual trials or a specific interest in effect modifiers.

Methodological issues
Pooling of data in an IPDMA can be achieved in two ways: (1) a two-step approach in which the data are first analyzed for each study separately in a uniform way and subsequently pooled; and (2) a one-step approach in which all the data are analyzed simultaneously in a single analysis while taking clustering within studies into account [4].The Preferred Reporting Items for Systematic Review and Meta-Analyses of Individual Participant Data (PRISMA-IPD) statement describes the preferred items for reporting [5].

Example: ADMA versus IPDMA
We illustrate the concepts by comparing an ADMA and a IPDMA performed by the same authors on the same topic [6,7].Both meta-analyses assessed (among others) the effect of adding docetaxel to androgen deprivation therapy on overall survival (OS) for patients with metastatic hormone-sensitive prostate cancer.The ADMA from 2016 and the IPDMA from 2023 included the same three trials and 2261 patients for this comparison.The OS estimates from the ADMA (hazard ratio [HR] 0.72, 95% confidence interval [CI] 0.61-0.80,which we recalculated) and IPDMA (HR 0.79, 95% CI 0.70-0.88)were comparable.
Both meta-analyses also aimed to assess the effectiveness in subgroups of patients by metastatic volume.In the ADMA this was not possible because ''. ..results by patient subgroup were either too sparse, or the definitions too inconsistent, to allow for meaningful analyses from the available reported data''.For the IPDMA, data were available to study these subgroup effects, which revealed that the intervention was more effective for patients with high metastatic volume (HR 0.60, 95% CI 0.53-0.69)than for those with low metastatic volume (HR 0.80, 95% CI 0.66-0.96).This clearly illustrates that an IPDMA can generate important information regarding subgroups that can be used for clinical decision-making.

Conclusions
IPDMA can help in studying the predictors of a treatment effect in subgroup analyses, but the added benefits do not always outweigh the extra effort that is needed to perform an IPDMA.It is therefore crucial to consider what benefits an IPDMA can bring in answering the research question in comparison to an ADMA before starting.
Fig. 1 -An example of ecological bias.At the study level it looks as if there is a relationship between the treatment effect and the proportion of males in the trial.However, in most trials this relationship does not exist and drawing a conclusion based on the study-level correlation is not correct and an ecological fallacy.Reproduced from Riley et al. [2].