The broad objective of this systematic review with meta-analysis of individual participant data is to compare the accuracy of plasma chitotriosidase activity and CCL18 for assessing non-neuronopathic Gaucher disease (type I) severity. The specific aims are:
1. To compare CCL18 activity level according to haematological parameters (haemoglobin concentration and platelet count), organ (spleen and liver) volumes, and symptomatic bone events reported at baseline and follow-up visits, respectively.
2. To compare plasma chitotriosidase activity according to haematological parameters (haemoglobin concentration and platelet count), organ (spleen and liver) volumes, and symptomatic bone events reported at baseline and follow-up visits, respectively.
3. To compare the accuracy of CCL18 relative to plasma chitotriosidase activiy for discriminating moderate and/or severe haematological (anemia and thrombopenia), visceral (splenomegaly and hepatomegaly), and skeletal (with X-ray confirmation) disease severity at baseline and follow-up visits, respectively.
4. To investigate demographics and clinical characteristics that may explain heterogeneity in accuracy for plasma chitotriosidase activity and CCL18 level.
5. To investigate study methodological characteristics that may explain heterogeneity in accuracy for plasma chitotriosidase activity and CCL18 level.
The MEDLINE search strategy will be developed by a member from the project team and critically reviewed by a health sciences librarian. The search concepts will include plasma chitotriosidase activity, CCL18, biological markers, enzyme replacement /substrate reduction therapy, and Gaucher disease.
We will use both standardized medical subject heading (MeSH) and text words. No type of document restrictions will be applied and no methodology filters will be used. The search will be limited to Humans. After the MEDLINE search strategy is finalized, adjustments will be made to account for differences in syntax and subject headings (EMTREE) across electronic bibliographic databases.
Studies will be identified by searching MEDLINE via PubMed (1995 onwards), EMBASE via Ovid (1995 onwards), and Cochrane Central Register of Controlled Trials (CENTRAL) via Wiley interface (current issue).
We will supplement the electronic search by scanning the reference lists of retrieved original articles and previously published review articles for additional studies. Additionally, we will contact research groups, authors of relevant articles, and prominent clinicians within the field in order to identify ongoing or completed but not yet published relevant studies.
Articles published in English, Russian, or French languages will be included. A list of potentially relevant articles published in other languages will be provided as an appendix.
Additional details of the search strategy can be found in the attached PDF document.
Eligible primary studies include cross-sectional and (prospective) cohort studies measuring both plasma chitotriosidase activity and serum CCL18 level at baseline and/or at follow up. Because randomized controlled trials evaluating enzyme replacement or substrate reduction therapy are a special case of prospective cohort studies, they will be also considered for inclusion. Yet, diagnostic accuracy studies or retrospective case-control studies evaluating the performance of chitotriosidase activity and/or CCL18 level in discriminating patients with Gaucher disease and healthy control subjects as well as case reports are not within the scope of this review.
Condition or domain being studied
Gaucher disease is the most frequent recessively inherited lysosomal storage disorder, with an estimated prevalence of one in 40,000 to 60,000 individuals in the general population. Gaucher disease is caused by the accumulation of glucosylceramide (glucocerebroside) in tissue macrophages resulting from deficiency in lysosomal acid ß-glucosidase (glucocerebrosidase) activity. Visceral manifestations observed in patients with Gaucher disease include splenomegaly, hepatomegaly, anaemia, thrombocytopenia, leukopenia, skeletal disease and, in more severe cases, pulmonary manifestations.
Gaucher disease type I, which accounts for 90% of all patients with Gaucher disease, is characterized by the absence of early neurological manifestation onset in contrast to the much less frequent acute neuronopathic (Gaucher disease type II) and chronic neuronopathic (Gaucher disease type III) variants.
Established in 1991, the International Collaborative Gaucher Group (ICGG) registry, has contributed to understanding the physiopathology of Gaucher disease, elucidating the variability in severity and natural course of this disease, optimizing patient management through the provision of guidelines for clinical monitoring, and evaluating the effectiveness of treatments. Since 1992, enzyme replacement therapy has become the standard of care for the treatment of patients with severe Gaucher disease. Enzyme replacement therapy, and to a lesser extent substrate reduction therapy, have proved to be safe and effective in reducing hepatosplenomegaly and improving haematological parameters. Quantitative measurement of surrogate biochemical markers may be useful for assisting clinicians in their decisions on initiation or adjustment of enzyme replacement and substrate reduction therapy, in patients with Gaucher disease. Current guidelines recommend measuring plasma chitotriosidase activity for monitoring Gaucher disease in conjunction with clinical assessments. Yet, analysis of plasma chitotriosidase activity is not standardized across laboratories, is irrelevant for 6% of Gaucher patients who are homozygous for a 24-base pair duplication in the chitotriosidase gene and are deficient in chitotriosidase activity, and may yield impaired enzyme activity due to polymorphism. Chitotriosidase activity may also be elevated in several diseases, including Niemann-Pick disease.
In contrast, measuring serum CCL18 level, a CC chemokine produced by macrophages in response to “alternative activation”, is amenable to standardization across laboratories through enzyme-linked immunosorbent assay (ELISA) technology and is suitable for monitoring Gaucher disease activity in patients who are deficient in chitotriosidase activity. Because serum CCL18 level is elevated in other conditions, its measurement cannot be advocated for primary diagnosis of Gaucher disease.
Plasma chitotriosidase activity and serum CCL18 level have been equivalently used for monitoring Gaucher disease in conjunction with clinical assessments over the past decade. Yet, large-scale direct comparison of these two biochemical markers is currently lacking.
Primary studies enrolling consecutive children or adult patients with objective confirmation of Gaucher disease will be included. We will focus on non-neuronopathic (type I) Gaucher disease because this is the most prevalent phenotype with visceral manifestations being improved by enzyme replacement or substrate therapy. Naïve participants (i.e., without a history of treatment) as well as those receiving symptomatic treatment or undergoing enzyme replacement therapy (ERT) will be considered for inclusion. Deficiency in chitotriosidase activity will not be an exclusion criterion. Studies with fewer than 10 participants will be excluded from the review.
Relevant index methods for quantitative measurement of serum CCL18 level include (commercially available) enzyme-linked immunosorbent assay (ELISA) and dissociation enhanced lanthanide fluoroimmunoassay (DELFIA). Studies using enhanced laser distortion/ionization time of flight (SELDI-TOF) mass spectrometric analyses will not be eligible because this technology underestimates CCL18 levels.
The comparator is quantitative measurement of plasma chitiotriosidase activity with the use of fluorogenic substrate molecules, including 4-methyllumbelliferyl-chitobiose, 4-methyllumbelliferyl-chitotriose, or 4-methyllumbelliferyl-deoxy-chitotrioside.
There will be no restrictions by type of setting.
Prespecified primary surrogate and clinical outcomes reflecting disease severity will be considered, including anemia and thrombocytopenia. The outcomes of interest are derived from haemoglobin concentration and platelet count. Anemia (i.e., mild to severe) will be defined as a haemoglobin concentration <11.0 g/dL for patients 6 to 59 months of age, <11.5 g/dL for patients 5 to 11 years of age, <12.0 g/dL for patients 12 to 14 years of age, <12.0 g/dL for (nonpregnant) female patients of 15 years of age or above, and <13.0 for male patients of 15 years of age or above. Moderate or severe anemia will be defined as a haemoglobin concentration <10.0 g/dL for patients 6 to 59 months of age, <11.0g/dL for patients 5 to 11 years of age, <11.0 g/dL for patients 12 to 14 years of age, <11.0 g/dL for (nonpregnant) female patients of 15 years of age or above, and <11.0 for male patients of 15 years of age or above. Severe anemia will be defined as a haemoglobin concentration <7.0 g/dL for patients 6 to 59 months of age, <8.0g/dL for patients 5 to 11 years of age, <8.0 g/dL for patients 12 to 14 years of age, <8.0 g/dL for (nonpregnant) female patients of 15 years of age or above, and <8.0 for male patients of 15 years of age or above.
Based on treatment goals in patients with Gaucher disease, thrombocytopenia (i.e., moderate to severe) will be defined as a platelet count <120 x 109/L. Severe thrombocytopenia will be defined as a platelet count <60 x 109/L. In exploratory analyses, other usual thresholds platelet counts will be used for defining thrombocytopenia (<100 x 109/L, <80 x 109/L and <50 x 109/L, respectively).
Studies providing baseline and/or follow up data for serum CCL18 level, plasma chitotriosidase activity, and one or more disease severity outcomes will be included. Because patients were enrolled at varying time from disease onset across studies, we will collect the age at Gaucher disease diagnosis and at enrolment. We anticipate that length of follow-up ranges between 6 and 12 months for most randomized controlled trials. However, longer follow-up times are expected for patients enrolled in observational prospective cohort studies or ongoing registries.
Prespecified secondary surrogate and clinical outcomes reflecting disease severity will include splenomegaly, hepatomegaly, and symptomatic bone events (avascular osteonecrosis and fracture, with imaging confirmation).
The outcomes of interest are derived from liver volume, spleen volume, and symptomatic bone events with imaging confirmation.
We anticipate that organ volumetric data will be derived by computed tomography, magnetic resonance, or ultrasound imaging, depending on the technology available. Organ volumes will be expressed as multiples of normal (MN) adjusted for body weight. The normal spleen volume will be computed as 2 mL/kg multiplied by weight in kg in patient who did not undergo any splenectomy procedure. Splenomegaly will be defined as spleen volume >5 MN while massive splenomegaly will be defined as spleen volume >15 MN. The normal liver volume will be computed as 25 mL/kg multiplied by weight in kg. Hepatomegaly will be defined as liver volume >1.25 MN while massive hepatomegaly will be defined as liver volume >2.5 MN.
Bone events include symptomatic avascular osteonecrosis and fracture, with imaging confirmation. Anticipated skeletal sites of fractures include spine, lower extremities (hip, femur, distal to the knee, and knee), upper extremities (shoulder, clavicle, humerus, elbow, distal to the elbow), and ribs. Bone crisis and chronic bone or joint pain without imaging confirmation will not be considered as bone event.
Data extraction, (selection and coding)
Two review authors will independently screen citation titles and abstracts, where available, yielded by the literature search against pre-specified eligibility criteria. The screening process will assess whether the citation 1) reported data from an original research study (i.e, cross-sectional study, cohort study, randomized controlled trial), 2) focused on patients with Gaucher disease type I 3) evaluated plasma chitotriosidase activity, 4) evaluated serum CCL18 level, and 5) captured one or more outcomes of interest (among anemia, thrombocytopenia, splenomegaly, hepatomegaly, and symptomatic bone events ). Citations will be saved in a tab-delimited text file and imported into Microsoft Office Excel spreadsheet, in order to expedite selection process. The two review authors will rate each citation using a “relevant”, “irrelevant”, “unsure” (whether relevant or irrelevant) designation, using drop-down menus. Full-text articles will be retrieved for citations that received a “relevant” or “unsure” classification from at least one of the two review authors. Records rated “irrelevant” by the two review authors will be discarded for further review.
Two review authors will independently assess full text articles and decide if they meet pre-specified eligibility criteria, using a standardized eligibility form. We will contact corresponding authors to obtain additional information about study eligibility, where necessary. Disagreements will be resolved by discussion between the two review authors. Reasons for excluding study records will be recorded. We will quantify inter-rater agreement for identifying potentially eligible records and for selecting studies for final inclusion using Cohen's Kappa.
Two review authors will independently perform qualitative and quantitative data extraction. For this purpose, a data extraction form will be developed, pretested and adjusted as necessary. Disagreements will be resolved by discussion between the two review authors.
Where possible, individual participant data will be extracted from published articles and entered into a databasis. Corresponding authors and/or principal investigators of eligible primary studies will be invited to collaborate in this systematic review project by providing us with individual participant data. Contact information will be retrieved from published articles and online search. Each corresponding author and/or principal investigator will be first contacted by both regular and electronic mails. A reminder will be e-mailed to non-respondents one month later. A co-author of the primary study will be contacted in case of non-response from both the principal investigator and corresponding author.
Correspondence will enclose a cover letter, the systematic review protocol, the list of requested variables, and a consent form for participating in the systematic review. The cover letter will emphasize the collaborative nature of the project, ensure that supplied data will be held securely and treated as confidential, and offer participants the opportunity for co-authoring the final report. The data request form will be designed in accordance with Cochrane Individual Participant Data Meta-analysis Methods Group recommendations.
If they agree to participate, investigators will be asked for providing de-identified individual data for all patients enrolled in studies. In order to ensure maximum participation, data will be accepted in any format, but an example format detailing the recommended coding will be provided to all investigators. Depending on the preference of investigators, raw data can be supplied in any suitable electronic format (preferably in Stata or ASCII format) or completed paper data forms. Raw data files will be preferentially transferred by e-mail, in a password protected compressed folder. The password will be sent in a separate e-mail. Primary study datasets will be saved in their original formats and then converted to a common format by renaming and labeling the variables for each study in a consistent manner. Ultimately, investigators unable to provide individual participant data will be invited to provide us with aggregated data in tabular formats for pre-specified analyses. Investigators who decline to provide individual participants data will be surveyed for identifying potential reasons for not participating in the project.
For each primary study, the data will be cross-checked with respect to range, internal consistency, missing values, and consistency with published reports. Because the relationship between the biomarkers under review (i.e., plasma chitotriosidase activity and CCL18) and the outcomes of interest is observational in nature, we will not assess randomization integrity for randomized controlled trials of enzyme replacement or substrate reduction therapy, which is not in the scope of this systematic review. Converted data will be sent to the investigators for verification and attempts will be made to resolve inconsistencies by discussion. Raw data from all primary studies will be then collated into a single meta-analysis databasis, including a primary study identifier. All primary study raw datasets and meta-analysis databasis will be stored in a password protected area of a secure server at Grenoble university hospital. Access to databases will be limited to staff working directly on the project. No data will be used for any other purpose without permission of all collaborators.
Risk of bias (quality) assessment
Two review authors will independently appraise the methodological quality of included studies for each outcome of interest, using a checklist adapted from the Quality Assessment of Diagnostic Accuracy Studies (QUADAS)-2 tool. The QUADAS-2 tool comprises four domains including patient selection, index test, reference standard, and flow and timing. Each domain is evaluated in terms of risk of bias while the first three domains are also assessed with respect to applicability to clinical practice. Disagreement in risk of bias and/or applicability between the two review authors will be resolved by discussion.
Strategy for data synthesis
Information on key primary study and participant characteristics will be summarized in a table. Assessment of methodological quality across primary studies will be tabulated for each outcome of interest separately.
We will assess homogeneity across primary studies in terms of age profiles (children versus adults), methods for quantitative measurement of plasma chitotriosidase activity and CCL18 level, upper reference limits for plasma chitotriosidase activity and CCL18 level, and fulfilment of QUADAS-2 criteria. Primary studies with selected age profiles (paediatrics patients only) will be considered for inclusion in the quantitative meta-analysis by review authors. Yet, we anticipate that no primary study will be excluded from meta-analysis based on methodological quality assessment results. Primary studies that did not gather data for a prespecified outcome of interest will not be included in the meta-analysis for this outcome.
Effect size estimates:
All analyses will use individual participant data. Categorical variables will be reported as numbers and percentages and continuous variables as means and standard deviations. For highly skewed distributions, we will use geometric means as summary statistics and logarithm transformations for comparisons. In primary analysis, patients with and without the outcomes of interest will be compared with regard to plasma chitotriosidase activity and CCL18 levels, respectively. The secondary analyses will compare the area under the receiver operating characteristics (ROC) curve for plasma chitotriosidase activity and CCL18 levels in discriminating patients with and without each outcome of interest. For each outcome of interest listed above, the results of primary studies will be presented in separate forest plots, reporting numbers of eligible participants for whom data would be obtained and summary effect estimates along with 95% confidence intervals, where appropriate.
As recommended, individual participant data meta-analysis will be conducted with both two- and one-stage approaches. In two-stage approach, the first stage implies analyzing individual patient data within primary studies in order to generate study-level effect size point estimates and variances. In the second stage, point estimates from each primary study are combined using conventional meta-analytical methods. This two-stage approach provides summary effect size point estimates along with 95% confidence intervals, which account for between-study heterogeneity. Basically, we will use a DerSimonian and Laird’s random-effects meta-analysis model for combining weighted mean differences in plasma chitotriosidase activity and CCL18 levels (after logarithm transformation, if necessary) for patients with and without outcome of interest. Differences in areas under the ROC curve estimates for plasma chitotriosidase activity and CCL18 levels will be pooled using a random-effects meta-analysis model.
One-stage approach synthesizes individual patient data in a single step, using a mixed-effects regression modelling that accounts for patient clustering within primary studies. This approach allows incorporation of patient-level covariates and exploration of potential interactions involving patient- or study-level covariates. Basically, we will develop one-stage random-effects model for continuous dependent variable (i.e., plasma chitotriosidase activity or serum CCL18 level), with the outcome of interest entered as a binary independent variable. Because observations at baseline and follow-up visits are nested to patients (who are grouped within studies), we will also fit mixed-effects multilevel model for continuous dependent variables. Area under the ROC curve estimates and differences for plasma chitotriosidase activity and CCL18 levels will be computed, using resampling techniques (i.e., bootstrap), which account for patient clustering within primary studies.
Despite our efforts to ensure data quality, we anticipate missing values for surrogate markers (i.e., plasma chitotriosidase activity and CCL18), outcomes of interest, and patient-level covariates. For transparency purpose, the completeness of individual participant data will be reported for each variable separately and overall for observations. Missing-data mechanisms will be examined. Complete case analysis, which discards all observations with any missing value, will be undertaken in main analysis. To assess the robustness of our findings, an imputation method will be used for replacing missing values. Imputation methods assume data are missing at random, although this assumption is generally unverifiable. Missing values will be imputed within primary studies in two-stage approach and across primary studies (i.e., meta-analysis databasis) in one-stage approach.
Analysis of subgroups or subsets
Between-study heterogeneity will be evaluated graphically by examining coupled forest plots of sensitivity and specificity and statistically by using the I-squared inconsistency index. The I-squared index provides an estimate of the percentage of total variance across studies due to heterogeneity rather than chance. An I-squared index of 0% indicates no evidence of heterogeneity and larger values reflect increasing heterogeneity.
Depending on the number of studies for each outcome of interest, the following patient-and study-level covariates will be entered in turn into one-stage approach model to examine the sources of heterogeneity in summary estimates: age category (i.e., pediatrics versus adult patient), gender, splenectomy, enzyme replacement therapy or SRT within the previous year, fulfilment of QUADAS-2 criteria, and funding sources. We will assume that the effects of covariate and covariate-outcome interaction are common across primary studies. As recommended, we will also check for potential aggregation bias by developing one-stage random-effects model separating within and across primary study information.
To assess the robustness of our findings, we will also conduct sensitivity analysis leaving out one primary study at a time.
The dissemination policy aims 1) to assure timely publication, 2) to avoid publication of immature data, 3) to maintain high quality of published material, 4) to prevent duplicate publications, and 5) to assure equitable authorship.
Efforts will be made to reduce the interval between data extraction completion and the release of the systematic review and meta-analysis results. The members of the coordinating systematic review group will prepare a comprehensive report detailing all the analyses described above, in accordance with the 2015 Preferred Reporting Items for a Systematic Review and Meta-analysis of Individual Participant Data (PRISMA-IPD) statement. They will also draft manuscripts submitted for publication in medical journals. The comprehensive report and manuscript drafts will be circulated for approval by the members of the IPD meta-analysis collaborative group before submission. Authors’ names will be listed in order of contribution among the members of the IPD meta-analysis collaborative group. Assistance for preparing and editing manuscripts (i.e., English language revision) provided by professional medical writers will be acknowledged.
Contact details for further information
Quality of care unit
Grenoble University Hospital
38043 Grenoble Cedex 9
Organisational affiliation of the review
Centre Hospitalier Universitaire, F-63003 Clermont-Ferrand, France
Dr Tatiana Raskovalova, Hématologie Biologique, Clermont-Ferrand University Hospital, France Professor Marc Berger, Hématologie Biologique, Clermont-Ferrand University Hospital, France Professor Jose Labarere, Quality of care unit, Grenoble University Hospital, France
Professor Jerome Stirnemann, Division of Internal Medicine, Geneva University Hospital, Switzerland
Anticipated or actual start date
04 January 2016
Anticipated completion date
31 January 2017
This systematic review has received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors. The sponsor was not involved in the study protocol development and will not have any role in the conduct of the systematic review, data analysis and interpretation, or publication of the results.
Formal screening of search results against eligibility criteria
Risk of bias (quality) assessment
PROSPERO This information has been provided by the named contact for this review. CRD has accepted this information in good faith and registered the review in PROSPERO. CRD bears no responsibility or liability for the content of this registration record, any associated files or external websites.