Of the 19 included studies, 11 were case-control studies, 2 used an indirect cohort design, and 6 were cohort studies. Two of these studies were not included in the analysis.
The observational studies included in the review were assessed to be of moderate quality, with the length of follow-up restricted in all of the studies. Three studies could not be assessed for study quality because of the limited methodological information provided.
There were 13 observational studies that used invasive disease as an outcome, with a total of 4,984 events. The combined estimate showed that the vaccine had an efficacy of 53% (95% CI: 46, 59); the pooled OR was 0.47 (95% CI: 0.41, 0.54). This did not change significantly when combining only the case-control studies or cohort studies (both were conducted mainly in the elderly and chronic diseased). No statistically significant heterogeneity was found.
Five observational studies in the elderly used all-cause pneumonia as an outcome, with 263,300 person-years of observation; 5,513 events were observed. The combined estimate of vaccine efficacy was 32% (95% CI: 7, 50); the pooled RR was 0.68 (95% CI: 0.50, 0.93). There was statistically significant heterogeneity (chi-squared 72.3, d.f.=4, P<0.001). The exclusion of a large study of influenza and pneumococcal vaccine resulted in a non significant, heterogeneous pooled estimate of efficacy of 35% (95% CI: -7, 61).
The authors compared these results with a meta-analysis of RCTs that showed no statistically significant difference in invasive disease (homogeneous) or pneumonia (heterogeneous).