Two hundred and one articles were included for review (n = more than 26,000); 77 post-test-only multiple group designs (n=9,881), 55 pre-test-post-test multiple group designs (n= 6,689; n not available for one study) and 69 repeated measures designs (n=9,562). Sample sizes ranged from 3 to 1,437 participants.
The mean quality score was 2.5 (SD 1.3) out of a potential 6 for no-intervention controlled studies and 3.5 (SD 1.4) for non-internet comparison studies. Loss to follow up was high (more than 25%) for many studies.
Comparison with no intervention (130 studies n=19,234).
Internet-based learning was associated with significant gains in knowledge (117 studies reporting on 126 interventions: pooled effect size = 1.00, 95% CI: 0.90, 1.10, p<0.001), skills (16 studies, pooled effect size 0.85, 95% CI:0.49, 1.20, p<0.001) and behaviours/effects on patient care (19 studies reporting 32 interventions, pooled effect size = 0.82, 95% CI: 0.63, 1.02, p<0.001). There was evidence of significant heterogeneity for all of these outcomes (Ι2 = 93.6%, 92.7% and 79.1%). Subgroup analyses revealed that low-quality studies demonstrated larger effect sizes for knowledge (mean score 1.07, 95% CI: 0.96, 1.18 versus mean score = 0.71 95% CI: 0.51, 0.92 p<0.003). Practice exercises were associated with a significant gain in skills (pooled effect size = 1.01, 95% CI: 0.60, 0.43 versus pooled effect size = 0.21, 95% CI: 0.04, 0.38 p<0.001) but with significantly worse behaviour and effect on patient outcomes (pooled effect size = 0.44, 95% CI: 0.33, 0.55 versus pooled effect size = 2.09, 95% CI: 1.38, 2.79, p<0.001). The use of tutorials, online peer discussion and longer duration courses were associated with significantly better outcomes for behaviours/effect on patients (no statistical data provided). High interactivity, access to ongoing course materials and online discussion were not associated with an increase in knowledge or skills.
Comparison with non-internet-based learning.
Internet-based learning was associated with a small but statistically significant increase in knowledge compared to non-internet-based learning methods (63 studies, pooled effect size = 0.12, 95% CI: 0.003, 0.24, p<0.045). There was evidence of significant heterogeneity (Ι2 = 88.1%). When blended interventions were removed in a sensitivity analysis the effect became non-significant. Internet learning did not significantly increase satisfaction, skills, or behaviours and effects on patient care when compared to non-internet-based learning. Subgroup analyses revealed that online courses using dicussion yielded significantly higher effects for knowledge (p=0.002), skills (p≤0.04) and behaviours/effect on patients (p=0.02). High levels of interactivity and the use of practice exercises increased the effect size for skills (p≤0.04) but not for knowledge. Single instance interventions showed greater effects in improving skills (p=0.02) and behaviour and effect on patients (p=0.006) when compared to ongoing courses. Short courses and single instance courses were associated with higher levels of learner satisfaction.