To examine the quality of oral health promotion research evidence, and to assess the effectiveness of health promotion aimed at improving oral health using a systematic and scientifically defensible methodology.

MEDLINE, CINAHL, the Social Sciences Citation Index, and Health Service Abstracts were searched using the keywords 'dental health education', 'oral health promotion', 'effectiveness' and 'evaluation'. A panel of experts in Europe and the USA, and members of an interest group, were contacted for additional studies.

The reference lists of retrieved papers were examined for additional studies, and any relevant journals identified were added to a list of journals to be handsearched. The handsearch included these journals and others known to publish material pertinent to the review.

Papers were included if they were published in the English language after 1979. Multiple reports from the same study only contributed once to the review, whereas papers which used a variety of outcome measures were included in each outcome category.

Study designs of evaluations included in the review

The studies had to report an evaluative component. Theoretical and purely descriptive studies were excluded.

Specific interventions included in the review

Studies reporting any oral health promotion intervention were eligible for inclusion. The following interventions were included in the review: tooth brushing programmes in schools; education of the parents on children's dental health; parental tooth brushing instruction; group and individual education sessions; mass media; prophylaxis; fluoride tablets, gels, drops and rinses; attending dental care; instruction or demonstration of denture cleaning; interview with psychologist; and computer games.

Participants included in the review

The authors did not report any inclusion criteria relating to the participants. The settings were clinical, community, schools or other institutions. The participants were children, the elderly, adults, and people with handicaps or disabilities.

Outcomes assessed in the review

The outcome measures assessed included caries, oral hygiene, oral health-related knowledge, attitudes and behaviours.

How were decisions on the relevance of primary studies made?

The authors do not state how the papers were selected for the review, or how many of the reviewers performed the selection.

Papers were scored using the criteria of Pettiti (see Other Publications of Related Interest). These criteria placed particular weight on the definition, reliability and validity of the outcome measures, and the drop-out rate.The authors do not state how the papers were assessed for validity, or how many of the reviewers performed the validity assessment.

The authors do not state how the data were extracted for the review, or how many of the reviewers performed the data extraction.

Data were extracted on the following categories: study setting; main outcome; target population; number of participants; mean change in outcome, plus the standard deviation; study methodology; 'success' of the intervention; and any details regarding costs. The effectiveness of each intervention was calculated if there were sufficient data. The formula for this involved the weighted average at baseline (calculated from baseline scores in the intervention and control groups) and the separate baseline and final scores for the control and intervention groups.

How were the studies combined?

When sufficient homogeneity between a number of studies could be identified, a pooled estimate of the mean effect of the intervention was calculated together with the 95% confidence interval (CI). For studies that did not report sufficient data to be included in the formal meta-analysis, a qualitative synthesis of the results was undertaken.

If the majority of studies within each methodological group (RCT, quasi-experimental, single group or other) showed a positive outcome effect, the conclusion drawn was that there was robust evidence of a strong effect, which was not masked by poor research design or small numbers of participants. If, however, the majority of studies within each methodological group showed no intervention effect, the conclusion drawn was that there was no evidence that oral health promotion was effective.

How were differences between studies investigated?

Before pooling, homogeneity was considered in terms of study design, interventions, outcome and target groups.

One hundred and sixty-four articles were included in the review. There were 36 randomised controlled trials (RCTs; n=3,353), 80 quasi- experimental studies (i.e. non-randomised clinical trials), 33 single-group pre-test post-test studies, 7 multiple baseline and 6 mass media studies; the number of participants for the other study designs was not reported.

Caries.

RCTs (n=7): 4 RCTs reported a small non significant effect on the level of caries. Almost all of the oral health promotion initiatives which looked at caries involved the use of fluoride. The meta-analysis showed that the mean intervention effect was a caries reduction of 1.8 surfaces (95% CI: 0.38, 3.26).

Quasi-experimental studies (n=11): these studies indicated that the levels of caries could be reduced by daily brushing with a fluoridated toothpaste. The size of the intervention effect was dependent on the length of time that elapsed between the introduction and evaluation of the programme. Greater reductions in caries were observed in the longer-term studies. There was no evidence that the levels of caries were affected by interventions that did not involve daily brushing.

Single-group studies (n=3): these were poor studies and no conclusions could be drawn.

Oral Hygiene.

RCTs (n=23): the majority of studies used plaque levels in the participants' mouths as the outcome measure. The conclusions drawn in each individual study varied according to the follow-up period.

The majority of studies with short follow-up showed significant improvements in plaque levels, whilst studies with long periods of follow-up suggested that instruction and education about plaque control were not effective in the long term. The more elaborate and theoretically based interventions appeared to be no more successful in reducing plaque levels than the more simple approaches. A meta-analysis showed that the mean intervention effect was a 0.316 reduction in the plaque index (95% CI: -0.063, 0.695).

Quasi-experimental studies (n=33): there was no convincing evidence that school-based education programmes had any effect on the plaque levels in the participants' mouths, even when daily brushing at school was part of the programme. School-based programmes, whether run by dental professionals, teachers, or older pupils teaching younger pupils, have not been demonstrated to affect oral hygiene. In clinic- and work-based interventions, some experimental plaque control programmes with adults demonstrated dramatic reductions in plaque levels. Educating the parents about plaque control in their young children was effective.

Single-group studies (n=22): these were poor studies and no conclusions could be drawn.

Sugar consumption. Study designs not reported (n=8): the outcomes reported by these studies were behavioural intentions or reported behaviour. Thus, it was difficult to draw definitive conclusions from these studies.

Knowledge, attitudes and behaviours. Study designs not reported (n=37): these studies indicated that knowledge levels were invariably altered by the interventions described. Complex and technical educative methods added little benefit, and simple provision of information was sufficient to increase knowledge levels. However, the studies that included other outcome measures also suggested that alterations in knowledge, attitudes and beliefs were not related to changes in behaviour or health.

Mass media. Study designs not reported (n=7): these studies suggested that oral health promotion via the mass media was ineffective for promoting both knowledge and behaviour change. However, the authors stated that the evaluation methodologies in these studies were inadequate and, therefore, no specific conclusion regarding the role of mass media could be drawn.

Details of the costs were qualitatively recorded. However, the authors stated that few data were reported and, therefore, no statements about cost-to-benefit ratios could be made.

Oral health promotion which brings about the use of fluoride is effective for reducing caries. Chairside oral health promotion has been shown to be effective more consistently than other methods of health promotion. Mass media programmes have not been shown to be effective. The quality of research evaluating oral health promotion needs to be improved.

This was an average review of the area. A reasonable literature search was conducted. However, only studies published in English were included, thus important studies may have been missed and the results may be subject to publication bias. The inclusion criteria were mentioned in the methods and the abstract, but it would have been helpful if these had been presented more clearly. Details of the review process, such has how many authors were involved in each stage of the review, were not reported. Appropriate study details were presented in the article.

The authors stated that a validity assessment was performed, but this was not presented in the results section, or related to the results presented. It was therefore difficult to interpret the reliability of the individual studies included in the review. Given the heterogeneity in the studies in terms of intervention, design, populations and outcomes, it does not appear to have been appropriate to have pooled the results, especially as heterogeneity was not formally assessed. The pooled results should, therefore, be interpreted with extreme caution.

Overall, the authors' conclusions appear to be supported by the results presented. However, they should be interpreted with some degree of caution due to the limitations highlighted.

The authors did not state any implications for further research and practice.

Pettiti DB. Meta-analysis, decision analysis and cost-effectiveness analysis. Oxford: Oxford University Press; 1994.

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.