The review question and the inclusion criteria were clear, except for study design. No study design criteria were initially defined, but the review effectively restricted itself to double-blinded randomised placebo-controlled trials, which also met other quality criteria through the application of a very rigorous quality assessment. The authors reported considerable details on the participants, treatments and conduct of the 14 studies that did not fulfil these criteria, but did not report their results.
The search strategy was good and included German articles, as many studies have apparently been conducted in Germany. No details of the study selection and data extraction processes were included, so it is impossible to judge whether these were adequate to guard against errors. The validity assessments were checked by a second reviewer.
Basic results from the three placebo-controlled RCTs were mentioned in the 'Discussion' section. However, the authors neither reported nor undertook any statistical analysis of the results, and so the statistical significance of the results could not be assessed. Whilst the authors are correct to point out that confidence intervals provide more information than statistical significance alone, a test of significance with an associated p-value would, nevertheless, have conveyed useful information. These three studies were rated as having inadequate evidence on the grounds that there were no details of the randomisation process and that, even where the investigators were blinded, the patients might not have been (although this would not have be a common scenario). The studies were also criticised for not reporting power calculations, confidence intervals and details of prognostic factors. Whilst all these considerations are certainly worthy of note in assessing study validity, it is the opinion of this reviewer that the results of such trials, although certainly not of the highest quality, should be fully reported to be more informative. The authors' conclusions appear to be justified, but more details of the trials constituting the 'preliminary evidence' would have been helpful.