This was a poorly designed study to estimate clinical effectiveness since the methodology was both insufficiently rigorous and inappropriate. In particular the patient characteristics admitted to services were not similar across the two periods. Indeed for service 1, the average severity of infection encountered during the intervention period compared with that of the baseline period was lower (1.6 vs. 2.1 infections per patient, p<0.01), and hence the lower average duration of therapy per patient and the average number of regimens may be a direct consequence of such a difference. As such the cost savings cannot be attributed to the use of consensus guidelines and concurrent intervention.
The measure of therapeutic outcome incorporated microbiologic outcome but this was considered unevaluable if no follow-up cultures were obtained, or if the infection type was considered either a "fever of unknown origin", "fever and neutropenia" or "unknown". Evaluation of microbiologic response was subsequently not undertaken for 46%, 65% and 81% of groups 1,2 and 3 respectively during the baseline evaluation period and for 55%, 82% and 70% for the evaluation period and as such the impact of consensus treatment on therapeutic outcome was likely to be biased.
The effectiveness of concurrent intervention by a clinical pharmacist was justified by comparison of therapeutic outcomes associated with services 1 and 2 across the two periods. Since service 2 evaluated consensus guidelines without concurrent intervention and did not have any improvements in outcome it was assumed that the improvements in service 1 (with the intervention) were as a result of concurrent intervention. However, this appears to be an inadequate basis for comparison since the underlying differences in the malignancies between services 1 and 2 may have influenced the effectiveness of treatment. The study would have benefited from undertaking an RCT in the intervention period examining the use of consensus guidelines with and without concurrent intervention within each service group rather than comparing across different service groups (although it must be recognised that the relatively small sample size for groups 1 and 3 would have prohibited such an approach). The study was neither internally or externally valid and there was no sensitivity analysis.