A randomized trial about the perceived informativeness of new empirical evidence. Does beta-carotene prevent (cervical) cancer?

The perceived informativeness of a publication can be assessed by measuring the change in belief it induces among the scientific public, regarding a certain hypothesis. In a randomized trial, we studied the effect of empirical evidence from a clinical experiment and a case-control study on the hypothesis that beta-carotene protects against (cervical) cancer. The study population consisted of first authors of recently published patient-oriented research papers. They received an abstract of the clinical experiment, of the case-control study, or a "placebo" abstract. The latter was used to assess the specific effect of the empirical evidence in the two real studies. The change in belief in the hypotheses was expressed as a likelihood ratio (LR). All three abstracts led to a decrease in belief in the hypothesis. The median LRs of the abstracts of the experiment, case-control study and "placebo" were 0.33, 0.45, 0.75 respectively. This paper shows that the belief in a certain hypothesis is influenced by the quality of empirical evidence in a study. The magnitude of change induced by the experimental and case-control abstract had the anticipated order, but the change in belief induced by the "placebo" abstract was larger than we had expected. Reasons for this may be the concise information in the abstract and the variable methodological training of the study population.