Do radiological research articles apply the term "pilot study" correctly? Systematic review.

Aim: To determine what proportion of radiological studies used the term "pilot" correctly.

Material And Methods: Indexed studies describing themselves as a "pilot" in their title were identified from four indexed radiological journals. The aim was to identify 20 consecutive, eligible studies from each journal, as this sample size was deemed sufficient to be representative as to how this methodological description was employed by authors of radiological articles. Data were extracted relating to study design and data presented. The review was reported according to PRISMA guidelines.

Results: The search string used identified 658 records across the four targeted journals. Ultimately, 78 reviews describing 5,572 patients were selected for systematic review. Median sample size was just 20 patients. No individual study qualified as a genuine pilot study when assessed against the a priori criteria. In reality, the large majority (66 studies, 84.6%) were framed as studies of diagnostic test accuracy. A significant proportion (21 studies, 26.9%) was retrospective, and the overwhelming majority were conducted in single centres (76 centres, 94.7%). Most (55 studies, 70.5%) stated no rationale for their sample size, and no study presented a formal power calculation.

Conclusion: Radiological "pilot" studies are mostly underpowered studies of diagnostic test accuracy. In order to have scientific credibility, authors, reviewers, and editors of radiological journals are encouraged to familiarise themselves with different methodological study designs and their precise implications.