Observations by a statistical watchdog.

During more than five decades, the author has kept a critical eye on how statistical methods are (mis-)used in medical research. Some areas are presented where serious statistical mistakes are prevalent. Two investigations with erroneous conclusions are described in detail.

Are degenerative spondylolisthesis and further slippage postoperatively really issues in spinal stenosis surgery?

Background and aims Opinions diverge concerning the prognostic importance of preoperative degenerative spondylolisthesis in patients with lumbar spinal stenosis, as well as the significance of further slippage post-operatively following decompression alone. However, a slip is only one among several factors related to the topic, e.g.

Superior outcomes following cervical fusion vs. multimodal rehabilitation in a subgroup of randomized Whiplash-Associated-Disorders (WAD) patients indicating somatic pain origin-Comparison of outcome assessments made by four examiners from different disciplines.

Background And Aims: Whiplash-Associated Disorders (WAD) are characterized by great variability in long-term symptoms. Patients with central neck and movement-induced stabbing pain participated in a randomized study comparing cervical fusion and multimodal rehabilitation. As reported in our previous paper, more patients treated by cervical fusion than by rehabilitation experienced pain relief.

A small group Whiplash-Associated-Disorders (WAD) patients with central neck pain and movement induced stabbing pain, the painful segment determined by mechanical provocation: Fusion surgery was superior to multimodal rehabilitation in a randomized trial.

Background: The majority of patients suffering from a whiplash injury will recover, but some will have symptoms (Whiplash Associated Disorders, WAD) for years despite conservative treatment. Some of these patients perceive neck pain that might come from a motion segment, possibly the disc. In comprehensive reviews no evidence has been found that fusion operations have a positive treatment effect on neck pain in WAD patients.

Symptoms and signs possibly indicating segmental, discogenic pain. A fusion study with 18 years of follow-up.

Background: Only two out of the five existing randomized studies have reported better results from fusion surgery for chronic low back pain (CLBP) compared to conservative treatment. In these studies the back symptoms of the patients were described simply as "chronic low back pain". One possible reason for the modest results of surgery is the lack of a description of specified symptoms that might be related to a painful segment/disc, and patient selection may therefore be more or less a matter of chance.

Clinical outcome following anterior arthrodesis in patients with presumed sacroiliac joint pain.

Background: It has been reported that in 13-32% of patients with chronic low back pain, the pain may originate in the sacroiliac (SI) joints. When treatment of these patients with analgesics and physiotherapy has failed, a surgical solution may be discussed. Results of such surgery are often based on small series, retrospective analyses or studies using a minimal invasive technique, frequently sponsored by manufacturers.

Private and Non-Private Disc Herniation Patients: Do they Differ?

Objectives: In the 2006 yearly report from the Swedish National Register for Lumbar Spine Surgery it was claimed that international studies show obvious differences between private and non-private patients with regard to results from back surgery. Therefore our aim was to reveal such possible differences by comparing the two categories of patients at a private clinic.

Material And Methods: The material comprises 1184 patients operated on for lumbar disc herniation during the period of 1987 to 2007.

Thyroid-associated ophthalmopathy after treatment for Graves' hyperthyroidism with antithyroid drugs or iodine-131.

Context: Previous randomized trials have suggested an association between radioiodine treatment for Graves' hyperthyroidism and thyroid-associated ophthalmopathy (TAO).

Objectives: The aim of the study was to compare the occurrence of worsening or development of TAO in patients who were treated with radioiodine or antithyroid drugs.

Design: We conducted a randomized trial (TT 96) with a follow-up of 4 yr.

Perspective on Roseroot (Rhodiola rosea) studies.

Rhodiola rosea (roseroot) extract is a commercially successful product, primarily used to reduce the effect of fatigue on physical and mental performance. In this perspective we present our investigation of the most recent studies performed on human subjects. With a focus on the statistical methods we found considerable shortcomings in all but one of the studies that claim significant improvement from roseroot extract.

Personality characteristics influencing determinacy of day and night blood pressure and heart rate.

The evidence of emotional impacts on day and night blood pressure (BP), heart rate (HR) and non-dipping of BP remains fragmentary. Personality traits previously tested by self-reports as determinants of office, screening or mean daytime BP produced mixed results. Therefore, we hypothesized that some traits are acting together as modifiers of 24-h and day/night ambulatory BP and HR.

Master blood pressure variation in office, by self, and in sleep recordings.

Variation inherent to each session in any recording situation has to be mastered to obtain informative mean values and differences. Variation due to time trend and unsteady consecutive readings versus inter-arm variations in blood pressure (BP) in the operator-initiated, self-initiated, and sleep recordings were studied by oscillometry. They were small in operator-initiated recordings, increased in self-recordings, and were paramount in sleep.

How to control measurement variation in estimating the mean intraocular pressure reducing effect at a specific point in time.

Objective: To clarify how to control measurement variation in estimating the mean intraocular pressure reducing effect at a specific point in time.

Methods: A set of 629 open-angle glaucoma or ocular hypertensive patients, eligible for treatment in both eyes, previously untreated, were reanalyzed at 3 or 6 months of intraocular pressure (IOP) reducing treatment. The trial design had parallel treatment groups; one group receives treatment A and the other group receives treatment B.

Which endpoint: mean intraocular pressure or proportion of successful patients?

A reduction in intraocular pressure in clinical trials can be determined through the mean intraocular pressure, through the proportion of patients who have the intraocular pressure reduced to a specific target intraocular pressure, or both. Since both these possible endpoints measure the shift of 2 intraocular pressure distributions, we recommend that only one of them be tested. In general, testing the difference between mean-values is much more efficient than testing the difference between proportions.

[Statistics must be seen and scrutinized! Fundamental to the progress of medical science--yet neglected in medical journals].

Even the most respectable medical journals contain questionable conclusions, frequently based on erroneous statistical analyses. A common attitude among medical researchers seems to be that a deeper knowledge of statistical theory is not necessary--it might suffice to be acquainted with the statistical program packages. The most common statistical errors in medical manuscripts are related either to the structure of the data or to misunderstandings regarding the logic in some basic statistical procedures, especially significance analysis.

["P" as in Program Package. About torturing data and significant "fishing expeditions"].

For the study of prognostic factors, the medical researchers have access to a number of advanced statistical techniques available in standard program packages. A tradition has developed where survival or time to relapse is analysed on the basis of statistical materials with few patients but a large number of possible explanatory variables. In statistical "fishing expeditions" the p-values are used to sort out potentially useful prognostic variables.