Simple and statistically sound recommendations for analysing physical theories.

Physical theories that depend on many parameters or are tested against data from many different experiments pose unique challenges to statistical inference. Many models in particle physics, astrophysics and cosmology fall into one or both of these categories. These issues are often sidestepped with statistically unsound ad hoc methods, involving intersection of parameter intervals estimated by multiple experiments, and random or grid sampling of model parameters.

Nested Sampling for Frequentist Computation: Fast Estimation of Small p-Values.

We propose a novel method for computing p-values based on nested sampling (NS) applied to the sampling space rather than the parameter space of the problem, in contrast to its usage in Bayesian computation. The computational cost of NS scales as log^{2}1/p, which compares favorably to the 1/p scaling for Monte Carlo (MC) simulations. For significances greater than about 4σ in both a toy problem and a simplified resonance search, we show that NS requires orders of magnitude fewer simulations than ordinary MC estimates.