Hypothesis testing, statistical power, and confidence limits in the presence of epistemic uncertainty.

Hypothesis testing, statistical power, and confidence limits are concepts from classical statistics that require data from observations. In some important recent applications some of the data are not observational but are reconstructed by computer models. There is generally epistemic uncertainty in model formulations, as well as in parameter and input values. The resulting epistemic uncertainty of the reconstructed data is determined by an uncertainty analysis and is expressed by subjective probability distributions. Sometimes only the mean or median values of the distributions are used in the concepts mentioned above, which hides the uncertainty of the data thereby rendering misleading results. Misleading results are also obtained if the epistemic uncertainty of the data is combined incorrectly with the stochastic variability of the outcome of the actual random complex concerned. This paper argues that an uncertainty analysis of the application of classical statistical concepts is essentially the correct way of dealing with the epistemic uncertainty of the data. A practical example serves as an illustration.