Quantifying Total Influence between Variables with Information Theoretic and Machine Learning Techniques.

The increasingly sophisticated investigations of complex systems require more robust estimates of the correlations between the measured quantities. The traditional Pearson correlation coefficient is easy to calculate but sensitive only to linear correlations. The total influence between quantities is, therefore, often expressed in terms of the mutual information, which also takes into account the nonlinear effects but is not normalized.

A Model Falsification Approach to Learning in Non-Stationary Environments for Experimental Design.

The application of data driven machine learning and advanced statistical tools to complex physics experiments, such as Magnetic Confinement Nuclear Fusion, can be problematic, due the varying conditions of the systems to be studied. In particular, new experiments have to be planned in unexplored regions of the operational space. As a consequence, care must be taken because the input quantities used to train and test the performance of the analysis tools are not necessarily sampled by the same probability distribution as in the final applications.

On the Use of Entropy to Improve Model Selection Criteria.

The most widely used forms of model selection criteria, the Bayesian Information Criterion (BIC) and the Akaike Information Criterion (AIC), are expressed in terms of synthetic indicators of the residual distribution: the variance and the mean-squared error of the residuals respectively. In many applications in science, the noise affecting the data can be expected to have a Gaussian distribution. Therefore, at the same level of variance and mean-squared error, models, whose residuals are more uniformly distributed, should be favoured.

On the Use of Transfer Entropy to Investigate the Time Horizon of Causal Influences between Signals.

Understanding the details of the correlation between time series is an essential step on the route to assessing the causal relation between systems. Traditional statistical indicators, such as the Pearson correlation coefficient and the mutual information, have some significant limitations. More recently, transfer entropy has been proposed as a powerful tool to understand the flow of information between signals.