The Rényi Entropies Operate in Positive Semifields.

We set out to demonstrate that the Rényi entropies are better thought of as operating in a type of non-linear semiring called a positive semifield. We show how the Rényi's postulates lead to Pap's g-calculus where the functions carrying out the domain transformation are Rényi's information function and its inverse. In its turn, Pap's g-calculus under Rényi's information function transforms the set of positive reals into a family of semirings where "standard" product has been transformed into sum and "standard" sum into a power-emphasized sum.

The Case for Shifting the Rényi Entropy.

We introduce a variant of the Rényi entropy definition that aligns it with the well-known Hölder mean: in the new formulation, the -th order Rényi Entropy is the logarithm of the inverse of the -th order Hölder mean. This brings about new insights into the relationship of the Rényi entropy to quantities close to it, like the information potential and the partition function of statistical mechanics. We also provide expressions that allow us to calculate the Rényi entropies from the Shannon cross-entropy and the escort probabilities.

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle.

Data transformation, e.g., feature transformation and selection, is an integral part of any machine learning procedure.

Interactive knowledge discovery and data mining on genomic expression data with numeric formal concept analysis.

Background: Gene Expression Data (GED) analysis poses a great challenge to the scientific community that can be framed into the Knowledge Discovery in Databases (KDD) and Data Mining (DM) paradigm. Biclustering has emerged as the machine learning method of choice to solve this task, but its unsupervised nature makes result assessment problematic. This is often addressed by means of Gene Set Enrichment Analysis (GSEA).

100% classification accuracy considered harmful: the normalized information transfer factor explains the accuracy paradox.

The most widely spread measure of performance, accuracy, suffers from a paradox: predictive models with a given level of accuracy may have greater predictive power than models with higher accuracy. Despite optimizing classification error rate, high accuracy models may fail to capture crucial information transfer in the classification task. We present evidence of this behavior by means of a combinatorial analysis where every possible contingency matrix of 2, 3 and 4 classes classifiers are depicted on the entropy triangle, a more reliable information-theoretic tool for classification assessment.