Higher-order probabilistic perceptrons as Bayesian inference engines.

An explicit structural connection is established between the Bayes optimal classifier operating on K binary input variables and a corresponding two-layer perceptron having normalized output activities and couplings from input to output units of all orders up to K. With suitable modification of connection weights and biases, such a higher-order probabilistic perceptron should in principle be able to learn the statistics of the classification problem and match the a posteriori probabilities given by Bayes optimal inference. Specific training algorithms are developed that allow this goal to be approximated in a controlled variational sense.