A novel correlation Gaussian process regression-based extreme learning machine.

An obvious defect of extreme learning machine (ELM) is that its prediction performance is sensitive to the random initialization of input-layer weights and hidden-layer biases. To make ELM insensitive to random initialization, GPRELM adopts the simple an effective strategy of integrating Gaussian process regression into ELM. However, there is a serious overfitting problem in kernel-based GPRELM (GPRELM). In this paper, we investigate the theoretical reasons for the overfitting of GPRELM and further propose a correlation-based GPRELM (GPRELM), which uses a correlation coefficient to measure the similarity between two different hidden-layer output vectors. GPRELM reduces the likelihood that the covariance matrix becomes an identity matrix when the number of hidden-layer nodes is increased, effectively controlling overfitting. Furthermore, GPRELM works well for improper initialization intervals where ELM and GPRELM fail to provide good predictions. The experimental results on real classification and regression data sets demonstrate the feasibility and superiority of GPRELM, as it not only achieves better generalization performance but also has a lower computational complexity.