Regularization parameter estimation for feedforward neural networks.

IEEE Trans Syst Man Cybern B Cybern

Dept. of Comput. Sci., Beijing Normal Univ., China.

Published: October 2012

AI Article Synopsis

  • The Kullback-Leibler (KL) distance framework reveals that a specific Gaussian probability function in feedforward neural networks simplifies to the first-order Tikhonov regularizer.
  • The smooth parameter in kernel density estimation acts as the regularization parameter, leading to an estimation formula derived from approximations of training data sets.
  • Comparisons with prior research highlight both similarities and differences, with experiments demonstrating the effectiveness of the estimation formula in scenarios with sparse and small training samples.

Article Abstract

Under the framework of the Kullback-Leibler (KL) distance, we show that a particular case of Gaussian probability function for feedforward neural networks (NNs) reduces into the first-order Tikhonov regularizer. The smooth parameter in kernel density estimation plays the role of regularization parameter. Under some approximations, an estimation formula is derived for estimating regularization parameters based on training data sets. The similarity and difference of the obtained results are compared with other work. Experimental results show that the estimation formula works well in sparse and small training sample cases.

Download full-text PDF

Source
http://dx.doi.org/10.1109/TSMCB.2003.808176DOI Listing

Publication Analysis

Top Keywords

regularization parameter
8
feedforward neural
8
neural networks
8
estimation formula
8
estimation
4
parameter estimation
4
estimation feedforward
4
networks framework
4
framework kullback-leibler
4
kullback-leibler distance
4

Similar Publications

Background: Changes in cardiac function and structure as well as their association with the cardiac autonomic nervous system remain incompletely characterized in children with stage 5 chronic kidney disease (CKD) receiving hemodialysis (HD).

Methods: A prospective observational cohort study was conducted on 40 Egyptian children with CKD on regular HD compared to 40 age- and sex-matched healthy children. All participants underwent thorough clinical examination, laboratory investigations, 24-h Holter monitoring, and 2D/4D echocardiographic study (conventional and advanced modalities).

View Article and Find Full Text PDF

Fast and interpretable mortality risk scores for critical care patients.

J Am Med Inform Assoc

January 2025

Department of Computer Science, Duke University, Durham, NC 27708, United States.

Objective: Prediction of mortality in intensive care unit (ICU) patients typically relies on black box models (that are unacceptable for use in hospitals) or hand-tuned interpretable models (that might lead to the loss in performance). We aim to bridge the gap between these 2 categories by building on modern interpretable machine learning (ML) techniques to design interpretable mortality risk scores that are as accurate as black boxes.

Material And Methods: We developed a new algorithm, GroupFasterRisk, which has several important benefits: it uses both hard and soft direct sparsity regularization, it incorporates group sparsity to allow more cohesive models, it allows for monotonicity constraint to include domain knowledge, and it produces many equally good models, which allows domain experts to choose among them.

View Article and Find Full Text PDF

Based on the DCV-C system of fullerene acceptor organic solar cell active materials, the charge transfer process of D-A type molecular materials under the action of an external electric field () was explored. Within the range of electric field application, the excited state characteristics exhibit certain regular changes. Based on reducing the excitation energy, the excitation mode shows a trend of developing toward low excited states.

View Article and Find Full Text PDF

Annual biological variation and personalized reference intervals of 8 serum liver enzymes and 3 noninvasive tests in fatty liver patients.

Clin Chim Acta

January 2025

National Clinical Research Center for Laboratory Medicine, Department of Laboratory Medicine, The First Hospital of China Medical University, Shenyang 110001 China. Electronic address:

Background: The liver function tests and noninvasive tests (NITs) play important roles in the follow-up and monitoring of fatty liver disease (FLD). Our aim is to establish annual biological variation (BV) and personalized reference intervals (prRIs) of liver function tests for the first time in order to accurately assess the status and progress of FLD.

Methods: 67 fatty liver patients who participated in regular physical examination once a year for six consecutive years, were enrolled.

View Article and Find Full Text PDF

This study combines experimental techniques and mathematical modeling to investigate the dynamics of C. elegans body-wall muscle cells. Specifically, by conducting voltage clamp and mutant experiments, we identify key ion channels, particularly the L-type voltage-gated calcium channel (EGL-19) and potassium channels (SHK-1, SLO-2), which are crucial for generating action potentials.

View Article and Find Full Text PDF

Want AI Summaries of new PubMed Abstracts delivered to your In-box?

Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!