This article proposes an approximated Bayesian entropy estimator for a discrete random variable. An entropy estimator that achieves least square error is obtained through Bayesian estimation of the occurrence probabilities of each value taken by the discrete random variable. This Bayesian entropy estimator requires large amount of calculation cost if the random variable takes numerous sorts of values. Therefore, the present article proposes a practical method for calculating an Bayesian entropy estimate; the proposed method utilizes approximation of the entropy function by a truncated Taylor series. Numerical experiments demonstrate that the proposed entropy estimation method improves estimation precision of entropy remarkably in comparison to the conventional entropy estimation method.
Download full-text PDF |
Source |
|---|---|
| http://dx.doi.org/10.1109/IEMBS.2004.1403100 | DOI Listing |
Publication Analysis
Top Keywords
Similar Publications
Forecasting Stock Market Indices Using Integration of Encoder, Decoder, and Attention Mechanism.
Entropy (Basel)
January 2025
Faculty of Mathematics and Statistics, Ton Duc Thang University, Ho Chi Minh City 700000, Vietnam.
Accurate forecasting of stock market indices is crucial for investors, financial analysts, and policymakers. The integration of encoder and decoder architectures, coupled with an attention mechanism, has emerged as a powerful approach to enhance prediction accuracy. This paper presents a novel framework that leverages these components to capture complex temporal dependencies and patterns within stock price data.
View Article and Find Full Text PDFReporting Standards for Bayesian Network Modelling.
Entropy (Basel)
January 2025
Bayesian Intelligence, Upwey, VIC 3158, Australia.
Reproducibility is a key measure of the veracity of a modelling result or finding. In other research areas, notably in medicine, reproducibility is supported by mandating the inclusion of an agreed set of details into every research publication, facilitating systematic reviews, transparency and reproducibility. Governments and international organisations are increasingly turning to modelling approaches in the development and decision-making for policy and have begun asking questions about accountability in model-based decision making.
View Article and Find Full Text PDFAdvanced Monte Carlo for Acquisition Sampling in Bayesian Optimization.
Entropy (Basel)
January 2025
Instituto Universitario de Investigacion en Ingenieria de Aragon (I3A), Universidad de Zaragoza, 50018 Zaragoza, Spain.
Optimizing complex systems usually involves costly and time-consuming experiments, where selecting the experiments to perform is fundamental. Bayesian optimization (BO) has proved to be a suitable optimization method in these situations thanks to its sample efficiency and principled way of learning from previous data, but it typically requires that experiments are sequentially performed. Fully distributed BO addresses the need for efficient parallel and asynchronous active search, especially where traditional centralized BO faces limitations concerning privacy in federated learning and resource utilization in high-performance computing settings.
View Article and Find Full Text PDFEntropy-Based Stochastic Optimization of Multi-Energy Systems in Gas-to-Methanol Processes Subject to Modeling Uncertainties.
Entropy (Basel)
January 2025
Shandong Rongxin Group Co., Ltd., Zoucheng 273517, China.
In gas-to-methanol processes, optimizing multi-energy systems is a critical challenge toward efficient energy allocation. This paper proposes an entropy-based stochastic optimization method for a multi-energy system in a gas-to-methanol process, aiming to achieve optimal allocation of gas, steam, and electricity to ensure executability under modeling uncertainties. First, mechanistic models are developed for major chemical equipments, including the desulfurization, steam boilers, air separation, and syngas compressors.
View Article and Find Full Text PDFModel Selection from Multiple Model Families in Species Distribution Modeling Using Minimum Message Length.
Entropy (Basel)
December 2024
Department of Data Science and Artificial Intelligence, Faculty of Information Technology, Monash University, Clayton, VIC 3800, Australia.
Species distribution modeling is fundamental to biodiversity, evolution, conservation science, and the study of invasive species. Given environmental data and species distribution data, model selection techniques are frequently used to help identify relevant features. Existing studies aim to find the relevant features by selecting the best models using different criteria, and they deem the predictors in the best models as the relevant features.
View Article and Find Full Text PDFWant AI Summaries of new PubMed Abstracts delivered to your In-box?
Enter search terms and have AI summaries delivered each week - change queries or unsubscribe any time!