A New Approach of Deriving Bounds between Entropy and Error from Joint Distribution: Case Study for Binary Classifications
Bao-Gang Hu, Hong-Jie Xing
备注：
11 pages, 5 figures, 2 Appendixes including Maple code. A refined version based on the previous version entitled "Analytical bounds between entropy and error probability in binary classifications", appeared as arXiv:1205.6602v1[cs.IT] in May 30, 2012
The existing upper and lower bounds between entropy and error are mostly derived through an inequality means without linking to joint distributions. In fact, from either theoretical or application viewpoint, there exists a need to achieve a complete set of interpretations to the bounds in relation to joint distributions. For this reason, in this work we propose a new approach of deriving the bounds between entropy and error from a joint distribution. The specific case study is given on binary classifications, which can justify the need of the proposed approach. Two basic types of classification errors are investigated, namely, the Bayesian and non-Bayesian errors. For both errors, we derive the closed-form expressions of upper bound and lower bound in relation to joint distributions. The solutions show that Fano's lower bound is an exact bound for any type of errors in a relation diagram of "Error Probability vs. Conditional Entropy". A new upper bound for the Bayesian error is derived with respect to the minimum prior probability, which is generally tighter than Kovalevskij's upper bound.
Fundamental Limits of Joint Target Detection and Parameter Estimation - Characterizing Mixed-State Sensing Limits via Posterior Entropy Volume
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Purple Mountain Laboratories（紫金山实验室）；Shenyang Aerospace University（沈阳航空航天大学）；Nanjing University of Posts and Telecommunications（南京邮电大学）
