arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1901.01137cs.ITmath.ITmath.STstat.TH
收藏

信息度量相似性理论:基于Shannon熵的信息重要性度量

Information Measure Similarity Theory: Message Importance Measure via Shannon Entropy

Rui She, Shanyun Liu, Pingyi Fan

摘要:

在异常检测、安全系统等大数据应用场景中,罕见事件引起了人们的广泛关注。为了从概率的角度刻画稀有事件的重要性,提出了消息重要性度量(MIM)作为一种语义分析工具。与Shannon熵相似,MIM在信息处理中有其特殊的功能,其中MIM的参数$\varpi$起着至关重要的作用。实际上,参数$\varpi$支配MIM的性质,基于此,MIM具有三个工作区域,其中对应的参数分别满足$ 0 \le \varpi \le 2/\max\{p(x_i)\}$、$\varpi > 2/\max\{p(x_i)\}$和$\varpi < 0$。此外,在$ 0 \le \varpi \le 2/\max\{p(x_i)\}$的情况下,MIM和Shannon熵在信息压缩和传输方面具有相似性,这为信息论提供了一个新的视角。本文首先构建了一个具有消息重要性度量的系统模型,提出了消息重要性损失的概念,丰富了信息处理策略。此外,我们提出了消息重要性损失容量来衡量信息的重要性收获在传输。此外,提出了消息重要性失真函数,给出了基于消息重要性测度的信息压缩的上界。此外,研究了信息重要性损失约束下的比特率传输问题,拓宽了Shannon信息论的适用范围。

英文摘要:

Rare events attract more attention and interests in many scenarios of big data such as anomaly detection and security systems. To characterize the rare events importance from probabilistic perspective, the message importance measure (MIM) is proposed as a kind of semantics analysis tool. Similar to Shannon entropy, the MIM has its special functional on information processing, in which the parameter $\varpi$ of MIM plays a vital role. Actually, the parameter $\varpi$ dominates the properties of MIM, based on which the MIM has three work regions where the corresponding parameters satisfy $ 0 \le \varpi \le 2/\max\{p(x_i)\}$, $\varpi > 2/\max\{p(x_i)\}$ and $\varpi < 0$ respectively. Furthermore, in the case $ 0 \le \varpi \le 2/\max\{p(x_i)\}$, there are some similarity between the MIM and Shannon entropy in the information compression and transmission, which provide a new viewpoint for information theory. This paper first constructs a system model with message importance measure and proposes the message importance loss to enrich the information processing strategies. Moreover, we propose the message importance loss capacity to measure the information importance harvest in a transmission. Furthermore, the message importance distortion function is presented to give an upper bound of information compression based on message importance measure. Additionally, the bitrate transmission constrained by the message importance loss is investigated to broaden the scope for Shannon information theory.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑