多周期过程:具有亚线性熵的遍历源
Multiperiodic Processes: Ergodic Sources with a Sublinear Entropy
AI总结:
本文提出多周期过程,一种由随机位移生成的确定性多周期序列构成的平稳遍历过程,具有消失的香农熵率,并满足齐夫定律和希尔伯格定律。
摘要:
已知几种显式随机过程满足希尔伯格定律,即块熵的幂律增长,推测用于自然语言,最近与神经扩展定律相关联。现有例子要么具有正的香农熵率,要么非遍历,或者需要较复杂的构造。本文引入多周期过程,一种新的平稳遍历过程,由随机位移确定性多周期序列生成。在温和条件下,多周期过程具有消失的香农熵率,并在适当参数化下满足符号频率的齐夫定律和块熵的希尔伯格定律。由于多周期过程不混合,本文识别了构造具有消失熵率和希尔伯格定律的简单强混合源的开放问题。
英文摘要:
Several explicit stochastic processes are known to satisfy Hilberg's law, a power-law growth of block entropy conjectured for natural language and recently connected to the neural scaling law. Existing examples either possess a positive Shannon entropy rate, are non-ergodic, or require comparatively involved constructions. We introduce multiperiodic processes, a new class of stationary ergodic processes over the natural numbers generated by random shifts of deterministic multiperiodic sequences. Under mild conditions, multiperiodic processes have vanishing Shannon entropy rate and, under a suitable parameterization, they satisfy both Zipf's law for symbol frequencies and Hilberg's law for block entropy. Since multiperiodic processes are not mixing, we identify the open problem of constructing an elementary strongly mixing source with vanishing entropy rate and Hilberg's law.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。