Learning disentangled representations of textual data is essential for many natural language tasks such as fair classification, style transfer and sentence generation, among others. The existent dominant approaches in the context of text data {either rely} on training an adversary (discriminator) that aims at making attribute values difficult to be inferred from the latent code {or rely on minimising variational bounds of the mutual information between latent code and the value attribute}. {However, the available methods suffer of the impossibility to provide a fine-grained control of the degree (or force) of disentanglement.} {In contrast to} {adversarial methods}, which are remarkably simple, although the adversary seems to be performing perfectly well during the training phase, after it is completed a fair amount of information about the undesired attribute still remains. This paper introduces a novel variational upper bound to the mutual information between an attribute and the latent code of an encoder. Our bound aims at controlling the approximation error via the Renyi's divergence, leading to both better disentangled representations and in particular, a precise control of the desirable degree of disentanglement {than state-of-the-art methods proposed for textual data}. Furthermore, it does not suffer from the degeneracy of other losses in multi-class scenarios. We show the superiority of this method on fair classification and on textual style transfer tasks. Additionally, we provide new insights illustrating various trade-offs in style transfer when attempting to learn disentangled representations and quality of the generated sentence.
时间序列表示学习是时间序列分析的一项基本任务。虽然已经取得了重大进展，以实现下游应用程序的准确表示，学习的表示往往缺乏可解释性，不暴露语义含义。不同于以往的纠缠特征空间的努力，我们的目标是提取语义丰富的时间相关性的潜在的可解释的因式分解表示的数据。受计算机视觉中解纠缠表示学习的成功启发，我们研究了学习语义丰富的时间序列表示的可能性，由于三个主要挑战，这一点尚未探索：1）顺序数据结构引入了复杂的时间相关性，使得潜在的表示难以解释，2）顺序模型遭受KL消失问题，3）时间序列的可解释语义概念往往依赖于多个因素而不是个体。为了弥补这一差距，我们提出了一种新的序列数据解纠缠增强框架--解纠缠时间序列（Disentangle Time Series，简称TSN）。具体而言，生成层次语义概念作为时间序列的可解释性和解纠缠表示，ESTA引入了多层次的解纠缠策略，涵盖个人的潜在因素和组语义段。我们进一步从理论上展示了如何缓解KL消失问题：KLM引入了一个互信息最大化项，同时保留了对总相关性和维数KL的更重惩罚以保持解纠缠特性。在各种真实世界的基准数据集上的实验结果表明，通过训练学习的表示在下游应用中具有优异的性能，具有较高的语义概念可解释性。
Task Adaptation from Skills: Information Geometry, Disentanglement, and New Objectives for Unsupervised Reinforcement Learning
Eindhoven University of Technology（埃因霍温理工大学）；University of Maryland, College Park（马里兰大学学院市分校）；Ruhr University Bochum（波鸿鲁尔大学）；Tencent Robotics X（腾讯机器人科技）；University of Liverpool（利物浦大学）
The Devil is in the Conflict: Disentangled Information Graph Neural Networks for Fraud Detection
Beijing Institute of Technology（北京理工大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
