具有序数嵌入的细粒度长度可控视频描述
Fine-grained length controllable video captioning with ordinal embeddings
AI总结：
本文提出基于 multi-hot 向量的 bit embedding 和 ordinal embedding 方法，通过非线性 MLP 实现细粒度视频描述长度控制，在多个数据集上验证了有效性并证明长度与语义可分离学习。
摘要：
本文提出了一种视频描述方法，能够控制生成描述的长度。以往关于长度控制的工作通常只有很少的级别来表达长度。在本研究中，我们提出了两种用于细粒度长度控制的长度嵌入方法。传统的嵌入方法是线性的，使用 one-hot 向量和嵌入矩阵。在本研究中，我们提出了用 multi-hot 向量表示长度的方法。一种是 bit embedding，用位表示来表达长度；另一种是 ordinal embedding，使用序数回归中常用的二进制表示。这些 multi-hot 向量的长度表示通过非线性 MLP 转换为长度嵌入。该方法不仅能控制描述句子的长度，还能控制阅读描述的时间。使用 ActivityNet Captions 和 Spoken Moments in Time 的实验表明，所提方法能有效控制生成描述的长度。用 ICA 对嵌入向量进行分析表明，长度和语义是分开学习的，证明了所提嵌入方法的有效性。
英文摘要：
This paper proposes a method for video captioning that controls the length of generated captions. Previous work on length control often had few levels for expressing length. In this study, we propose two methods of length embedding for fine-grained length control. A traditional embedding method is linear, using a one-hot vector and an embedding matrix. In this study, we propose methods that represent length in multi-hot vectors. One is bit embedding that expresses length in bit representation, and the other is ordinal embedding that uses the binary representation often used in ordinal regression. These length representations of multi-hot vectors are converted into length embedding by a nonlinear MLP. This method allows for not only the length control of caption sentences but also the control of the time when reading the caption. Experiments using ActivityNet Captions and Spoken Moments in Time show that the proposed method effectively controls the length of the generated captions. Analysis of the embedding vectors with ICA shows that length and semantics were learned separately, demonstrating the effectiveness of the proposed embedding methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
