UnMA-CapSumT：统一多头注意力驱动的标题摘要Transformer
UnMA-CapSumT: Unified and Multi-Head Attention-driven Caption Summarization Transformer
AI总结：
本文提出UnMA-CapSumT框架，结合事实性与风格化（浪漫、幽默）图像描述，通过多头注意力摘要Transformer解决词汇外和重复问题，在Flickr8K和FlickrStyle10K上验证有效性。
摘要：
图像描述生成是为图像生成自然语言描述的任务，近年来获得了极大的普及。为此，研究者们设计了不同的深度学习技术来开发事实性和风格化的图像描述生成模型。以往的模型侧重于分别生成事实性和风格化的描述，为单张图像提供多个描述。这些生成的描述存在词汇外和重复问题。据我们所知，目前尚无工作提供一种整合不同描述方法、以事实性和风格化（浪漫和幽默）元素描述图像内容的描述。为克服这些局限，本文提出了一种新颖的基于统一注意力和多头注意力驱动的标题摘要Transformer（UnMA-CapSumT）的描述生成框架。该框架分别利用由改进的自适应注意力事实性图像描述模型（MAA-FIC）生成的事实性描述和由风格因子Bi-LSTM与注意力（SF-Bi-ALSTM）驱动的风格化图像描述模型生成的风格化描述。基于SF-Bi-ALSTM的风格化图像描述模型生成两种主要的表达风格——浪漫和幽默。所提出的摘要器UnMHA-ST结合了输入图像的事实性和风格化描述，以生成风格丰富、连贯的摘要描述。所提出的UnMHA-ST Transformer通过引入所提出的词嵌入fastText与注意力词嵌入（fTA-WE）以及带覆盖机制的指针生成器网络，高效地学习和总结不同的语言风格，以解决词汇外问题和重复问题。在Flickr8K和FlickrStyle10K的子集上进行了大量实验，并辅以消融研究，以证明所提出框架的效率和有效性。
英文摘要：
Image captioning is the generation of natural language descriptions of images which have increased immense popularity in the recent past. With this different deep-learning techniques are devised for the development of factual and stylized image captioning models. Previous models focused more on the generation of factual and stylized captions separately providing more than one caption for a single image. The descriptions generated from these suffer from out-of-vocabulary and repetition issues. To the best of our knowledge, no such work exists that provided a description that integrates different captioning methods to describe the contents of an image with factual and stylized (romantic and humorous) elements. To overcome these limitations, this paper presents a novel Unified Attention and Multi-Head Attention-driven Caption Summarization Transformer (UnMA-CapSumT) based Captioning Framework. It utilizes both factual captions and stylized captions generated by the Modified Adaptive Attention-based factual image captioning model (MAA-FIC) and Style Factored Bi-LSTM with attention (SF-Bi-ALSTM) driven stylized image captioning model respectively. SF-Bi-ALSTM-based stylized IC model generates two prominent styles of expression- {romance, and humor}. The proposed summarizer UnMHA-ST combines both factual and stylized descriptions of an input image to generate styled rich coherent summarized captions. The proposed UnMHA-ST transformer learns and summarizes different linguistic styles efficiently by incorporating proposed word embedding fastText with Attention Word Embedding (fTA-WE) and pointer-generator network with coverage mechanism concept to solve the out-of-vocabulary issues and repetition problem. Extensive experiments are conducted on Flickr8K and a subset of FlickrStyle10K with supporting ablation studies to prove the efficiency and efficacy of the proposed framework.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
