Deep Architectures for Content Moderation and Movie Content Rating

Movies2Scenes: Using Movie Metadata to Learn Scene Representation

Understanding scenes in movies is crucial for a variety of applications such as video moderation, search, and recommendation. However, labeling individual scenes is a time-consuming process. In contrast, movie level metadata (e.g., genre, synopsis, etc.) regularly gets produced as part of the film production process, and is therefore significantly more commonly available. In this work, we propose a novel contrastive learning approach that uses movie metadata to learn a general-purpose scene representation. Specifically, we use movie metadata to define a measure of movie similarity, and use it …

Edge Data Based Trailer Inception Probabilistic Matrix Factorization for Context-Aware Movie Recommendation

The rapid growth of edge data generated by mobile devices and applications deployed at the edge of the network has exacerbated the problem of information overload. As an effective way to alleviate information overload, recommender system can improve the quality of various services by adding application data generated by users on edge devices, such as visual and textual information, on the basis of sparse rating data. The visual information in the movie trailer is a significant part of the movie recommender system. However, due to the complexity of visual information extraction, data sparsity c…

Latent Factor Modeling of Users Subjective Perception for Stereoscopic 3D Video Recommendation

Numerous stereoscopic 3D movies are released every year to theaters and created large revenues. Despite the improvement in stereo capturing and 3D video post-production technology, stereoscopic artifacts which cause viewer discomfort continue to appear even in high-budget films. Existing automatic 3D video quality measurement tools can detect distortions in stereoscopic images or videos, but they fail to consider the viewer's subjective perception of those artifacts, and how these distortions affect their choices. In this paper, we introduce a novel recommendation system for stereoscopic 3D mo…

SenBen: 用于可解释内容审核的敏感场景图

SenBen: Sensitive Scene Graphs for Explainable Content Moderation

提出SenBen基准和紧凑学生模型，通过多任务训练和词汇平衡策略实现敏感内容的空间定位与可解释性，在场景图生成上超越多数VLM。

视听情感分析在电影情感弧线学习中的应用

Audio-Visual Sentiment Analysis for Learning Emotional Arcs in Movies

故事可以有巨大的力量--不仅对娱乐有用，它们还可以激活我们的兴趣，调动我们的行动。一个故事与观众产生共鸣的程度可能部分反映在它带给观众的情感旅程中。在本文中，我们使用机器学习方法来构建电影中的情感弧，计算弧族，并证明某些弧预测观众参与的能力。该系统适用于好莱坞电影和网络上找到的高质量短片。我们首先使用深度卷积神经网络进行音频和视觉情感分析。这些模型在新的和现有的大规模数据集上进行训练，之后它们可以用于计算单独的音频和视觉情感弧。然后，我们众包从弧中的高点和低点提取的30秒视频剪辑的注释，以评估系统的微观精度，精度根据系统预测和注释者评级之间的极性一致性来衡量。这些注释还用于组合音频和视觉预测。接下来，我们看看宏观层面上的表征的电影调查是否存在“普遍形状”的情感弧线。特别是，我们开发了一种聚类方法来发现不同类别的情感弧。最后，我们展示了短网络视频的样本语料库，某些情感弧是视频收到的评论数量的统计显著预测。这些结果表明，通过我们的方法学习的情感弧成功地代表了视频故事的宏观方面，这些方面推动了观众的参与。这种机器理解可以用来预测观众对视频故事的反应，最终提高我们作为讲故事者相互交流的能力。