arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2407.12438cs.LG
收藏

多模态数据的语义感知表示用于数据入库:文献综述

Semantic-Aware Representation of Multi-Modal Data for Data Ingress: A Literature Review

University of Gothenburg(哥德堡大学);Chalmers University of Technology(查尔姆斯理工大学)

Pierre Lamart, Yinan Yu, Christian Berger

AI总结:

本文献综述探讨了从单模态、多模态和跨模态数据中提取嵌入的语义感知技术,以提升数据湖中的信息检索能力,并总结了各类模态的最新应用进展。

摘要:

机器学习(ML)正不断渗透到越来越多的应用领域。生成式人工智能,如大型语言模型(LLMs),也被广泛采用以处理文本、图像、音频和视频等多模态数据。虽然当前的趋势是使用越来越大的数据集进行训练,但在工业界,高效管理这些数据已成为一项重大的实际挑战——数据量翻倍并不意味着数据质量翻倍。恰恰相反,更重要的是认识到底层数据湖中固有质量和多样性的理解正成为应用特定ML以及微调基础模型日益增长的挑战。此外,从不断扩展的数据湖中进行信息检索(IR)因时间序列数据固有的时间维度而变得复杂,必须考虑该维度以确定其语义价值。本研究聚焦于不同的语义感知技术,以从单模态、多模态和跨模态数据中提取嵌入,从而增强不断增长的数据湖中的信息检索能力。收集了相关文章,以总结针对三种不同数据模态类别的嵌入应用的最先进技术信息。

英文摘要:

Machine Learning (ML) is continuously permeating a growing amount of application domains. Generative AI such as Large Language Models (LLMs) also sees broad adoption to process multi-modal data such as text, images, audio, and video. While the trend is to use ever-larger datasets for training, managing this data efficiently has become a significant practical challenge in the industry-double as much data is certainly not double as good. Rather the opposite is important since getting an understanding of the inherent quality and diversity of the underlying data lakes is a growing challenge for application-specific ML as well as for fine-tuning foundation models. Furthermore, information retrieval (IR) from expanding data lakes is complicated by the temporal dimension inherent in time-series data which must be considered to determine its semantic value. This study focuses on the different semantic-aware techniques to extract embeddings from mono-modal, multi-modal, and cross-modal data to enhance IR capabilities in a growing data lake. Articles were collected to summarize information about the state-of-the-art techniques focusing on applications of embedding for three different categories of data modalities.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑