In this paper, we propose a new setting for generating product descriptions from images, augmented by marketing keywords. It leverages the combined power of visual and textual information to create descriptions that are more tailored to the unique features of products. For this setting, previous methods utilize visual and textual encoders to encode the image and keywords and employ a language model-based decoder to generate the product description. However, the generated description is often inaccurate and generic since same-category products have similar copy-writings, and optimizing the overall framework on large-scale samples makes models concentrate on common words yet ignore the product features. To alleviate the issue, we present a simple and effective Multimodal In-Context Tuning approach, named ModICT, which introduces a similar product sample as the reference and utilizes the in-context learning capability of language models to produce the description. During training, we keep the visual encoder and language model frozen, focusing on optimizing the modules responsible for creating multimodal in-context references and dynamic prompts. This approach preserves the language generation prowess of large language models (LLMs), facilitating a substantial increase in description diversity. To assess the effectiveness of ModICT across various language model scales and types, we collect data from three distinct product categories within the E-commerce domain. Extensive experiments demonstrate that ModICT significantly improves the accuracy (by up to 3.3% on Rouge-L) and diversity (by up to 9.4% on D-5) of generated results compared to conventional methods. Our findings underscore the potential of ModICT as a valuable tool for enhancing automatic generation of product descriptions in a wide range of applications. Code is at: https://github.com/HITsz-TMG/Multimodal-In-Context-Tuning
Captions Speak Louder than Images: Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Data
Department of Computer Science and Engineering, The Ohio State University（计算机科学与工程系，俄亥俄州立大学）；Translational Data Analytics Institute, The Ohio State University（转化数据分析研究所，俄亥俄州立大学）；Department of Biomedical Informatics, The Ohio State University（生物医学信息学系，俄亥俄州立大学）
EcomMMMU: Strategic Utilization of Visuals for Robust Multimodal E-commerce Models
Department of Computer Science and Engineering, The Ohio State University（俄亥俄州立大学计算机科学与工程系）；Translational Data Analytics Institute, The Ohio State University（俄亥俄州立大学转化数据分析研究所）；Department of Biomedical Informatics, The Ohio State University（俄亥俄州立大学生物医学信息学系）
EVADE-Bench: Multimodal Benchmark for Evaluating and Enhancing Evasive Content Detection
SIAT, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Alibaba Group（阿里巴巴集团）；University of New South Wales（新南威尔士大学）
MENTOR: Efficient Multimodal-Conditioned Tuning for Autoregressive Vision Generation Models
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；Tsinghua University（清华大学）；Peking University（北京大学）；Microsoft（微软公司）
MLAN: Language-Based Instruction Tuning Preserves and Transfers Knowledge in Multimodal Language Models
Washington University in St. Louis（华盛顿大学圣路易斯分校）；The University of British Columbia（不列颠哥伦比亚大学）；Google Research（谷歌研究）；Virginia Tech（弗吉尼亚理工大学）；University of California, Davis（加州大学戴维斯分校）；Sony AI（索尼人工智能）；University of California, Berkeley（加州大学伯克利分校）
A survey on fairness of large language models in e-commerce: progress, application, and challenge
Carnegie Mellon University（卡内基梅隆大学）；San Francisco State University（旧金山州立大学）；Columbia University（哥伦比亚大学）；University of Southern California（南加州大学）；Virginia Tech（弗吉尼亚理工大学）；Khoury College of Computer Sciences, Northeastern University（东北大学科里计算机科学学院）；New York University（纽约大学）
NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMs
College of Intelligent Robotics and Advanced Manufacturing, Fudan University（复旦大学智能机器人与先进制造学院）；Zhongguancun Academy（中关村学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Tianjin University（天津大学）；Nankai University（南开大学）
M$^2$PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning
Rochester Institute of Technology（罗切斯特理工学院）；U.S. Naval Research Laboratory（美国海军研究实验室）；Harvard Medical School（哈佛医学院）；ByteDance（字节跳动）；Meta AI；Meituan（美团）；University of Science and Technology of China（中国科学技术大学）；Shanghai Academy of AI for Science（上海人工智能科学研究院）；Fudan University（复旦大学）；University of Missouri - Kansas City（密苏里大学堪萨斯城分校）；University of California - Davis（加利福尼亚大学戴维斯分校）
Multimodal Detection of Fake Reviews using BERT and ResNet-50
College of Business Administration Kansas State University Manhattan, USA（商学院学院 华盛顿州立大学 曼哈顿 美国）；Department of Computer Science Kansas State University Manhattan, USA（计算机科学系 华盛顿州立大学 曼哈顿 美国）
针对单模态检测无法捕捉跨模态语义不一致的问题，提出集成 BERT 文本与 ResNet-50 图像特征的多模态虚假评论检测框架，在 21,142 张图像数据集上 F1 达 0.934，有效识别图文不一致的虚假评论。
FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Data
Nanjing University of Science and Technology（南京理工大学）；National University of Singapore（新加坡国立大学）；Institute of High-Performance Computing, A*STAR（新加坡高性能计算研究院（新加坡科研局下属机构））
LLaVAC: Fine-tuning LLaVA as a Multimodal Sentiment Classifier
iApp Technology Co., Ltd.（iApp科技有限公司）；Panasonic Research and Development on Artificial Intelligence (AI)（松下人工智能研发中心）；Sirindhorn International Institute of Technology, Thammasat University（泰国法政大学诗琳通国际理工学院）
An Empirical Study on Parameter-Efficient Fine-Tuning for MultiModal Large Language Models
Waseda University（早稻田大学）；School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；School of Computer Science and Informatics, Cardiff University（卡迪夫大学计算机科学与信息学院）
