Recent advancements in large language models have demonstrated enhanced capabilities in visual reasoning tasks by employing additional encoders for aligning different modalities. While the Q-Former has been widely used as a general encoder for aligning several modalities including image, video, audio, and 3D with large language models, previous works on its efficient training and the analysis of its individual components have been limited. In this work, we investigate the effectiveness of parameter efficient fine-tuning (PEFT) the Q-Former using InstructBLIP with visual reasoning benchmarks ScienceQA and IconQA. We observe that applying PEFT to the Q-Former achieves comparable performance to full fine-tuning using under 2% of the trainable parameters. Additionally, we employ AdaLoRA for dynamic parameter budget reallocation to examine the relative importance of the Q-Former's sublayers with 4 different benchmarks. Our findings reveal that the self-attention layers are noticeably more important in perceptual visual-language reasoning tasks, and relative importance of FFN layers depends on the complexity of visual-language patterns involved in tasks. The code is available at https://github.com/AttentionX/InstructBLIP_PEFT.
Unlocking LLM Safeguards for Low-Resource Languages via Reasoning and Alignment with Minimal Training Data
Guangdong University of Foreign Studies(广东外语外贸大学);Guangzhou Key Laboratory of Multilingual Intelligent Processing(广州多语智能处理重点实验室);University of Pittsburgh(匹兹堡大学)
Does Reasoning Preserve Alignment? On the Trustworthiness of Large Reasoning Models
University of Colorado Boulder(科罗拉多大学博尔德分校);University of Central Florida(中佛罗里达大学);University of Maryland College Park(马里兰大学帕克分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校)
Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学计算机科学与技术系);Alibaba Group, China(阿里巴巴集团)
MR-Align: Meta-Reasoning Informed Factuality Alignment for Large Reasoning Models
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Zhongguancun Academy(中关村学院);University of Chinese Academy of Sciences(中国科学院大学);Tencent(腾讯)
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University(浙江大学Hangzhou全球科技创新中心);Polytechnic Institute, Zhejiang University(浙江大学 polytechnic 院);School of Software Technology, Zhejiang University(浙江大学软件技术学院);ZJU-UIUC Institute, Zhejiang University(浙江大学UIUC研究院)
TACR: A Table-alignment-based Cell-selection and Reasoning Model for Hybrid Question-Answering
Tokyo Institute of Technology(东京工业大学);Nanyang Technological University(南洋理工大学);Microsoft Research Asia(微软亚洲研究院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
Reasoning over Boundaries: Enhancing Specification Alignment via Test-time Deliberation
School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China(上海交通大学人工智能学院);Shanghai Artificial Intelligence Laboratory, Shanghai, China(上海人工智能实验室);University of Science and Technology of China, Hefei, Anhui, China(中国科学技术大学);The Chinese University of Hong Kong, Hong Kong, China(香港中文大学);University of Illinois Urbana-Champaign, Urbana, IL, USA(伊利诺伊大学厄巴纳-香槟分校)
SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning
School of Software Engineering, Sun Yat-sen University(中山大学软件工程学院);School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院);Merchants Union Consumer Finance Company Limited(招商局联融消费金融股份有限公司)
DARA: Decomposition-Alignment-Reasoning Autonomous Language Agent for Question Answering over Knowledge Graphs
Technical University of Darmstadt(达姆施塔特工业大学);Hessian Center for AI (hessian.AI)(黑森人工智能中心);Queen’s University(女王大学);Ingenuity Labs Research Institute(Ingenuity 实验室研究院)
本文提出 DARA 框架,通过高层任务分解与低层任务落地将问题解析为形式化查询,并用少量推理轨迹微调 LLM,在零样本 KGQA 基准上超过 GPT-4 上下文学习方案并媲美枚举排序方法。
Mining or Synthesis? Rethinking Exploration Efficiency in Iterative Alignment of Mathematical Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China(哈尔滨工业大学深圳研究院);Huawei Large Model Data Technology Lab(华为大模型数据技术实验室);Huawei Multimodal Model Lab(华为多模态模型实验室);Department of Statistics and Data Science, Tsinghua University, Beijing, China(清华大学统计与数据科学系)
MERLIN: Multi-Stage Curriculum Alignment for Multilingual Encoder-LLM Integration in Cross-Lingual Reasoning
University of Toronto(多伦多大学);Mila-Quebec AI Institute, McGill University(魁北克AI研究所,麦吉尔大学);OntarioTech University(安大略技术大学);Saarland University(萨尔兰州立大学);Canada CIFAR AI Chair(加拿大CIFAR人工智能主席)
AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));State University of New York at Buffalo(纽约州立大学布法罗分校);Huawei International, Singapore(新加坡华为国际)
UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
University of California San Diego(加利福尼亚大学圣迭戈分校);University of Southern California(南加利福尼亚大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Carnegie Mellon University(卡内基梅隆大学)
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Institute for Artificial Intelligence, Peking University(北京大学人工智能研究院);Microsoft Research(微软研究院);University of Michigan(密歇根大学);Shanghai Jiao Tong University(上海交通大学);CUHKSZ(香港中文大学(深圳));THU(清华大学)