大模型推理能力

ScholarChemQA: Unveiling the Power of Language Models in Chemical Research Question Answering

Xiuying Chen, Tairan Wang, Taicheng Guo, Kehan Guo, Juexiao Zhou, Haoyang Li, Mingchen Zhuge, Jürgen Schmidhuber, Xin Gao, Xiangliang Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments 14 pages

BiasAlert: A Plug-and-play Tool for Social Bias Detection in LLMs

Zhiting Fan, Ruizhe Chen, Ruiling Xu, Zuozhu Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction

Hang Hua, Jing Shi, Kushal Kafle, Simon Jenni, Daoan Zhang, John Collomosse, Scott Cohen, Jiebo Luo

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ECCV 2024

X-Former: Unifying Contrastive and Reconstruction Learning for MLLMs

Sirnam Swetha, Jinyu Yang, Tal Neiman, Mamshad Nayeem Rizve, Son Tran, Benjamin Yao, Trishul Chilimbi, Mubarak Shah

专题命中 推理评测 ：reasoning(abstract)；分类 cs.LG

Comments Accepted at ECCV2024

BEAF: Observing BEfore-AFter Changes to Evaluate Hallucination in Vision-language Models

Moon Ye-Bin, Nam Hyeon-Woo, Wonseok Choi, Tae-Hyun Oh

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted at ECCV 2024. [Project Pages] https://beafbench.github.io/

Learning-From-Mistakes Prompting for Indigenous Language Translation

You-Cheng Liao, Chen-Jui Yu, Chi-Yi Lin, He-Feng Yun, Yen-Hsiang Wang, Hsiao-Min Li, Yao-Chung Fan

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

A LLM Benchmark based on the Minecraft Builder Dialog Agent Task

Chris Madge, Massimo Poesio

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Emergent Visual-Semantic Hierarchies in Image-Text Representations

Morris Alper, Hadar Averbuch-Elor

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Accepted to ECCV 2024. Project page: https://hierarcaps.github.io/

LokiLM: Technical Report

Justin Kiefel, Shrey Shah

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Iris: An AI-Driven Virtual Tutor For Computer Science Education

Patrick Bassner, Eduard Frankford, Stephan Krusche

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.AI

Comments Published in Proceedings of the 2024 Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2024), Pages 534 - 540, July 8--10, 2024, Milan, Italy

CRAFT: Extracting and Tuning Cultural Instructions from the Wild

Bin Wang, Geyu Lin, Zhengyuan Liu, Chengwei Wei, Nancy F. Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Aceepted to ACL 2024 Workshop - C3NLP (Workshop on Cross-Cultural Considerations in NLP)

Video-STaR: Self-Training Enables Video Instruction Tuning with Any Supervision

Orr Zohar, Xiaohan Wang, Yonatan Bitton, Idan Szpektor, Serena Yeung-Levy

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments Project page: https://orrzohar.github.io/projects/video-star/

Temporal Grounding of Activities using Multimodal Large Language Models

Young Chol Song

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Progress or Regress? Self-Improvement Reversal in Post-training

Ting Wu, Xuefeng Li, Pengfei Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context

Zishan Gu, Changchang Yin, Fenglin Liu, Ping Zhang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

ACR: A Benchmark for Automatic Cohort Retrieval

Dung Ngoc Thai, Victor Ardulov, Jose Ulises Mena, Simran Tiwari, Gleb Erofeev, Ramy Eskander, Karim Tarabishy, Ravi B Parikh, Wael Salloum

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer

Hele-Andra Kuulmets, Taido Purason, Agnes Luhtaru, Mark Fishel

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Journal ref Findings of the Association for Computational Linguistics: NAACL 2024, pages 3309-3325

STBench: Assessing the Ability of Large Language Models in Spatio-Temporal Analysis

Wenbin Li, Di Yao, Ruibo Zhao, Wenjie Chen, Zijie Xu, Chengxue Luo, Chang Gong, Quanliang Jing, Haining Tan, Jingping Bi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation

Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Multiple-Choice Questions are Efficient and Robust LLM Evaluators

Ziyin Zhang, Zhaokun Jiang, Lizhen Xu, Hongkun Hao, Rui Wang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments data at https://github.com/Geralt-Targaryen/MC-Evaluation

Marathon: A Race Through the Realm of Long Context with Large Language Models

Lei Zhang, Yunshui Li, Ziqiang Liu, Jiaxi yang, Junhao Liu, Longze Chen, Run Luo, Min Yang

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

S$^3$HQA: A Three-Stage Approach for Multi-hop Text-Table Hybrid Question Answering

Fangyu Lei, Xiang Li, Yifan Wei, Shizhu He, Yiming Huang, Jun Zhao, Kang Liu

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments ACL 2023

Exploring Factual Entailment with NLI: A News Media Study

Guy Mor-Lan, Effi Levi

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments Presented at *SEM 2024

Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models

Jesse Atuhurra, Iqra Ali, Tatsuya Hiraoka, Hidetaka Kamigaito, Tomoya Iwakura, Taro Watanabe

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Harmful Speech Detection by Language Models Exhibits Gender-Queer Dialect Bias

Rebecca Dorn, Lee Kezar, Fred Morstatter, Kristina Lerman

专题命中 推理评测 ：chain-of-thought(abstract)；分类 cs.CL

MedOdyssey: A Medical Domain Benchmark for Long Context Evaluation Up to 200K Tokens

Yongqi Fan, Hongli Sun, Kui Xue, Xiaofan Zhang, Shaoting Zhang, Tong Ruan

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs

Yuxuan Qiao, Haodong Duan, Xinyu Fang, Junming Yang, Lin Chen, Songyang Zhang, Jiaqi Wang, Dahua Lin, Kai Chen

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification

Gregor Geigle, Radu Timofte, Goran Glavaš

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks

Jack Gallifant, Shan Chen, Pedro Moreira, Nikolaj Munch, Mingye Gao, Jackson Pond, Leo Anthony Celi, Hugo Aerts, Thomas Hartvigsen, Danielle Bitterman

专题命中 推理评测 ：reasoning(abstract)；分类 cs.CL

Comments submitted for review, total 15 pages

RS-GPT4V: A Unified Multimodal Instruction-Following Dataset for Remote Sensing Image Understanding

Linrui Xu, Ling Zhao, Wang Guo, Qiujun Li, Kewang Long, Kaiqi Zou, Yuhan Wang, Haifeng Li

专题命中 推理评测 ：reasoning(abstract)；分类 cs.AI

Comments 14 pages, 6 figures, 4 tables