语言大模型 / LLM

Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection

Gaetan Lopez Latouche, Marc-André Carbonneau, Ben Swanson

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments Submitted to EMNLP 2024

Repurformer: Transformers for Repurposing-Aware Molecule Generation

Changhun Lee, Gyumin Lee

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 12 pages, 8 figures, conference

Weighted Grouped Query Attention in Transformers

Sai Sena Chinnakonduru, Astarag Mohapatra

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Improving Text-To-Audio Models with Synthetic Captions

Zhifeng Kong, Sang-gil Lee, Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, Rafael Valle, Soujanya Poria, Bryan Catanzaro

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

Lost in Translation: The Algorithmic Gap Between LMs and the Brain

Tommaso Tosato, Pascal Jr Tikeng Notsawo, Saskia Helbling, Irina Rish, Guillaume Dumas

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Natural Language Can Help Bridge the Sim2Real Gap

Albert Yu, Adeline Foote, Raymond Mooney, Roberto Martín-Martín

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.LG

Comments To appear in RSS 2024. Project website at https://robin-lab.cs.utexas.edu/lang4sim2real/

Semantic Compositions Enhance Vision-Language Contrastive Learning

Maxwell Aladago, Lorenzo Torresani, Soroush Vosoughi

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Jina CLIP: Your CLIP Model Is Also Your Text Retriever

Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mohammad Kalim Akram, Joan Fontanals Martínez, Saahil Ognawala, Susana Guzman, Maximilian Werk, Nan Wang, Han Xiao

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.AI

Comments 4 pages, MFM-EAI@ICML2024

Data curation via joint example selection further accelerates multimodal learning

Talfan Evans, Nikhil Parthasarathy, Hamza Merzic, Olivier J. Henaff

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Main text: 9 pages, 5 figures, 3 tables, 1 algorithm. Appendix: 7 pages, 5 figures, 1 table, 2. algorithm

Mind's Eye: Image Recognition by EEG via Multimodal Similarity-Keeping Contrastive Learning

Chi-Sheng Chen, Chun-Shu Wei

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 19 pages, 14 figures

MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions

Kai Zhang, Yi Luan, Hexiang Hu, Kenton Lee, Siyuan Qiao, Wenhu Chen, Yu Su, Ming-Wei Chang

专题命中 预训练与数据 ：foundation model(abstract)；分类 cs.CL、cs.AI

Comments ICML 2024 (Oral); Project Website: https://open-vision-language.github.io/MagicLens/

Learning Action-based Representations Using Invariance

Max Rudolph, Caleb Chuck, Kevin Black, Misha Lvovsky, Scott Niekum, Amy Zhang

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Published at the Reinforcement Learning Conference 2024

Outliers Dimensions that Disrupt Transformers Are Driven by Frequency

Giovanni Puccetti, Anna Rogers, Aleksandr Drozd, Felice Dell'Orletta

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments To appear in Findings of EMNLP 2022

Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?

Mark Ibrahim, David Klindt, Randall Balestriero

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning

Mayur Patidar, Riya Sawhney, Avinash Singh, Biswajit Chatterjee, Mausam, Indrajit Bhattacharya

专题命中 预训练与数据 ：LLM(abstract)；分类 cs.CL、cs.AI

Comments ACL-2024 camera-ready version

CTC-based Non-autoregressive Textless Speech-to-Speech Translation

Qingkai Fang, Zhengrui Ma, Yan Zhou, Min Zhang, Yang Feng

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.AI

Comments ACL 2024 Findings

Meta Learning Text-to-Speech Synthesis in over 7000 Languages

Florian Lux, Sarina Meyer, Lyonel Behringer, Frank Zalkow, Phat Do, Matt Coler, Emanuël A. P. Habets, Ngoc Thang Vu

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.LG

Comments accepted at Interspeech 2024

CountCLIP -- [Re] Teaching CLIP to Count to Ten

Harshvardhan Mestha, Tejas Agrawal, Karan Bania, Shreyas V, Yash Bhisikar

专题命中 预训练与数据 ：language model(abstract)；分类 cs.AI、cs.LG

Scaling and evaluating sparse autoencoders

Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, Jeffrey Wu

专题命中 预训练与数据 ：language model(abstract)；分类 cs.AI、cs.LG

Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems

David T. Hoffmann, Simon Schrodi, Jelena Bratulić, Nadine Behrmann, Volker Fischer, Thomas Brox

专题命中 预训练与数据 ：language model(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ICML 2024

FILS: Self-Supervised Video Feature Prediction In Semantic Language Space

Mona Ahmadian, Frank Guerin, Andrew Gilbert

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Error-preserving Automatic Speech Recognition of Young English Learners' Language

Janick Michot, Manuela Hürlimann, Jan Deriu, Luzia Sauer, Katsiaryna Mlynchyk, Mark Cieliebak

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted at ACL 2024 Main Conference

Landscape-Aware Growing: The Power of a Little LAG

Stefani Karp, Nikunj Saunshi, Sobhan Miryoosefi, Sashank J. Reddi, Sanjiv Kumar

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.LG

Self-Pro: A Self-Prompt and Tuning Framework for Graph Neural Networks

Chenghua Gong, Xiang Li, Jianxiang Yu, Cheng Yao, Jiaqi Tan, Chengcheng Yu

专题命中 预训练与数据 ：prompting(abstract)；分类 cs.AI、cs.LG

Comments Accepted at ECML-PKDD 2024

Enhancing Low-Resource Relation Representations through Multi-View Decoupling

Chenghao Fan, Wei Wei, Xiaoye Qu, Zhenyi Lu, Wenfeng Xie, Yu Cheng, Dangyang Chen

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted to AAAI 2024

JADS: A Framework for Self-supervised Joint Aspect Discovery and Summarization

Xiaobo Guo, Jay Desai, Srinivasan H. Sengamedu

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.AI

Comments preprint

A Graph is Worth $K$ Words: Euclideanizing Graph using Pure Transformer

Zhangyang Gao, Daize Dong, Cheng Tan, Jun Xia, Bozhen Hu, Stan Z. Li

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Multi-level Interaction Modeling for Protein Mutational Effect Prediction

Yuanle Mo, Xin Hong, Bowen Gao, Yinjun Jia, Yanyan Lan

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

WeatherFormer: A Pretrained Encoder Model for Learning Robust Weather Representations from Small Datasets

Adib Hasan, Mardavij Roozbehani, Munther Dahleh

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Masked Face Recognition with Generative-to-Discriminative Representations

Shiming Ge, Weijia Guo, Chenyu Li, Junzheng Zhang, Yong Li, Dan Zeng

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments Accepted by International Conference on Machine Learning 2024