语言大模型 / LLM

AlcLaM: Arabic Dialectal Language Model

Murtadha Ahmed, Saghir Alfasly, Bo Wen, Jamaal Qasem, Mohammed Ahmed, Yunfeng Liu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted by ArabicNLP 2024, presented in ACL 2024

A Language Modeling Approach to Diacritic-Free Hebrew TTS

Amit Roth, Arnon Turetzky, Yossi Adi

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted at Interspeech24

Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining

Florian Tramèr, Gautam Kamath, Nicholas Carlini

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments Full and unabridged version of paper ICML 2024

Ancient Korean Archive Translation: Comparison Analysis on Statistical phrase alignment, LLM in-context learning, and inter-methodological approach

Sojung Lucia Kim, Taehong Jang, Joonmo Ahn

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL

Comments ACL2024 submitted

Self-supervised Pretraining for Partial Differential Equations

Varun Madhavan, Amal S Sebastian, Bharath Ramsundar, Venkatasubramanian Viswanathan

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

BAMBINO-LM: (Bilingual-)Human-Inspired Continual Pretraining of BabyLM

Zhewen Shen, Aditya Joshi, Ruey-Cheng Chen

专题命中 预训练与数据 ：pretraining(title)；language model(abstract)；分类 cs.CL

Comments 5 pages + references; Selected at CMCL Workshop@ACL 2024

Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units

Bolaji Yusuf, Jan "Honza" Černocký, Murat Saraçlar

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments Interspeech 2024. KWS code at: https://github.com/bolajiy/golden-retriever; AUD code at https://github.com/beer-asr/beer/tree/master/recipes/hshmm

Compact Speech Translation Models via Discrete Speech Units Pretraining

Tsz Kin Lam, Alexandra Birch, Barry Haddow

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments 11 pages, accepted at IWSLT 2024

InstructPatentGPT: Training patent language models to follow instructions with human feedback

Jieh-Sheng Lee

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments 41 pages. Artif Intell Law (2024)

Reducing Privacy Risks in Online Self-Disclosures with Language Models

Yao Dou, Isadora Krsek, Tarek Naous, Anubha Kabra, Sauvik Das, Alan Ritter, Wei Xu

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted at ACL 2024

Protecting Privacy Through Approximating Optimal Parameters for Sequence Unlearning in Language Models

Dohyun Lee, Daniel Rim, Minseok Choi, Jaegul Choo

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Accepted to ACL2024 findings

Pretraining Strategy for Neural Potentials

Zehua Zhang, Zijie Li, Amir Barati Farimani

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

GFM4MPM: Towards Geospatial Foundation Models for Mineral Prospectivity Mapping

Angel Daruna, Vasily Zadorozhnyy, Georgina Lukoczki, Han-Pang Chiu

专题命中 预训练与数据 ：foundation model(title)；pretraining(abstract)；分类 cs.LG

Comments 12 pages, 16 figures, 7 tables

Taiyi-Diffusion-XL: Advancing Bilingual Text-to-Image Generation with Large Vision-Language Model Support

Xiaojun Wu, Dixiang Zhang, Ruyi Gan, Junyu Lu, Ziwei Wu, Renliang Sun, Jiaxing Zhang, Pingjian Zhang, Yan Song

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments Taiyi-Diffusion-XL Tech Report

Benchmarking Zero-Shot Recognition with Vision-Language Models: Challenges on Granularity and Specificity

Zhenlin Xu, Yi Zhu, Tiffany Deng, Abhay Mittal, Yanbei Chen, Manchen Wang, Paolo Favaro, Joseph Tighe, Davide Modolo

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI

Comments CVPR2024 MMFM workshop

Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models

Shimin Chen, Yitian Yuan, Shaoxiang Chen, Zequn Jie, Lin Ma

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI

The Curse of Popularity: Popular Entities have Catastrophic Side Effects when Deleting Knowledge from Language Models

Ryosuke Takahashi, Go Kamoda, Benjamin Heinzerling, Keisuke Sakaguchi, Kentaro Inui

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

MLIP: Efficient Multi-Perspective Language-Image Pretraining with Exhaustive Data Utilization

Yu Zhang, Qi Zhang, Zixuan Gong, Yiwei Shi, Yepeng Liu, Duoqian Miao, Yang Liu, Ke Liu, Kun Yi, Wei Fan, Liang Hu, Changwei Wang

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.AI

Comments ICML 2024

Position: Graph Foundation Models are Already Here

Haitao Mao, Zhikai Chen, Wenzhuo Tang, Jianan Zhao, Yao Ma, Tong Zhao, Neil Shah, Mikhail Galkin, Jiliang Tang

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

Comments 23 pages, 2 figures

Evaluating Vision-Language Models on Bistable Images

Artemis Panagopoulou, Coby Melkin, Chris Callison-Burch

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.AI

FaultFormer: Pretraining Transformers for Adaptable Bearing Fault Classification

Anthony Zhou, Amir Barati Farimani

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 10 pages, 6 figures

Journal ref in IEEE Access, vol. 12, pp. 70719-70728, 2024

A Study on Self-Supervised Pretraining for Vision Problems in Gastrointestinal Endoscopy

Edward Sanderson, Bogdan J. Matuszewski

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Analysis of Atom-level pretraining with Quantum Mechanics (QM) data for Graph Neural Networks Molecular property models

Jose Arjona-Medina, Ramil Nugmanov

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Comments 6 pages + 11 Supplement Materials

ReactXT: Understanding Molecular "Reaction-ship" via Reaction-Contextualized Molecule-Text Pretraining

Zhiyuan Liu, Yaorui Shi, An Zhang, Sihang Li, Enzhi Zhang, Xiang Wang, Kenji Kawaguchi, Tat-Seng Chua

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

Comments ACL 2024 Findings, 9 pages

PRISM: A Multi-Modal Generative Foundation Model for Slide-Level Histopathology

George Shaikovski, Adam Casson, Kristen Severson, Eric Zimmermann, Yi Kan Wang, Jeremy D. Kunz, Juan A. Retamero, Gerard Oakley, David Klimstra, Christopher Kanan, Matthew Hanna, Michal Zelechowski, Julian Viret, Neil Tenenholtz, James Hall, Nicolo Fusi, Razik Yousfi, Peter Hamilton, William A. Moye, Eugene Vorontsov, Siqi Liu, Thomas J. Fuchs

专题命中 预训练与数据 ：foundation model(title,abstract)；分类 cs.LG

TransliCo: A Contrastive Learning Framework to Address the Script Barrier in Multilingual Pretrained Language Models

Yihong Liu, Chunlan Ma, Haotian Ye, Hinrich Schütze

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Comments ACL 2024

Incubating Text Classifiers Following User Instruction with Nothing but LLM

Letian Peng, Jingbo Shang

专题命中 预训练与数据 ：LLM(title,abstract)；分类 cs.CL

CLIP with Quality Captions: A Strong Pretraining for Vision Tasks

Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Oncel Tuzel

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.LG

Investigating the 'Autoencoder Behavior' in Speech Self-Supervised Models: a focus on HuBERT's Pretraining

Valentin Vielzeuf

专题命中 预训练与数据 ：pretraining(title,abstract)；分类 cs.CL

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts

Rabindra Lamsal, Maria Rodriguez Read, Shanika Karunasekera

专题命中 预训练与数据 ：language model(title,abstract)；分类 cs.CL

Journal ref Knowledge-Based Systems, 111916 (2024)