语言大模型 / LLM

gzip Predicts Data-dependent Scaling Laws

Rohan Pandey

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

Comments 9 pages, 9 figures

Foundation Policies with Hilbert Representations

Seohong Park, Tobias Kreiman, Sergey Levine

专题命中 预训练与数据 ：prompting(abstract)；分类 cs.AI、cs.LG

Comments ICML 2024

Towards Retrieval-Augmented Architectures for Image Captioning

Sara Sarto, Marcella Cornia, Lorenzo Baraldi, Alessandro Nicolosi, Rita Cucchiara

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments ACM Transactions on Multimedia Computing, Communications and Applications (2024)

MoSECroT: Model Stitching with Static Word Embeddings for Crosslingual Zero-shot Transfer

Haotian Ye, Yihong Liu, Chunlan Ma, Hinrich Schütze

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

MIMIC: Masked Image Modeling with Image Correspondences

Kalyani Marathe, Mahtab Bigverdi, Nishat Khan, Tuhin Kundu, Patrick Howe, Sharan Ranjit S, Anand Bhattad, Aniruddha Kembhavi, Linda G. Shapiro, Ranjay Krishna

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Self-supervised vision-langage alignment of deep learning representations for bone X-rays analysis

Alexandre Englebert, Anne-Sophie Collin, Olivier Cornu, Christophe De Vleeschouwer

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.AI

InsightNet: Structured Insight Mining from Customer Feedback

Sandeep Sricharan Mukku, Manan Soni, Jitenkumar Rana, Chetan Aggarwal, Promod Yenigalla, Rashmi Patange, Shyam Mohan

专题命中 预训练与数据 ：LLM(abstract)；分类 cs.CL、cs.AI

Comments EMNLP 2023

A systematic investigation of learnability from single child linguistic input

Yulu Qin, Wentao Wang, Brenden M. Lake

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

Comments Please cite as Qin, Y., Wang, W., and Lake, B. M. (2024). A systematic investigation of learnability from single child linguistic input. In Proceedings of the 46th Annual Conference of the Cognitive Science Society

Explaining Text Similarity in Transformer Models

Alexandros Vasileiou, Oliver Eberle

专题命中 预训练与数据 ：foundation model(abstract)；分类 cs.CL、cs.LG

Comments Accepted to NAACL 2024

Leveraging tropical reef, bird and unrelated sounds for superior transfer learning in marine bioacoustics

Ben Williams, Bart van Merriënboer, Vincent Dumoulin, Jenny Hamer, Eleni Triantafillou, Abram B. Fleishman, Matthew McKown, Jill E. Munger, Aaron N. Rice, Ashlee Lillis, Clemency E. White, Catherine A. D. Hobbs, Tries B. Razak, Kate E. Jones, Tom Denton

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 5 figures

Identifying Narrative Patterns and Outliers in Holocaust Testimonies Using Topic Modeling

Maxim Ifergan, Renana Keydar, Omri Abend, Amit Pinchevski

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments 9 pages, 7 figures, LREC-COLING 2024

A New Linear Scaling Rule for Private Adaptive Hyperparameter Optimization

Ashwinee Panda, Xinyu Tang, Saeed Mahloujifar, Vikash Sehwag, Prateek Mittal

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments ICML 2024

The Dark Side of the Language: Pre-trained Transformers in the DarkNet

Leonardo Ranaldi, Aria Nourbakhsh, Arianna Patrizi, Elena Sofia Ruzzetti, Dario Onorati, Francesca Fallucchi, Fabio Massimo Zanzotto

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

Journal ref 2023.ranlp-1.102

Self-supervised Pre-training of Text Recognizers

Martin Kišš, Michal Hradiš

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 18 pages, 6 figures, 4 tables, accepted to ICDAR24

Never Train from Scratch: Fair Comparison of Long-Sequence Models Requires Data-Driven Priors

Ido Amos, Jonathan Berant, Ankit Gupta

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.LG

LANISTR: Multimodal Learning from Structured and Unstructured Data

Sayna Ebrahimi, Sercan O. Arik, Yihe Dong, Tomas Pfister

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Self-Supervised Learning for User Localization

Ankan Dash, Jingyi Gu, Guiling Wang, Nirwan Ansari

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

What do Transformers Know about Government?

Jue Hou, Anisia Katinskaia, Lari Kotilainen, Sathianpong Trangcasanchai, Anh-Duc Vu, Roman Yangarber

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

Automated Text Mining of Experimental Methodologies from Biomedical Literature

Ziqing Guo

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

ScaleFold: Reducing AlphaFold Initial Training Time to 10 Hours

Feiwen Zhu, Arkadiusz Nowaczynski, Rundong Li, Jie Xin, Yifei Song, Michal Marcinkiewicz, Sukru Burc Eryilmaz, Jun Yang, Michael Andersch

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Unlocking Bias Detection: Leveraging Transformer-Based Models for Content Analysis

Shaina Raza, Oluwanifemi Bamgbose, Veronica Chatrath, Shardul Ghuge, Yan Sidyakin, Abdullah Y Muaad

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.AI

Comments Accepted in IEEE Transactions on Computational Social Systems

ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

Otto Brookes, Majid Mirmehdi, Hjalmar Kuhl, Tilo Burghardt

专题命中 预训练与数据 ：language model(abstract)；分类 cs.AI、cs.LG

SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery

Konstantin Klemmer, Esther Rolf, Caleb Robinson, Lester Mackey, Marc Rußwurm

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

ComCLIP: Training-Free Compositional Image and Text Matching

Kenan Jiang, Xuehai He, Ruize Xu, Xin Eric Wang

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.CL、cs.AI

Distilled Self-Critique of LLMs with Synthetic Data: a Bayesian Perspective

Victor Gallego

专题命中 预训练与数据 ：LLM(abstract)；分类 cs.CL、cs.LG

Comments Accepted to ICLR 2024 (TinyPapers track)

Journal ref The Second Tiny Papers Track at ICLR 2024

PejorativITy: Disambiguating Pejorative Epithets to Improve Misogyny Detection in Italian Tweets

Arianna Muti, Federico Ruggeri, Cagri Toraman, Lorenzo Musetti, Samuel Algherini, Silvia Ronchi, Gianmarco Saretto, Caterina Zapparoli, Alberto Barrón-Cedeño

专题命中 预训练与数据 ：prompting(abstract)；分类 cs.CL、cs.AI

Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP

Reza Abbasi, Mohammad Samiei, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah

专题命中 预训练与数据 ：language model(abstract)；分类 cs.CL、cs.LG

Comments Oral accepted at OODCV 2023(http://www.ood-cv.org)

Exploring CausalWorld: Enhancing robotic manipulation via knowledge transfer and curriculum learning

Xinrui Wang, Yan Jin

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

LSTTN: A Long-Short Term Transformer-based Spatio-temporal Neural Network for Traffic Flow Forecasting

Qinyao Luo, Silu He, Xing Han, Yuhan Wang, Haifeng Li

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG

Comments 15 pages, 10 figures, 6 tables

Journal ref Knowledge-Based Systems 2024

Deep Active Learning: A Reality Check

Edrina Gashi, Jiankang Deng, Ismail Elezi

专题命中 预训练与数据 ：pretraining(abstract)；分类 cs.AI、cs.LG