Visually-situated language is ubiquitous -- sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms. Perhaps due to this diversity, previous work has typically relied on domain-specific recipes with limited sharing of the underlying data, model architectures, and objectives. We present Pix2Struct, a pretrained image-to-text model for purely visual language understanding, which can be finetuned on tasks containing visually-situated language. Pix2Struct is pretrained by learning to parse masked screenshots of web pages into simplified HTML. The web, with its richness of visual elements cleanly reflected in the HTML structure, provides a large source of pretraining data well suited to the diversity of downstream tasks. Intuitively, this objective subsumes common pretraining signals such as OCR, language modeling, image captioning. In addition to the novel pretraining strategy, we introduce a variable-resolution input representation and a more flexible integration of language and vision inputs, where language prompts such as questions are rendered directly on top of the input image. For the first time, we show that a single pretrained model can achieve state-of-the-art results in six out of nine tasks across four domains: documents, illustrations, user interfaces, and natural images.
Universidade da Coruña（拉科鲁尼亚大学）；CITIC, Ciencias de la Computación y Tecnologías de la Información (CC&TI)（CITIC，计算机科学与信息技术（CC&TI））；University of Copenhagen, Department of Computer Science（哥本哈根大学计算机科学系）；Google Research, Berlin（谷歌研究院柏林分院）；Google Research（谷歌研究院）；Berlin（柏林（注：此处仅保留机构名本身，Berlin为城市名，按要求不提取额外信息，故仅提取Google Research））
Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank
Paul G. Allen School of Computer Science & Engineering, University of Washington（华盛顿大学保罗·G·艾伦计算机科学与工程学院）；Department of Linguistics, University of Washington（华盛顿大学语言学系）；Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Large Language Model Department, Tencent（腾讯大语言模型部）；Nankai University（南开大学）
研究改进轻量级端到端OCR视觉语言模型。基于HunyuanOCR-1.0架构，用DFlash提升效率，提出Agentic Data Flow增强能力，在多任务表现出色，还将发布模型权重和代码。
Adaptive Markup Language Generation for Contextually-Grounded Visual Document Understanding
CUHK MMLab（香港中文大学MML实验室）；vivo AI Lab（vivo人工智能实验室）；CPII under InnoHK（创新工场下的CPII）；Shanghai AI Lab & Shenzhen Institute of Advanced Technology, CAS（上海人工智能实验室及深圳先进技术研究所，中国科学院）
SciBERT: A Pretrained Language Model for Scientific Text
Allen Institute for Artificial Intelligence（艾伦人工智能研究所）
为科学领域的NLP任务获取大规模注释数据是具有挑战性和昂贵的。我们发布了SciBERT，一种基于BERT的预训练语言模型（Devlin et al.，2018年），以解决缺乏高质量，大规模的标记科学数据。SciBERT利用大型多领域科学出版物语料库的无监督预训练来提高下游科学NLP任务的性能。我们评估了一套任务，包括序列标记，句子分类和依存句法分析，从各种科学领域的数据集。我们证明了统计上显着的改进，超过BERT，并实现了新的国家的最先进的结果，这些任务。代码和预训练模型可在https://github.com/allenai/scibert/上获得。
Upstage；KAIST（韩国科学技术院）；NAVER AI Lab（NAVER AI 实验室）；Seoul National University（首尔大学）；Yonsei University（延世大学）；Scatter Lab；KyungHee University（庆熙大学）；Kakao Enterprise；Sogang University（西江大学）；Hanbat National University（韩巴国立大学）；Riiid AI Research（Riiid AI 研究院）；New York University（纽约大学）
