Enabling machines to understand structured visuals like slides and user interfaces is essential for making them accessible to people with disabilities. However, achieving such understanding computationally has required manual data collection and annotation, which is time-consuming and labor-intensive. To overcome this challenge, we present a method to generate synthetic, structured visuals with target labels using code generation. Our method allows people to create datasets with built-in labels and train models with a small number of human-annotated examples. We demonstrate performance improvements in three tasks for understanding slides and UIs: recognizing visual elements, describing visual content, and classifying visual content types.
AI-Generated Lecture Slides for Improving Slide Element Detection and Retrieval
Indian Institute of Technology(印度理工学院);Sardar Vallabhbhai National Institute of Technology(萨达尔·瓦拉巴希国家理工学院);CVIT, International Institute of Information Technology(信息技术国际学院)
HistomicsML2.0: Fast interactive machine learning for whole slide imaging data
Weisberg Division of Computer Science, Marshall University(马歇尔大学魏斯伯格计算机科学分部);Kitware, Inc.(Kitware公司);Emory University School of Medicine(埃默里大学医学院);National Cancer Institute(美国国家癌症研究所);Cairo University(开罗大学);Northwestern University Feinberg School of Medicine(西北大学费恩伯格医学院)
Scalable, Trustworthy Generative Model for Virtual Multi-Staining from H&E Whole Slide Images
Sorbonne Université(索邦大学);CNRS(法国国家科学研究中心);Inserm(法国国家健康与医学研究院);AP-HP(巴黎公共医疗集团);Inria(法国国家信息与自动化研究所);Paris Brain Institute-ICM(巴黎大脑研究所-脑研究所);Paris Cité University(巴黎城市大学);Necker-Enfants Malades University Hospitals(内克尔-儿童马尔代夫大学医院);Centre de recherche sur l’inflammation(炎症研究中心);Robert Debré University Hospitals(罗伯特德布雷大学医院)
VAPO: End-to-end Slide-Enhanced Speech Recognition with Omni-modal Large Language Models
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence(北京交通数据挖掘与具身智能重点实验室);Unisound AI Technology Co., Ltd.(Unisound人工智能技术有限公司);State Key Laboratory of AI Safety, Beijing(人工智能安全国家重点实验室)
FedWSIDD: Federated Whole Slide Image Classification via Dataset Distillation
Union Hospital, Tongji Medical College, Huazhong University of Science and Technology(华中科技大学同济医学院附属同济医院);Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University(澳大利亚健康创新研究所健康信息中心,麦觉大学)
Dataset Distillation for Histopathology Image Classification
Australian Institute of Health Innovation (AIHI), Macquarie University(澳大利亚麦考瑞大学澳大利亚健康创新研究所);School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Computer Science, Peking University(北京大学计算机学院);School of Computer Science and Engineering, University of New South Wales(新南威尔士大学计算机科学与工程学院)
Federated Distillation for Whole Slide Image via Gaussian-Mixture Feature Alignment and Curriculum Integration
School of Computer Science, Peking University, Beijing, China(北京大学计算机科学系);Center for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW 2113, Australia(健康信息学中心,澳大利亚健康创新研究所,麦考利大学,悉尼,NSW 2113,澳大利亚);School of Data Science, University of Virginia, Charlottesville, VA, USA(数据科学学院,弗吉尼亚大学,夏洛特维尔,VA,美国)
Collection Space Navigator: An Interactive Visualization Interface for Multidimensional Datasets
Tallinn University(塔林大学);School of Digital Technologies, Tallinn University(塔林大学数字技术学院);Baltic Film, Media and Arts School, Tallinn University(塔林大学波罗的海电影、媒体与艺术学院);School of Humanities, Tallinn University(塔林大学人文学院)
针对多维数据相关视觉数字工件集合探索分析的挑战,本文提出开源的Collection Space Navigator(CSN)交互式可视化工具,结合二维投影与多维过滤器,支持缩放、投影转换等操作,助力用户探索、整理集合并反馈见解。
Learning to Infer User Interface Attributes from Images
ETH Zürich(苏黎世联邦理工学院)
我们探索了一个新的学习领域,以推断用户界面属性,帮助开发人员自动化的过程中,用户界面的实现。具体地说,给定一个由设计师创建的输入图像,我们学习推断它的实现,当渲染时,在视觉上看起来与输入图像相同。为了实现这一点,我们采用黑盒渲染引擎和它支持的一组属性(例如,颜色、边界半径、阴影或文本属性),使用它来生成合适的合成训练数据集,然后训练专门的神经模型来预测每个属性值。为了提高像素级精度,我们还使用模仿学习来训练神经策略,该神经策略通过学习计算原始图像和渲染图像在其属性空间中的相似性来细化预测属性值,而不是基于像素值的差异。我们实例化了我们的方法来推断Android Button属性值,并在由真实世界的Google Play Store应用程序组成的数据集上实现了92.5%的准确率。