Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT for Mining Insights at Scale
University of Oulu, Center for Applied Computing, Faculty of Information Technology and Electrical Engineering(奥卢大学,应用计算中心,信息科技与电气工程学院);University of Jyväskylä, Software and Communications Engineering, Faculty of Information Technology(耶夫尼耶斯基大学,软件与通信工程,信息科技学院)
Jonas Oppenlaender, Joonas Hämäläinen
备注:
Accepted in International Journal of Human-Computer Interaction; 45 pages, 7 figures, 4 tables
Large language models (LLMs) are increasingly used for analytical tasks, yet their effectiveness in real-world applications remains underexamined, partly due to the opacity of proprietary models. We evaluate ChatGPT (GPT-3.5 and GPT-4) on the practical task of extracting research challenges from a large scholarly corpus in Human-Computer Interaction (HCI). Using a two-step approach, we first apply GPT-3.5 to extract candidate challenges from the 879 papers in the 2023 ACM CHI Conference proceedings, then use GPT-4 to select the most relevant challenges per paper. This process yielded 4,392 research challenges across 113 topics, which we organized through topic modeling and present in an interactive visualization. We compare the identified challenges with previously established HCI grand challenges and the United Nations Sustainable Development Goals, finding both strong alignment in areas such as ethics and accessibility, and gaps in areas such as human-AI collaboration. A task-specific evaluation with human raters confirmed near-perfect agreement that the extracted statements represent plausible research challenges (\k{appa} = 0.97). The two-step approach proved cost-effective at approximately US$50 for the full corpus, suggesting that LLMs offer a practical means for qualitative text analysis at scale, particularly for prototyping research ideas and examining corpora from multiple analytical perspectives.
Bioinformatics and Biomedical Informatics with ChatGPT: Year One Review
West Virginia University(西弗吉尼亚大学);University of Nebraska-Lincoln(内布拉斯加大学林肯分校);Arizona State University(亚利桑那州立大学);Biodesign Institute, Arizona State University(亚利桑那州立大学生物设计研究所)
Using LLMs in Software Design: An Empirical Study of GitHub and A Practitioner Survey
School of Computer Science, Wuhan University(武汉大学计算机学院);School of Computer Science, Central China Normal University(中央财经大学计算机学院);School of Computing Technologies, RMIT University(皇家墨尔本理工大学计算技术学院);M3S Research Unit, University of Oulu(奥卢大学M3S研究单位);School of Computer Science, Nanjing University of Science(南京理工大学计算机学院)
Prompt4Vis: Prompting Large Language Models with Example Mining and Schema Filtering for Tabular Data Visualization
The Hong Kong Polytechnic University(香港理工大学);University of Chinese Academy of Sciences(中国科学院大学);The Hong Kong University of Science and Technology(香港科技大学);WeBank Co., Ltd(微众银行);University of Liverpool(利物浦大学)
针对自然语言转数据可视化查询性能不足的问题,提出基于大语言模型与上下文学习的 Prompt4Vis 框架,通过多目标示例挖掘和模式过滤模块显著提升了生成性能,在 NVBench 数据集上大幅超越现有 SOTA 方法。
Image and Data Mining in Reticular Chemistry Using GPT-4V
University of California, Berkeley(加州大学伯克利分校);Kavli Energy Nanoscience Institute(卡夫利能源纳米科学研究所);Bakar Institute of Digital Materials for the Planet(巴卡尔行星数字材料研究所);Stanford University(斯坦福大学);King Abdulaziz City for Science and Technology(阿卜杜勒阿齐兹国王科技城)
BioPars: A Pretrained Biomedical Large Language Model for Persian Biomedical Text Mining
Department of Computer Science, Faculty of Education, University of Kufa(计算机科学系,教育学院,科菲大学);Department of Computer Science, Florida State University(计算机科学系,佛罗里达州立大学);Department of Computer Engineering, Velayat University(计算机工程系,瓦莱yat大学);Department of Mechanical Engineering, Amirkabir University of Technology(机械工程系,阿米卡比尔技术大学);Department of Computer Engineering, Ferdowsi University of Mashhad(计算机工程系,法尔德大学)
A comprehensive study of LLM-based argument classification: from LLAMA through GPT-4o to Deepseek-R1
Faculty of Computer Science, Electronics, and Telecommunications AGH University of Krakow(计算机科学、电子与电信学院AGH克拉科夫大学);Faculty of Humanities AGH University of Krakow(人文学院AGH克拉科夫大学);Institute of Mathematics University of Silesia(数学学院桑特学院)
Annotation Guidelines-Based Knowledge Augmentation: Towards Enhancing Large Language Models for Educational Text Classification
National Engineering Research Center of Educational Big Data, Faculty of Artificial Intelligence in Education, Central China Normal University(国家教育大数据工程研究中心,教育人工智能学院,中央财经大学)