Architectural Knowledge Management (AKM) involves the organized handling of information related to architectural decisions and design within a project or organization. An essential artifact of AKM is the Architecture Decision Records (ADR), which documents key design decisions. ADRs are documents that capture decision context, decision made and various aspects related to a design decision, thereby promoting transparency, collaboration, and understanding. Despite their benefits, ADR adoption in software development has been slow due to challenges like time constraints and inconsistent uptake. Recent advancements in Large Language Models (LLMs) may help bridge this adoption gap by facilitating ADR generation. However, the effectiveness of LLM for ADR generation or understanding is something that has not been explored. To this end, in this work, we perform an exploratory study that aims to investigate the feasibility of using LLM for the generation of ADRs given the decision context. In our exploratory study, we utilize GPT and T5-based models with 0-shot, few-shot, and fine-tuning approaches to generate the Decision of an ADR given its Context. Our results indicate that in a 0-shot setting, state-of-the-art models such as GPT-4 generate relevant and accurate Design Decisions, although they fall short of human-level performance. Additionally, we observe that more cost-effective models like GPT-3.5 can achieve similar outcomes in a few-shot setting, and smaller models such as Flan-T5 can yield comparable results after fine-tuning. To conclude, this exploratory study suggests that LLM can generate Design Decisions, but further research is required to attain human-level generation and establish standardized widespread adoption.
MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair
University of British Columbia, Department of Computer Science（不列颠哥伦比亚大学计算机科学系）；École de technologie supérieure, Department of Software and IT Engineering（高等技术学院软件与信息技术工程系）
Efficient Fine-Tuning Methods for Portuguese Question Answering: A Comparative Study of PEFT on BERTimbau and Exploratory Evaluation of Generative LLMs
Institute of Science and Technology（科学技术研究院）；Federal University of São Paulo (UNIFESP)（圣保罗联邦大学）
TestAgent: Automatic Benchmarking and Exploratory Interaction for Evaluating LLMs in Vertical Domains
Shanghai Key Laboratory of Computer Software Testing and Evaluating, Shanghai Development Center of Computer Software Technology（上海软件测试与评估关键实验室、上海计算机软件技术发展中心）；Shanghai Normal University（上海师范大学）；Shanghai Artificial Intelligence Lab（上海人工智能实验室）；Shanghai University（上海大学）；Shanghai Jiao Tong University（上海交通大学）
Text-to-Layout: A Generative Workflow for Drafting Architectural Floor Plans Using LLMs
Department of Civil and Environmental Engineering, University of Houston（土木与环境工程系，休斯顿大学）；Center for Transportation Research, The University of Texas at Austin（交通研究中心，德克萨斯大学奥斯汀分校）；Department of Information Science Technology, University of Houston（信息科学与技术系，休斯顿大学）
Large Language Models as Code Executors: An Exploratory Study
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Xinjiang University（新疆大学）；University of Melbourne（墨尔本大学）；Tsinghua University（清华大学）；Shanghai Jiao Tong University（上海交通大学）；IBM Research（IBM研究院）；Nantong University（南通大学）；Alibaba（阿里巴巴）
A Novel Hierarchical Integration Method for Efficient Model Merging in Medical LLMs
Deakin Cyber Research（德肯网络研究）；Innovation Centre Deakin University / Universal Higher Education Melbourne, Australia（德肯大学创新中心/通用高等教育墨尔本，澳大利亚）；School of IT（信息科技学院）；Engineering Melbourne Institute of Technology Melbourne, Australia（工程墨尔本技术学院墨尔本，澳大利亚）；Innovation Centre Deakin University Geelong, Australia（德肯大学创新中心吉朗，澳大利亚）
Harnessing LLMs for Automated Video Content Analysis: An Exploratory Workflow of Short Videos on Depression
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Institute of High Performance Computing (IHPC), A*STAR（高性能计算研究所（A*STAR））；Pennsylvania State University（宾夕法尼亚州立大学）；Clark University（克拉克大学）；Columbia University（哥伦比亚大学）
Mining for Species, Locations, Habitats, and Ecosystems from Scientific Papers in Invasion Biology: A Large-Scale Exploratory Study with Large Language Models
TIB Leibniz Information Centre for Science and Technology（TIB莱布尼茨科学与技术信息中心）；University of Colorado Boulder（科罗拉多大学博尔德分校）；Friedrich Schiller University Jena（弗里德里希·席勒·耶拿大学）；Bielefeld University（比勒费尔德大学）；University College London（伦敦大学学院）
DataClawBench: An Agent Benchmark for Exploratory Real-World Financial Data Analysis
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Software Engineering, Sun Yat-sen University（中山大学软件学院）；Lingnan College, Sun Yat-sen University（中山大学岭南学院）
Automated Data Visualization from Natural Language via Large Language Models: An Exploratory Study
Huazhong University of Science and Technology（华中科技大学）；Chongqing University（重庆大学）；University of New South Wales（新南威尔士大学）；University of Technology Sydney（悉尼科技大学）
Large Language Model-Based Agents for Automated Research Reproducibility: An Exploratory Study in Alzheimer's Disease
Biomedical Informatics & Data Science, Johns Hopkins University（约翰霍普金斯大学生物医学信息学与数据科学）；Biomedical & Health Informatics, University of Washington（华盛顿大学生物医学与健康信息学）
DenseMLLM: Standard Multimodal LLMs for Dense Prediction
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong, China（香港科技大学电子与计算机工程系）；Tencent, Youtu-Lab, China（腾讯优图实验室）
