Computer programming textbooks and software documentations often contain flowcharts to illustrate the flow of an algorithm or procedure. Modern OCR engines often tag these flowcharts as graphics and ignore them in further processing. In this paper, we work towards making flowchart images machine-interpretable by converting them to executable Python codes. To this end, inspired by the recent success in natural language to code generation literature, we present a novel transformer-based framework, namely FloCo-T5. Our model is well-suited for this task,as it can effectively learn semantics, structure, and patterns of programming languages, which it leverages to generate syntactically correct code. We also used a task-specific pre-training objective to pre-train FloCo-T5 using a large number of logic-preserving augmented code samples. Further, to perform a rigorous study of this problem, we introduce theFloCo dataset that contains 11,884 flowchart images and their corresponding Python codes. Our experiments show promising results, and FloCo-T5 clearly outperforms related competitive baselines on code generation metrics. We make our dataset and implementation publicly available.
Explainable and Interpretable Diabetic Retinopathy Classification Based on Neural-Symbolic Learning
Massachusetts General Hospital(麻省总医院);Harvard Medical School(哈佛医学院);Singapore Eye Research Institute(新加坡眼科研究所);Duke-NUS Medical School(杜克-新加坡国立大学医学院);Institute for Molecular and Clinical Ophthalmology(分子与临床眼科研究所);National University of Singapore(新加坡国立大学)
VLMs Guided Interpretable Decision Making for Autonomous Driving
Department of Computer Science, Tulane University(路易斯安那大学计算机科学系);Qualcomm(高通公司);Department of Industrial and Systems Engineering, North Carolina State University(北卡罗来纳州立大学工业与系统工程系)
Prompt-CAM: Making Vision Transformers Interpretable for Fine-Grained Analysis
The Ohio State University(俄亥俄州立大学);University of Tsukuba(筑波大学);Virginia Tech(弗吉尼亚理工大学);Princeton University(普林斯顿大学);Rensselaer Polytechnic Institute(伦斯勒理工学院)
DECIS: Dual-Evidence Corrective Verification for Interpretable Strabismus Diagnostic Decision-Making
School of Information and Software Engineering, University of Electronic Science and Technology of China(电子科技大学信息与软件工程学院);Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China(电子科技大学深圳高等研究院);Joint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong(汕头大学·香港中文大学联合汕头国际眼科中心);School of Artificial Intelligence, Guangzhou City Polytechnic(广州城市职业学院人工智能学院);Medical College, Shantou University(汕头大学医学院);College of Engineering, Shantou University(汕头大学工学院);Department of Ophthalmology, Xinhua Hospital Affiliated to Shanghai Jiaotong University School of Medicine(上海交通大学医学院附属新华医院眼科);Shenzhen Loop Area Institute(深圳河套学院)
A Modified VGG19-Based Framework for Accurate and Interpretable Real-Time Bone Fracture Detection
East West University(东西大学);Islamic University of Technology(伊斯兰技术大学);Ahsanullah University of Science and Technology(阿赫萨努拉科技大学);Bangladesh Agricultural University(孟加拉国农业大学);Brac University(布拉克大学)
Towards a Transparent and Interpretable AI Model for Medical Image Classifications
School of Artificial Intelligence, Guilin University of Electronic Technology(桂林电子科技大学人工智能学院);The Laboratory for Imagery, Vision and Artificial Intelligence, Ecole de Technologie Superieure(图像、视觉与人工智能实验室,超技术学院);College of Medicine, Taibah University(塔伊巴大学医学院);Prince Mohammed Bin Abdulaziz Hospital, Ministry of National Guard Health Affairs(穆罕默德·本·阿卜杜勒阿齐兹医院,国家守卫健康事务部)