Neural approaches to program synthesis and understanding have proliferated widely in the last few years; at the same time graph based neural networks have become a promising new tool. This work aims to be the first empirical study comparing the effectiveness of natural language models and static analysis graph based models in representing programs in deep learning systems. It compares graph convolutional networks using different graph representations in the task of program embedding. It shows that the sparsity of control flow graphs and the implicit aggregation of graph convolutional networks cause these models to perform worse than naive models. Therefore it concludes that simply augmenting purely linguistic or statistical models with formal information does not perform well due to the nuanced nature of formal properties introducing more noise than structure for graph convolutional networks.
Learning Syntactic Program Transformations from Examples
UFCG(坎皮纳联邦大学);UC Berkeley(加州大学伯克利分校);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);University of Washington(华盛顿大学);Microsoft(微软);University of Colorado Boulder(科罗拉多大学博尔德分校)
Functional Consistency of LLM Code Embeddings: A Self-Evolving Data Synthesis Framework for Benchmarking
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);School of Software Engineering, Sun Yat-Sen University(中山大学软件工程学院);School of Artificial Intelligence, Sun Yat-Sen University(中山大学人工智能学院);Key Laboratory of Sustainable Tourism Smart Assessment Technology, Ministry of Culture and Tourism of China(文化和旅游部可持续旅游智能评估技术重点实验室);Department of Computer Science, City University of Hong Kong(香港城市大学计算机科学系)
Beyond Basic Specifications? A Systematic Study of Logical Constructs in LLM-based Specification Generation
School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);National Key Laboratory of Science and Technology on Information System Security, AMS(信息系统安全国家重点实验室,AMS);Unit 32092 of PLA(中国人民解放军第32092部队);School of Cyber Engineering, Xidian University(西安电子科技大学网络空间安全学院);Systems Engineering Institute, AMS(系统工程研究院,AMS)
AST-Probe: Recovering abstract syntax trees from hidden representations of pre-trained language models
University of Murcia(穆尔西亚大学);University of Montreal(蒙特利尔大学);Institute for Clarity in Documentation(文档清晰化研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗克库尔中心);Rajiv Gandhi University(拉吉夫·甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)