Recent advancements in large language models (LLMs) have revolutionized code intelligence by improving programming productivity and alleviating challenges faced by software developers. To further improve the performance of LLMs on specific code intelligence tasks and reduce training costs, researchers reveal a new capability of LLMs: in-context learning (ICL). ICL allows LLMs to learn from a few demonstrations within a specific context, achieving impressive results without parameter updating. However, the rise of ICL introduces new security vulnerabilities in the code intelligence field. In this paper, we explore a novel security scenario based on the ICL paradigm, where attackers act as third-party ICL agencies and provide users with bad ICL content to mislead LLMs outputs in code intelligence tasks. Our study demonstrates the feasibility and risks of such a scenario, revealing how attackers can leverage malicious demonstrations to construct bad ICL content and induce LLMs to produce incorrect outputs, posing significant threats to system security. We propose a novel method to construct bad ICL content called DICE, which is composed of two stages: Demonstration Selection and Bad ICL Construction, constructing targeted bad ICL content based on the user query and transferable across different query inputs. Ultimately, our findings emphasize the critical importance of securing ICL mechanisms to protect code intelligence systems from adversarial manipulation.
Mitigating Many-shot Jailbreak Attacks with One Single Demonstration
Zhejiang University(浙江大学);Nanyang Technological University(南洋理工大学);Sun Yat-sen University(中山大学);University of Science and Technology of China(中国科学技术大学);Virginia Tech(弗吉尼亚理工大学)
Adversarial Demonstration Attacks on Large Language Models
University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Arizona State University(亚利桑那州立大学);Tokyo Metropolitan University(东京都立大学);University of Southern California(南加州大学);University of Michigan - Ann Arbor(密歇根大学安娜堡分校);University of California, Davis(加州大学戴维斯分校)
National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学);Fudan University(复旦大学);Cyber Special Ops-R&D, NCS Group(NCS集团网络特种作战研发部)
A Demonstration of Self-Adaptive Jamming Attack Detection in AI/ML Integrated O-RAN
Commonwealth Cyber Initiative, Virginia Tech, VA, USA(弗吉尼亚理工学院公共网络安全倡议);NextG Wireless Lab, North Carolina State University, NC, USA(北卡罗来纳州立大学NextG无线实验室);George Mason University, VA, USA(乔治·梅森大学)
An Automated Framework for Extracting Reachable Attack Chains from Cyber Threat Intelligence Reports
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies(广东新型安全情报技术重点实验室);School of Computer Science and Technology(计算机科学与技术学院);Harbin Institute of Technology(哈尔滨工业大学);New Network Research Department(新网络研究部);Peng Cheng Laboratory(鹏城实验室);Great Bay University(大湾区大学)
EI-MTD:Moving Target Defense for Edge Intelligence against Adversarial Attacks
Zhejiang University of Science and Technology(浙江科技大学);Zhejiang University(浙江大学);Guangzhou University(广州大学);Hangzhou Hikvision Digital Technology Co., Ltd.(杭州海康威视数字技术股份有限公司)
Mitigating Attacks on Artificial Intelligence-based Spectrum Sensing for Cellular Network Signals
University of Stavanger(斯塔万格大学);Old Dominion University(奥尔多明尼安大学);Virginia Commonwealth University(弗吉尼亚联邦大学);Norwegian University of Science and Technology(挪威科技大学)
From Threat Reports to Continuous Threat Intelligence: A Comparison of Attack Technique Extraction Methods from Textual Artifacts
North Carolina State University(北卡罗来纳州立大学)
网络威胁形势在不断演变。因此,持续监控和共享威胁情报已成为组织的优先事项。网络安全供应商发布的威胁报告包含以非结构化文本格式编写的攻击策略、技术和程序(TTP)的详细描述。从这些报告中提取TTP有助于网络安全从业人员和研究人员学习和适应不断变化的攻击,并规划威胁缓解。研究人员在文献中提出了TTP提取方法,然而,并非所有这些提出的方法都相互比较或与基线进行比较。\textit{The goal of this study is to aid cybersecurity researchers and practitioners choose attack technique extraction methods for monitoring and sharing threat intelligence by comparing the underlying methods from the TTP extraction studies in the literature.}在这项工作中,我们确定了十个现有的TTP提取研究的文献,并实施五种方法,从十项研究。我们发现两种方法,基于词频逆文档频率(TFIDF)和潜在语义索引(LSI),优于其他三种方法的F1得分分别为84%和83%。我们观察到所有方法的性能在F1分数下降的情况下,类标签呈指数增长。我们还实施和评估过采样策略,以减轻类不平…
Has the Two-Decade-Old Prophecy Come True? Artificial Bad Intelligence Triggered by Merely a Single-Bit Flip in Large Language Models
Henan Key Laboratory of Network Cryptography Technology, Information Engineering University(河南网络密码技术重点实验室,信息工程大学);State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所,信息安全国家重点实验室);Zhejiang University(浙江大学)
Security and Privacy for Artificial Intelligence: Opportunities and Challenges
The Cyber Security Cooperative Research Centre (CSCRC)(网络安全合作研究中心(CSCRC));Penn State(宾夕法尼亚州立大学);RMIT University(皇家墨尔本理工大学);The University of Technology Sydney(悉尼科技大学)
Attacking Automatic Video Analysis Algorithms: A Case Study of Google Cloud Video Intelligence API
University of Washington(华盛顿大学);Worcester Polytechnic Institute(伍斯特理工学院)
由于互联网上视频数据的增长,自动视频分析得到了学术界以及Facebook,Twitter和Google等公司的大量关注。在本文中,我们研究了视频分析算法在对抗环境中的鲁棒性。具体来说,我们提出了有针对性的攻击两个基本类的视频分析算法,即视频分类和镜头检测。我们表明,对手可以巧妙地操纵视频的方式,人类观察者会感知到原始视频的内容,但视频分析算法将返回对手的期望输出。然后,我们将攻击应用于最近发布的Google Cloud Video Intelligence API。API获取视频文件并返回视频标签(视频中的对象)、镜头变化(视频中的场景变化)和镜头标签(随时间推移的视频事件描述)。通过实验,我们表明,该API生成视频和镜头标签,只处理视频的每一秒的第一帧。因此,攻击者可以通过以每秒一帧的速率周期性地将图像插入到视频中来欺骗API仅输出她想要的视频和镜头标签。我们还表明,API返回的镜头变化的模式可以主要恢复的算法,比较连续帧的直方图。基于我们的等效模型,我们开发了一种稍微修改视频帧的方法,以欺骗API生成我们想要的镜头变化模式。我们对不同的视频进行了广泛的实验,并表明我们的攻击在具有不同特征的视频中始终是成功的。最后,我们提出了引入随机性的视频分析算法作为我们的攻击对策。
A Streamlit-based Artificial Intelligence Trust Platform for Next-Generation Wireless Networks
Old Dominion University(奥多明尼昂大学);University of Stavanger(斯塔万格大学);Virginia Commonwealth University(弗吉尼亚联邦大学);Norwegian University of Science and Technology(挪威科技大学);eKare, Inc.(eKare公司)
随着人工智能(AI)方法在下一代网络(NextG)中的快速发展和集成,AI算法在频谱使用、带宽、延迟和安全性方面为NextG提供了显著优势。NextG的一个关键特征是AI的集成,即,基于自监督算法的自学习架构,以提高网络的性能。一个安全的人工智能驱动的结构也有望保护NextG网络免受网络攻击。然而,AI本身可能会受到攻击,即,模型中毒是攻击者的目标,它会导致网络安全违规。本文提出了一个使用Streamlit for NextG网络的AI信任平台,允许研究人员评估,防御,认证和验证他们的AI模型和应用程序,以抵御规避,中毒,提取和干扰等对抗性威胁。