The advancements of Large Language Models (LLMs) have decentralized the responsibility for the transparency of AI usage. Specifically, LLM users are now encouraged or required to disclose the use of LLM-generated content for varied types of real-world tasks. However, an emerging phenomenon, users' secret use of LLM, raises challenges in ensuring end users adhere to the transparency requirement. Our study used mixed-methods with an exploratory survey (125 real-world secret use cases reported) and a controlled experiment among 300 users to investigate the contexts and causes behind the secret use of LLMs. We found that such secretive behavior is often triggered by certain tasks, transcending demographic and personality differences among users. Task types were found to affect users' intentions to use secretive behavior, primarily through influencing perceived external judgment regarding LLM usage. Our results yield important insights for future work on designing interventions to encourage more transparent disclosure of the use of LLMs or other AI technologies.
MemCut: Every Secret Leaves a Trail in the Long-Term Memory of LLM Agents
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));University of Science and Technology of China(中国科学技术大学);The Hong Kong University of Science and Technology(香港科技大学)
Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like One
Nanyang Technological University(南洋理工大学);Sea AI Lab(Sea AI实验室);Xi’an Jiaotong University(西安交通大学);Centre for Frontier AI Research (CFAR), A*STAR(新加坡科技研究局前沿人工智能研究中心)
API Secrets Should Never Become Tokens in the LLM's Vocabulary: A Threat Analysis of API Credential Handling in LLM Agent Systems and an Empirical Evaluation of a Vault-Mediated Execution Boundary
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院,清华大学);Harbin Institute of Technology(哈尔滨工业大学);Pengcheng Laboratory(鹏城实验室)
Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory
University of Washington(华盛顿大学);Allen Institute for Artificial Intelligence(艾伦人工智能研究所);Carnegie Mellon University(卡内基梅隆大学);National University of Singapore(新加坡国立大学)
Your Pre-trained LLM is Secretly an Unsupervised Confidence Calibrator
Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Department of Computer Sciences, University of Wisconsin-Madison(计算机科学系,威斯康星大学麦迪逊分校)
Dataset and Lessons Learned from the 2024 SaTML LLM Capture-the-Flag Competition
ETH Zurich(苏黎世联邦理工学院);CISPA Helmholtz Center for Information Security(CISPA亥姆霍兹信息安全中心);West University of Timisoara(蒂米什瓦拉西部大学);New York University(纽约大学);Microsoft(微软公司);Zurich University of the Arts(苏黎世艺术大学)
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs
Red Hat AI Innovation(红帽人工智能创新中心);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);IBM Research(IBM研究院);Massachusetts Institute of Technology(麻省理工学院);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)
Insider Knowledge: How Much Can RAG Systems Gain from Evaluation Secrets?
University of New Hampshire(新罕布什尔大学);University of Pennsylvania(宾夕法尼亚大学);Human Language Technology Center of Excellence, Johns Hopkins University(约翰霍普金斯大学人类语言技术卓越中心)
Multi-PA: A Multi-perspective Benchmark on Privacy Assessment for Large Vision-Language Models
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(人工智能安全国家重点实验室,计算技术研究所,中国科学院);University of Chinese Academy of Sciences(中国科学院大学)
Large Language Models as Master Key: Unlocking the Secrets of Materials Science with GPT
University of New South Wales(新南威尔士大学);GreenDynamics Pty. Ltd(绿动力私人有限公司);City University of Hong Kong(香港城市大学);University of Sydney(悉尼大学);DARE ARC Training Centre in Data Analytics for Resources and Environments(DARE ARC资源与环境数据分析培训中心)
Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models
UCL Centre for Artificial Intelligence, University College London, UK(伦敦大学学院人工智能中心,大学学院伦敦);School of Computing and Communications, Lancaster University, UK(兰卡斯特大学计算机与通讯学院);Department of Informatics, University of Sussex, UK(苏塞克斯大学信息学院);Centre for Psychedelic Research and Centre for Complexity Science, Imperial College London, UK(伦敦帝国学院迷幻研究与复杂科学中心);Centre for Eudaimonia and Human Flourishing, University of Oxford, UK(牛津大学幸福与人类繁荣中心)
How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study
Griffith University(格里菲斯大学);Wake Forest University(威克森林大学);Nanyang Technological University(南洋理工大学);University of New South Wales(新南威尔士大学);Zhejiang Sci-Tech University(浙江科技学院);The University of Tokyo(东京大学);University of Alberta(阿尔伯塔大学);Independent Researcher(独立研究者)