Social media bot detection has always been an arms race between advancements in machine learning bot detectors and adversarial bot strategies to evade detection. In this work, we bring the arms race to the next level by investigating the opportunities and risks of state-of-the-art large language models (LLMs) in social bot detection. To investigate the opportunities, we design novel LLM-based bot detectors by proposing a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities. To illuminate the risks, we explore the possibility of LLM-guided manipulation of user textual and structured information to evade detection. Extensive experiments with three LLMs on two datasets demonstrate that instruction tuning on merely 1,000 annotated examples produces specialized LLMs that outperform state-of-the-art baselines by up to 9.1% on both datasets, while LLM-guided manipulation strategies could significantly bring down the performance of existing bot detectors by up to 29.6% and harm the calibration and reliability of bot detection systems.
Say It Another Way: Auditing LLMs with a User-Grounded Automated Paraphrasing Framework
Mila - Quebec AI Institute(魁北克人工智能研究所);McGill University(麦吉尔大学);University of Waterloo(滑铁卢大学);Vector Institute(向量研究所);Université de Sherbrooke(舍布鲁克大学);Université de Montréal(蒙特利尔大学)
AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Risks
Institute of Artificial Intelligence, Leibniz University Hannover(汉诺威莱布尼茨大学人工智能学院);Alma Mater Studiorum – University of Bologna(博洛尼亚大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);L3S Research Center, Leibniz University Hannover(汉诺威莱布尼茨大学L3S研究中心)
See or Say Graphs: Agent-Driven Scalable Graph Structure Understanding with Vision-Language Models
University of Science and Technology of China(中国科学技术大学);Data Darkness Lab, MIRACLE Center, USTC(数据黑暗实验室,MIRACLE中心,USTC);MIRACLE Center, Suzhou Institute for Advance Research, USTC(MIRACLE中心,苏州先进研究所,USTC)
What Do Patients Say About Their Disease Symptoms? Deep Multilabel Text Classification With Human-in-the-Loop Curation for Automatic Labeling of Patient Self Reports of Problems