MacBehaviour：一款用于大语言模型行为实验的R包
MacBehaviour: An R package for behavioural experimentation on large language models
AI总结：
本研究开发了R包MacBehaviour，可对接60余种大语言模型，提供覆盖实验全流程的功能，通过三项复现声音-性别关联的验证实验证实其有效性，为机器行为研究提供标准化实验工具。
摘要：
当前，研究者们越来越关注通过将大语言模型（LLM）作为心理学实验参与者的方式，来探究大语言模型及基于大语言模型的聊天机器人的行为。为此，我们开发了名为“MacBehaviour”的R包，该包旨在实现单个包内与60余种语言模型（例如OpenAI的GPT系列、Claude系列、Gemini、Llama系列以及各类开源模型）的交互，并简化大语言模型行为实验的实验流程。该包提供了一套专为大语言模型实验设计的全面功能，涵盖实验设计、刺激呈现、模型行为操控、响应记录与token概率记录。为验证“MacBehaviour”的实用性与有效性，我们针对三款大语言模型（GPT-3.5、Llama-2 7B和Vicuna-1.5 13B）开展了三项验证实验，以复现大语言模型中的声音-性别关联效应。结果一致显示，这些模型表现出类人倾向，会根据新颖人名的语音推断性别，这与Cai等人（2023）此前的研究结果一致。综上，“MacBehaviour”是一款面向机器行为研究的R包，它提供了友好的用户界面与全面的功能，可简化并标准化实验流程。
英文摘要：
There has been increasing interest in investigating the behaviours of large language models (LLMs) and LLM-powered chatbots by treating an LLM as a participant in a psychological experiment. We therefore developed an R package called "MacBehaviour" that aims to interact with more than 60 language models in one package (e.g., OpenAI's GPT family, the Claude family, Gemini, Llama family, and open-source models) and streamline the experimental process of LLMs behaviour experiments. The package offers a comprehensive set of functions designed for LLM experiments, covering experiment design, stimuli presentation, model behaviour manipulation, logging response and token probability. To demonstrate the utility and effectiveness of "MacBehaviour," we conducted three validation experiments on three LLMs (GPT-3.5, Llama-2 7B, and Vicuna-1.5 13B) to replicate sound-gender association in LLMs. The results consistently showed that they exhibit human-like tendencies to infer gender from novel personal names based on their phonology, as previously demonstrated (Cai et al., 2023). In summary, "MacBehaviour" is an R package for machine behaviour studies which offers a user-friendly interface and comprehensive features to simplify and standardize the experimental process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
