Linear socio-demographic representations emerge in Large Language Models from indirect cues
Complex Systems Institute of Paris Ile-de-France CNRS(巴黎法兰西大区复杂系统研究所 CNRS);médialab, Sciences Po(媒体实验室,巴黎政治学院);Learning Planet Institute, CY Cergy Paris University(学习星球研究所,CY塞quenes巴黎大学)
We investigate how LLMs encode sociodemographic attributes of human conversational partners inferred from indirect cues such as names and occupations. We show that LLMs develop linear representations of user demographics within activation space, wherein stereotypically associated attributes are encoded along interpretable geometric directions. We first probe residual streams across layers of four open transformer-based LLMs (Magistral 24B, Qwen3 14B, GPT-OSS 20B, OLMo2-1B) prompted with explicit demographic disclosure. We show that the same probes predict demographics from implicit cues: names activate census-aligned gender and race representations, while occupations trigger representations correlated with real-world workforce statistics. These linear representations allow us to explain demographic inferences implicitly formed by LLMs during conversation. We demonstrate that these implicit demographic representations actively shape downstream behavior, such as career recommendations. Our study further highlights that models that pass bias benchmark tests may still harbor and leverage implicit biases, with implications for fairness when applied at scale.
CityRiSE: Reasoning Urban Socio-Economic Status in Large Vision-Language Models via Reinforcement Learning
Information Hub, The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州)信息中心);Department of Electronic Engineering, BNRist, Tsinghua University(清华大学电子工程系)