大型语言模型嵌入中的语义结构
Semantic Structure in Large Language Model Embeddings
AI总结：
研究发现LLM嵌入中的语义关联与人类评分高度相关，可降维至3维子空间，且特征存在纠缠，指出在引导特征时需考虑该低维语义结构以避免意外后果。
摘要：
心理学研究一致发现，人类对词汇在多种语义量表上的评分可以降维成低维形式，且信息损失相对较少。我们发现大型语言模型（LLM）嵌入矩阵中编码的语义关联也表现出类似的结构。我们表明，词汇在由反义词对（例如 kind - cruel）定义的语义方向上的投影与人类评分高度相关，并进一步发现这些投影在LLM嵌入中有效地降维至一个3维子空间，与人类调查响应得出的模式非常相似。此外，我们发现沿某一语义方向移动token会对几何上对齐的特征造成与其余弦相似度成正比的偏移效应。这些发现表明，语义特征在LLM中的纠缠方式与其在人类语言中的互联方式相似，并且大量语义信息尽管表面上复杂，却出人意料地呈现低维性。此外，在引导特征时，考虑到这种语义结构对于避免意外后果可能至关重要。
英文摘要：
Psychological research consistently finds that human ratings of words across diverse semantic scales can be reduced to a low-dimensional form with relatively little information loss. We find that the semantic associations encoded in the embedding matrices of large language models (LLMs) exhibit a similar structure. We show that the projections of words on semantic directions defined by antonym pairs (e.g. kind - cruel) correlate highly with human ratings, and further find that these projections effectively reduce to a 3-dimensional subspace within LLM embeddings, closely resembling the patterns derived from human survey responses. Moreover, we find that shifting tokens along one semantic direction causes off-target effects on geometrically aligned features proportional to their cosine similarity. These findings suggest that semantic features are entangled within LLMs similarly to how they are interconnected in human language, and a great deal of semantic information, despite its apparent complexity, is surprisingly low-dimensional. Furthermore, accounting for this semantic structure may prove essential for avoiding unintended consequences when steering features.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
