超越余弦相似度的多维比较：维度不敏感欧几里得度量
Surpassing Cosine Similarity for Multidimensional Comparisons: Dimension Insensitive Euclidean Metric
AI总结：
本文针对余弦相似度在高维比较中因维度依赖而产生偏差的问题，提出维度不敏感欧几里得度量（DIEM），并通过大语言模型应用验证其在多维数据分析中更强的鲁棒性和可解释性。
摘要：
计算能力和硬件效率的提升使得处理日益复杂的高维问题成为可能。尽管人工智能（AI）取得了显著成果，但高维解决方案的可解释性仍然具有挑战性。一个关键问题是如何比较多维量，这在主成分分析等技术中至关重要。余弦相似度等度量标准经常被使用，例如在自然语言处理算法或推荐系统的开发中。然而，随着维度的增加，此类度量的可解释性会下降。本文分析了维度的影响，揭示了余弦相似度的显著局限性，尤其是其对向量维度的依赖性，导致结果存在偏差且难以解释。为解决这一问题，我们提出了一种维度不敏感欧几里得度量（DIEM），该度量在不同维度上表现出更强的鲁棒性和泛化能力。DIEM 保持了一致的变异性，并消除了传统度量中观察到的偏差，使其成为高维比较的可靠工具。文中报告了一个大语言模型应用示例，展示了 DIEM 相对于余弦相似度的优势。这一新颖度量有潜力取代余弦相似度，为从神经运动控制到机器学习等领域分析多维数据提供更准确、更具洞察力的方法。
英文摘要：
Advances in computational power and hardware efficiency have enabled tackling increasingly complex, high-dimensional problems. While artificial intelligence (AI) achieves remarkable results, the interpretability of high-dimensional solutions remains challenging. A critical issue is the comparison of multidimensional quantities, essential in techniques like Principal Component Analysis. Metrics such as cosine similarity are often used, for example in the development of natural language processing algorithms or recommender systems. However, the interpretability of such metrics diminishes as dimensions increase. This paper analyzes the effects of dimensionality, revealing significant limitations of cosine similarity, particularly its dependency on the dimension of vectors, leading to biased and poorly interpretable outcomes. To address this, we introduce a Dimension Insensitive Euclidean Metric (DIEM) which demonstrates superior robustness and generalizability across dimensions. DIEM maintains consistent variability and eliminates the biases observed in traditional metrics, making it a reliable tool for high-dimensional comparisons. An example of the advantages of DIEM over cosine similarity is reported for a large language model application. This novel metric has the potential to replace cosine similarity, providing a more accurate and insightful method to analyze multidimensional data in fields ranging from neuromotor control to machine learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
