TransBench：面向工业级应用的机器翻译基准测试
TransBench: Benchmarking Machine Translation for Industrial-Scale Applications
AI总结：
针对通用机器翻译评估基准与工业实际应用效果脱节的问题，本研究提出三级翻译能力评估框架，推出面向电商场景的工业机器翻译基准TransBench，整合多维度指标并开源工具，填补了专业领域翻译评估的空白。
摘要：
机器翻译（MT）已成为电商、金融、法律服务等全球化行业跨境沟通不可或缺的工具，近期大语言模型（LLM）的进展显著提升了翻译质量。然而，将通用机器翻译模型应用于工业场景时暴露出关键局限，原因在于通用基准中缺少领域特定术语、文化细微差异和文体惯例。现有评估框架无法充分评估模型在专业场景下的性能，导致学术基准与实际应用效果之间存在差距。为解决这一问题，我们提出一个三级翻译能力框架：（1）基础语言能力，（2）领域特定熟练度，（3）文化适配，强调需从这些维度开展整体评估。我们推出TransBench——专为工业机器翻译打造的基准，初期以国际电商为目标，包含17000条专业翻译句子，覆盖4大主要场景和33个语言对。TransBench将传统指标（BLEU、TER）与领域特定评估模型Marco-MOS相结合，并提供可复现基准构建的指导原则。我们的贡献包括：（1）结构化的工业机器翻译评估框架，（2）首个公开可用的电商翻译基准，（3）探测多层级翻译质量的新型指标，（4）开源评估工具。本工作填补了评估空白，助力研究者和从业者系统评估并优化满足行业特定需求的机器翻译系统。
英文摘要：
Machine translation (MT) has become indispensable for cross-border communication in globalized industries like e-commerce, finance, and legal services, with recent advancements in large language models (LLMs) significantly enhancing translation quality. However, applying general-purpose MT models to industrial scenarios reveals critical limitations due to domain-specific terminology, cultural nuances, and stylistic conventions absent in generic benchmarks. Existing evaluation frameworks inadequately assess performance in specialized contexts, creating a gap between academic benchmarks and real-world efficacy. To address this, we propose a three-level translation capability framework: (1) Basic Linguistic Competence, (2) Domain-Specific Proficiency, and (3) Cultural Adaptation, emphasizing the need for holistic evaluation across these dimensions. We introduce TransBench, a benchmark tailored for industrial MT, initially targeting international e-commerce with 17,000 professionally translated sentences spanning 4 main scenarios and 33 language pairs. TransBench integrates traditional metrics (BLEU, TER) with Marco-MOS, a domain-specific evaluation model, and provides guidelines for reproducible benchmark construction. Our contributions include: (1) a structured framework for industrial MT evaluation, (2) the first publicly available benchmark for e-commerce translation, (3) novel metrics probing multi-level translation quality, and (4) open-sourced evaluation tools. This work bridges the evaluation gap, enabling researchers and practitioners to systematically assess and enhance MT systems for industry-specific needs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
