MiA-Signature: Approximating Global Activation for Long-Context Understanding
MiA-Signature:近似全局激活以实现长上下文理解
机构 * Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所) ; School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院) ; Pattern Recognition Center, WeChat AI, Tencent(微信AI模式识别中心) ; Hunyuan Team, Tencent(腾讯文心一言团队)
专题命中 长文档RAG :RAG(abstract,abstract_cn);分类 cs.CL
AI总结 本文提出MiA-Signature,通过压缩表示近似全局激活对下游处理的影响,提升长上下文理解任务的性能。
Comments This is a work in progress; we will continue to revise and improve the manuscript