Graph convolutional network (GCN) has been successfully applied to capture global non-consecutive and long-distance semantic information for text classification. However, while GCN-based methods have shown promising results in offline evaluations, they commonly follow a seen-token-seen-document paradigm by constructing a fixed document-token graph and cannot make inferences on new documents. It is a challenge to deploy them in online systems to infer steaming text data. In this work, we present a continual GCN model (ContGCN) to generalize inferences from observed documents to unobserved documents. Concretely, we propose a new all-token-any-document paradigm to dynamically update the document-token graph in every batch during both the training and testing phases of an online system. Moreover, we design an occurrence memory module and a self-supervised contrastive learning objective to update ContGCN in a label-free manner. A 3-month A/B test on Huawei public opinion analysis system shows ContGCN achieves 8.86% performance gain compared with state-of-the-art methods. Offline experiments on five public datasets also show ContGCN can improve inference quality. The source code will be released at https://github.com/Jyonn/ContGCN.
Information-Theoretic Complementary Prompts for Improved Continual Text Classification
Mohamed bin Zayed University of Artificial Intelligence(莫莫德·本·扎耶德人工智能大学);Center for Excellence in Brain Science and Intelligence Technology(脑科学与智能技术 excellence 中心);Chinese Academy of Sciences(中国科学院);Tencent AI Lab(腾讯人工智能实验室)
CTTA-T: Continual Test-Time Adaptation for Text Understanding via Teacher-Student with a Domain-aware and Generalized Teacher
College of Computer Science and Technology, National University of Defense Technology(计算机科学与技术学院,国防科技大学);The Hong Kong University of Science and Technology(香港科技大学)