arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.08652cs.CL
收藏

Mind Your Language: Abuse and Offense Detection for Code-Switched Languages

NSIT-Delhi(德里内塔吉·苏巴斯理工学院);Adobe Systems(奥多比系统公司);IIIT, Delhi(德里英迪拉·甘地信息技术学院);NUS, Singapore(新加坡国立大学)

Raghav Kapoor, Yaman Kumar, Kshitij Rajput, Rajiv Ratn Shah, Ponnurangam Kumaraguru, Roger Zimmermann

英文摘要:

In multilingual societies like the Indian subcontinent, use of code-switched languages is much popular and convenient for the users. In this paper, we study offense and abuse detection in the code-switched pair of Hindi and English (i.e. Hinglish), the pair that is the most spoken. The task is made difficult due to non-fixed grammar, vocabulary, semantics and spellings of Hinglish language. We apply transfer learning and make a LSTM based model for hate speech classification. This model surpasses the performance shown by the current best models to establish itself as the state-of-the-art in the unexplored domain of Hinglish offensive text classification.We also release our model and the embeddings trained for research purposes

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑