Aligning multilingual assistants with culturally grounded user preferences is essential for serving India's linguistically diverse population of over one billion speakers across multiple scripts. However, existing benchmarks either focus on a single language or conflate retrieval with generation, leaving open the question of whether current embedding models can encode persona-instruction compatibility without relying on response synthesis. We present a unified benchmark spanning 12 Indian languages and four evaluation tasks: monolingual and cross-lingual persona-to-instruction retrieval, reverse retrieval from instruction to persona, and binary compatibility classification. Eight multilingual embedding models are evaluated in a frozen-encoder setting with a thin logistic regression head for classification. E5-Large-Instruct achieves the highest Recall@1 of 27.4\% on monolingual retrieval and 20.7\% on cross-lingual transfer, while BGE-M3 leads reverse retrieval at 32.1\% Recall@1. For classification, LaBSE attains 75.3\% AUROC with strong calibration. These findings offer practical guidance for model selection in Indic multilingual retrieval and establish reproducible baselines for future work\footnote{Code, datasets, and models are publicly available at https://github.com/aryashah2k/PI-Indic-Align.
IBM Research AI(IBM研究院);International Institute of Information Technology, Bangalore(班加罗尔国际信息技术学院)
Popular dialog datasets such as MultiWOZ are created by providing crowd workers an instruction, expressed in natural language, that describes the task to be accomplished. Crowd workers play the role of a user and an agent to generate dialogs to accomplish tasks involving booking restaurant tables, calling a taxi etc. In this paper, we present a data creation strategy that uses the pre-trained language model, GPT2, to simulate the interaction between crowd workers by creating a user bot and an agent bot. We train the simulators using a smaller percentage of actual crowd-generated conversations …
One Size doesn't Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instruction
School of Computer Science, Wuhan University(武汉大学计算机学院);DAMO Academy, Alibaba Group(阿里巴巴达摩院);College of Computer and Cyber Security, Hebei Normal University(河北师范大学计算机与网络空间安全学院)
针对数学教学中学习者差异被忽视的问题,本文提出 PACE 个性化对话辅导智能体,结合学习风格画像、苏格拉底式反馈与个性化训练,提升教学适应性和学习效果。
Persistent Personas? Role-Playing, Instruction Following, and Safety in Extended Interactions
University of Vienna, Faculty of Computer Science(维也纳大学计算机科学系);Doctoral School Computer Science, University of Vienna(维也纳大学计算机科学博士学院);Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning(慕尼黑机器学习中心);University of Vienna, Faculty of Philological and Cultural Studies(维也纳大学文学与文化研究系)
Large Language Models Empowered Personalized Web Agents
National University of Singapore(新加坡国立大学);The Hong Kong Polytechnic University(香港理工大学);University of Science and Technology of China(中国科学技术大学);Eastern Institute of Technology(宁波东方理工大学)
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi(印度理工学院德里分校传感器、仪器仪表和网络物理系统工程中心(传感中心)本征实验室);RSL Quantum, FITT, IIT Delhi(印度理工学院德里分校 FITT 学院 RSL 量子实验室)
IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languages
Nilekani Centre at AI4Bharat(尼莱卡尼中心(AI4Bharat));Indian Institute of Technology, Madras(马德拉斯印度理工学院);Sant Longowal Institute of Engineering and Technology(桑特隆戈瓦尔工程技术学院);IIIT D&M Kancheepuram(坎奇普兰印度信息技术学院(设计与制造));Maharaja Agrasen Institute of Technology(Maharaja Agrasen工程技术学院);Microsoft(微软公司);Sarvam AI;National Institute of Information and Communications Technology, Kyoto, Japan(日本京都信息通信技术研究所)
SANSKRITI: A Comprehensive Benchmark for Evaluating Language Models' Knowledge of Indian Culture
Department of Computer Science and Engineering, Indian Institute of Technology Patna(印度理工学院帕纳布分校计算机科学与工程系);Department of Computer Science, Banasthali Vidyapeeth University(班斯塔利大学计算机科学系)
An increase in the use of smartphones has laid to the use of the internet and social media platforms. The most commonly used social media platforms are Twitter, Facebook, WhatsApp and Instagram. People are sharing their personal experiences, reviews, feedbacks on the web. The information which is available on the web is unstructured and enormous. Hence, there is a huge scope of research on understanding the sentiment of the data available on the web. Sentiment Analysis (SA) can be carried out on the reviews, feedbacks, discussions available on the web. There has been extensive research carried…
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
PRISM AI(PRISM人工智能公司);Harmony Intelligence(和谐智能公司);Leap Laboratories(leap实验室);MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);ETH AI Center, ETH Zurich(苏黎世联邦理工学院人工智能中心)
Indian Institute of Technology Roorkee(印度理工学院鲁尔基分校)
Government-sponsored policy-making and scheme generations is one of the means of protecting and promoting the social, economic, and personal development of the citizens. The evaluation of effectiveness of these schemes done by government only provide the statistical information in terms of facts and figures which do not include the in-depth knowledge of public perceptions, experiences and views on the topic. In this research work, we propose an improved text classification framework that classifies the Twitter data of different health-based government schemes. The proposed framework leverages …
LLaVA-Docent: Instruction Tuning with Multimodal Large Language Model to Support Art Appreciation Education
Korea University(高丽大学);University of Nebraska-Lincoln(内布拉斯加大学林肯分校);Seoul Metropolitan Office of Education(首尔市教育厅);Seoul National University(首尔国立大学);Seoul National University of Education(首尔国立教育大学)