Recent advancements in Large Audio Language Models (LALMs) have demonstrated exceptional performance in speech recognition and translation. However, existing models often suffer from a disconnect between perception and expression, resulting in a robotic "read-speech" style that lacks the spontaneity and emotional resonance of real human interaction. In this report, we introduce Hello-Chat, an end-to-end audio language model designed for realistic social scenarios. By leveraging a massive dataset of real-life conversations and employing a modality-interleaved training strategy, Hello-Chat achieves a breakthrough in anthropomorphic generation. Experimental results show that our model not only reaches state-of-the-art (SOTA) performance on specific audio understanding tasks but also significantly outperforms existing baselines in prosodic naturalness and emotional alignment, paving the way for the next generation of empathetic AI agents.
Not in Sync: Unveiling Temporal Bias in Audio Chat Models
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of California, Merced(加州大学默塞德分校);Beijing University of Posts and Telecommunications(北京邮电大学);University of Southern California(南加州大学)
ASR Adaptation for E-commerce Chatbots using Cross-Utterance Context and Multi-Task Language Modeling
Amazon AWS AI(亚马逊AWS人工智能部门)
对插槽实体的自动语音识别(ASR)鲁棒性在涉及货币交易和购买的电子商务语音助理中至关重要。随着有效的领域适应,它是直观的,跨话语上下文线索在消歧领域特定的内容词从语音中起着重要的作用。在本文中,我们研究了各种技术,以提高上下文,内容词的鲁棒性和域自适应的transformer-XL神经语言模型(NLM),以rescore ASR N-best假设。为了提高语境化,我们利用轮级对话行为以及跨话语语境结转。此外,为了使我们的域通用NLM适应电子商务的运行,我们在域内数据上使用来自微调掩码LM的嵌入。最后,为了提高对域内内容词的鲁棒性,我们提出了一个多任务模型,可以联合执行内容词检测和语言建模任务。与非上下文LSTM LM基线相比,我们表现最好的NLM重新评分器在电子商务音频测试集上的内容WER减少了19.2%,插槽标签F1提高了6.4%。
VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation
Massachusetts Institute of Technology, USA(麻省理工学院, 美国);Seoul National University Hospital, South Korea(首尔国立大学医院, 韩国);Doctor Diary, South Korea(Doctor Diary, 韩国);Google Research, USA(谷歌研究, 美国);Independent Researcher(独立研究者)
Low Resource Multimodal Translation of Nepali Spoken Words into Emotion-Conditioned Sign Language Avatars
Center for Human Mobility and Communications, Prateek Innovations(普拉蒂克创新公司人类移动与通信中心);Sunway International Business School, Birmingham City University(双威国际商学院,伯明翰城市大学)
Nexus: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
Imperial College London(伦敦帝国学院);University of Manchester(曼彻斯特大学);HiThink Research(HiThink研究院);Soochow University(苏州大学);Hong Kong Baptist University(香港 Baptist大学);Idiap Research Institute(Idiap研究 institute);Meta AI;Fudan University(复旦大学)