Recent advancements enlarge the capabilities of large language models (LLMs) in zero-shot image-to-text generation and understanding by integrating multi-modal inputs. However, such success is typically limited to English scenarios due to the lack of large-scale and high-quality non-English multi-modal resources, making it extremely difficult to establish competitive counterparts in other languages. In this paper, we introduce the Ziya-Visual series, a set of bilingual large-scale vision-language models (LVLMs) designed to incorporate visual semantics into LLM for multi-modal dialogue. Composed of Ziya-Visual-Base and Ziya-Visual-Chat, our models adopt the Querying Transformer from BLIP-2, further exploring the assistance of optimization schemes such as instruction tuning, multi-stage training and low-rank adaptation module for visual-language alignment. In addition, we stimulate the understanding ability of GPT-4 in multi-modal scenarios, translating our gathered English image-text datasets into Chinese and generating instruction-response through the in-context learning method. The experiment results demonstrate that compared to the existing LVLMs, Ziya-Visual achieves competitive performance across a wide range of English-only tasks including zero-shot image-text retrieval, image captioning, and visual question answering. The evaluation leaderboard accessed by GPT-4 also indicates that our models possess satisfactory image-text understanding and generation capabilities in Chinese multi-modal scenario dialogues. Code, demo and models are available at ~\url{https://huggingface.co/IDEA-CCNL/Ziya-BLIP2-14B-Visual-v1}.
Dólares or Dollars? Unraveling the Bilingual Prowess of Financial LLMs Between Spanish and English
The Fin AI；University of Manchester（曼彻斯特大学）；School of Computer Science Sichuan University（四川大学计算机学院）；School of Computer Science Wuhan University（武汉大学计算机学院）；University of Florida（佛罗里达大学）；Columbia University（哥伦比亚大学）
针对西班牙语金融NLP研究不足的问题，提出首个双语框架Toisón de Oro，构建指令数据集与评估基准，并推出FinMA-ES模型，在西班牙语金融任务中超越GPT-4等SOTA模型。
InstructIE: A Bilingual Instruction-based Information Extraction Dataset
Zhejiang University（浙江大学）；Zhejiang University - Ant Group Joint Laboratory of Knowledge Graph（浙江大学-蚂蚁集团知识图谱联合实验室）；Ant Group（蚂蚁集团）；University of Edinburgh（爱丁堡大学）
No Language is an Island: Unifying Chinese and English in Financial Large Language Models, Instruction Data, and Benchmarks
Yunnan University（云南大学）；The University of Manchester（曼彻斯特大学）；Wuhan University（武汉大学）；University of Florida（佛罗里达大学）；The Chinese University of Hong Kong（香港中文大学）；The Fin AI
Golden Touchstone: A Comprehensive Bilingual Benchmark for Evaluating Financial Large Language Models
IDEA Research（IDEA研究机构）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；Nanjing University（南京大学）；South China Normal University（华南师范大学）；DataArcTech Ltd.（DataArcTech有限公司）
Doctor Sun: A Bilingual Multimodal Large Language Model for Biomedical AI
Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education East China University of Science and Technology（能源化工过程智能制造重点实验室，东华大学）；Research Institute of Intelligent Control and Systems Harbin Institute of Technology（智能控制与系统研究室，哈尔滨工业大学）；Department of Emergency Medicine, Sir Run Run Shaw Hospital Zhejiang University School of Medicine（浙江大学医学院急诊医学科）；Provincial Key Laboratory of Precise Diagnosis Treatment of Abdominal Infection, Sir Run Run Shaw Hospital Zhejiang University School of Medicine（腹部感染精准诊断治疗省级重点实验室，浙江大学医学院）；School of Medicine Shaoxing University（绍兴大学医学院）
Doctor Sun是一种双语多模态大语言模型，通过整合预训练视觉编码器和医学LLM，提升生物医学多模态任务的性能，并提供SunMed-VL数据集支持研究进展。
PersianMedQA: Evaluating Large Language Models on a Persian-English Bilingual Medical Question Answering Benchmark
School of Electrical and Computer Engineering, University of Tehran（德黑兰大学电气与计算机工程学院）；Shahid Beheshti University of Medical Sciences（沙希德·贝赫什提医学科学大学）；Institute for Research in Fundamental Sciences (IPM)（基础科学研究所（IPM））
Data Science for Social Impact University of Pretoria School of Computing Clemson University（数据科学与社会影响 乌得勒支大学 学术 computing 勃林茅伊大学）；Data Science for Social Impact Department of Computer Science University of Pretoria（数据科学与社会影响 计算机科学系 乌得勒支大学）
IKS-Instruct: A 24,000-Example Multilingual Dataset for Teaching Language Models Indian Knowledge Systems
Intrinsic Lab, Centre for Sensors, Instrumentation and Cyber-Physical System Engineering (Centre for SeNSE), Indian Institute of Technology Delhi（印度理工学院德里分校传感器、仪器仪表和网络物理系统工程中心（传感中心）本征实验室）；RSL Quantum, FITT, IIT Delhi（印度理工学院德里分校 FITT 学院 RSL 量子实验室）
MMBench: Is Your Multi-modal Model an All-around Player?
Shanghai AI Laboratory（上海人工智能实验室）；Nanyang Technological University（南洋理工大学）；National University of Singapore（新加坡国立大学）；Zhejiang University（浙江大学）；The Chinese University of Hong Kong（香港中文大学）
