In this research, patent prosecution is conceptualized as a system of reinforcement learning from human feedback. The objective of the system is to increase the likelihood for a language model to generate patent claims that have a higher chance of being granted. To showcase the controllability of the language model, the system learns from granted patents and pre-grant applications with different rewards. The status of "granted" and "pre-grant" are perceived as labeled human feedback implicitly. In addition, specific to patent drafting, the experiments in this research demonstrate the model's capability to learn from adjusting claim length and inclusion of limiting terms for narrowing claim scope. As proof of concept, the experiments focus on claim ones only and the training data originates from a patent dataset tailored specifically for artificial intelligence. Although the available human feedback in patent prosecution are limited and the quality of generated patent text requires improvement, the experiments following the 3-stage reinforcement learning from human feedback have demonstrated that generative language models are capable of reflecting the human feedback or intent in patent prosecution. To enhance the usability of language models, the implementation in this research utilizes modern techniques that enable execution on a single consumer-grade GPU. The demonstrated proof of concept, which reduces hardware requirements, will prove valuable in the future as more human feedback in patent prosecution become available for broader use, either within patent offices or in the public domain.
The Master-Slave Encoder Model for Improving Patent Text Summarization: A New Approach to Combining Specifications and Claims
School of Information Management, Nanjing University(南京大学信息管理学院);Key Laboratory of Data Engineering and Knowledge Services in Jiangsu Provincial Universities, Nanjing University(江苏省高校数据工程与知识服务重点实验室(南京大学));Baidu, Inc.(百度公司)
ParroT: Translating during Chat using Large Language Models tuned with Human Translation and Feedback
Tencent AI Lab(腾讯人工智能实验室);The Chinese University of Hong Kong(香港中文大学);Shanghai Jiao Tong University(上海交通大学);Tsinghua Shenzhen International Graduate School(清华大学深圳国际研究生院)
Instruct-of-Reflection: Enhancing Large Language Models Iterative Reflection Capabilities via Dynamic-Meta Instruction
Beijing University of Posts and Telecommunications(北京邮电大学);AI Lab of Lenovo Research(联想研究院AI实验室);Tianjin University(天津大学);The Hong Kong University of Science and Technology(香港科技大学)
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback
School of Computation, Information and Technology, TUM(慕尼黑工业大学计算、信息与技术学院);Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Center for Machine Learning(慕尼黑机器学习中心);Bosch Center for Artificial Intelligence(博世人工智能中心)
Align Anything: Training All-Modality Models to Follow Instructions with Language Feedback
Institute for AI, Peking University(北京大学人工智能研究院);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院);Huawei Noah’s Ark LAB(华为诺亚方舟实验室);Baichuan Inc.(百川智能);Taobao & Tmall Group of Alibaba(阿里巴巴淘宝天猫集团)
Training language models to follow instructions with human feedback
OpenAI(开放人工智能研究中心);Anthropic(Anthropic公司);Alignment Research Center(对齐研究中心)
使语言模型更大并不能使它们更好地遵循用户的意图。例如,大语言模型可能会生成不真实的、有害的或对用户毫无帮助的输出。换句话说,这些模型并不符合其用户。在本文中,我们展示了一种通过人工反馈进行微调,使语言模型与用户在各种任务上的意图保持一致的方法。从一组标注器编写的提示和通过OpenAI API提交的提示开始,我们收集了所需模型行为的标注器演示数据集,我们使用监督学习来微调GPT-3。然后,我们收集一个模型输出排名的数据集,我们使用它来进一步微调这个监督模型,使用基于人类反馈的强化学习。我们将生成的模型称为InstructGPT。在对我们的即时分布的人类评估中,尽管参数少了100倍,但1.3B参数InstructGPT模型的输出优于175 B GPT-3的输出。此外,InstructGPT模型在真实性方面有所改进,减少了有毒输出的生成,同时在公共NLP数据集上具有最小的性能回归。尽管InstructGPT仍然会犯一些简单的错误,但我们的研究结果表明,利用人类反馈进行微调是将语言模型与人类意图对齐的一个有希望的方向。
FASIONAD++ : Integrating High-Level Instruction and Information Bottleneck in FAt-Slow fusION Systems for Enhanced Safety in Autonomous Driving with Adaptive Feedback
The School of Vehicle and Mobility, Tsinghua University(清华大学车辆与出行学院);McGill University(麦吉尔大学);University of Wisconsin-Madison(威斯康星大学麦迪逊分校);Waseda University(早稻田大学);University of Minnesota(明尼苏达大学);AI2Robotics
CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback
Shanghai AI Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学);New York University(纽约大学);Carnegie Mellon University(卡内基梅隆大学);Shanghai Innovation Institute(上海创新研究院)
How Real Is AI Tutoring? Comparing Simulated and Human Dialogues in One-on-One Instruction
Shanghai Institute of Artificial Intelligence for Education, East China Normal University(华东师范大学上海人工智能教育研究院);Lab of Artificial Intelligence for Education, East China Normal University(华东师范大学人工智能教育实验室)
BenCao: An Instruction-Tuned Large Language Model for Traditional Chinese Medicine
Community Health Service Center Shanghai Pudong New Area(上海浦东新区社区卫生服务中心);School of Acupuncture-Moxibustion and Tuina, Shanghai University of Traditional Chinese Medicine(上海中医药大学针灸推拿学院)
本文提出基于 ChatGPT 的中医药多模态助手 BenCao,通过自然语言指令微调、知识库、思维链模拟、专家反馈及外部 API 整合,提升诊断、药材识别与体质分类性能并实现部署。