使用ClimaText微调ClimateBert Transformer,用于气候相关金融风险的披露分析
Fine-tuning ClimateBert transformer with ClimaText for the disclosure analysis of climate-related financial risks
摘要:
近年来,金融代理人,特别是特定投资者和机构投资者越来越多地要求公司报告与气候相关的金融风险。公司为了在财务和非财务报告中确定这些类型的风险,预计将在短期内披露大量文本格式的信息,特别是为了应对正在通过的关于这一问题的越来越多的法规。为此,本文采用最先进的自然语言处理技术来实现文本语料库中的气候变化检测。我们使用迁移学习来微调两个Transformer模型,BERT和ClimateBert -一个最近发布的基于DistillRoBERTA的模型,专门为气候文本分类量身定制。这两个算法是基于Transformer架构,使学习文本中的单词之间的上下文关系。我们在新的Clima-Text数据库上对这两种模型进行了微调,该数据库包括从维基百科,10 K文件报告和基于网络的索赔中收集的数据。我们的文本分类模型从ClimateBert对ClimaText的微调过程中获得,在这个特定的问题上优于使用BERT和当前最先进的Transformer创建的模型。我们的研究是第一个在ClimaText数据库上实现最近发表的ClimateBert算法的研究。根据我们的研究结果,可以说ClimateBert在ClimaText上微调是NLP预训练Transformer模型中的一个出色工具,投资者,机构代理人和公司本身可以并且应该使用它来监控财务报告中的气候风险披露。此外,我们的迁移学习方法在计算方面是便宜的,因此允许任何组织执行它。
英文摘要:
In recent years there has been a growing demand from financial agents, especially from particular and institutional investors, for companies to report on climate-related financial risks. A vast amount of information, in text format, can be expected to be disclosed in the short term by firms in order to identify these types of risks in their financial and non financial reports, particularly in response to the growing regulation that is being passed on the matter. To this end, this paper applies state-of-the-art NLP techniques to achieve the detection of climate change in text corpora. We use transfer learning to fine-tune two transformer models, BERT and ClimateBert -a recently published DistillRoBERTa-based model that has been specifically tailored for climate text classification-. These two algorithms are based on the transformer architecture which enables learning the contextual relationships between words in a text. We carry out the fine-tuning process of both models on the novel Clima-Text database, consisting of data collected from Wikipedia, 10K Files Reports and web-based claims. Our text classification model obtained from the ClimateBert fine-tuning process on ClimaText, outperforms the models created with BERT and the current state-of-the-art transformer in this particular problem. Our study is the first one to implement on the ClimaText database the recently published ClimateBert algorithm. Based on our results, it can be said that ClimateBert fine-tuned on ClimaText is an outstanding tool within the NLP pre-trained transformer models that may and should be used by investors, institutional agents and companies themselves to monitor the disclosure of climate risk in financial reports. In addition, our transfer learning methodology is cheap in computational terms, thus allowing any organization to perform it.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。