End2You --通过端到端学习实现多模式分析的帝国工具包
End2You -- The Imperial Toolkit for Multimodal Profiling by End-to-End Learning
摘要:
我们介绍End 2 You--伦敦帝国理工学院通过端到端深度学习进行多模态分析的工具包。End 2 You是一个用Python实现的开源工具包,基于Tensorflow。它提供了以端到端方式训练和评估模型的能力,即,使用原始输入。它支持原始音频、视觉、生理或其他类型的信息或这些信息的组合的输入,并且输出可以是任意表示,用于分类或回归任务。据我们所知,这是第一个提供通用端到端学习的工具包,用于在单峰或多峰情况下进行分析。为了测试我们的工具包,我们使用了在AVEC 2016挑战赛中使用的RECOLA数据库。实验结果表明,End 2 You可以提供与最先进的方法相当的结果,尽管不需要类似专家的特征表示,但可以从数据中“端到端”自我学习。
英文摘要:
We introduce End2You -- the Imperial College London toolkit for multimodal profiling by end-to-end deep learning. End2You is an open-source toolkit implemented in Python and is based on Tensorflow. It provides capabilities to train and evaluate models in an end-to-end manner, i.e., using raw input. It supports input from raw audio, visual, physiological or other types of information or combination of those, and the output can be of an arbitrary representation, for either classification or regression tasks. To our knowledge, this is the first toolkit that provides generic end-to-end learning for profiling capabilities in either unimodal or multimodal cases. To test our toolkit, we utilise the RECOLA database as was used in the AVEC 2016 challenge. Experimental results indicate that End2You can provide comparable results to state-of-the-art methods despite no need of expert-alike feature representations, but self-learning these from the data "end to end".
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。