The statistical methods derived and described in this thesis provide new ways to elucidate the structural properties of text and other symbolic sequences. Generically, these methods allow detection of a difference in the frequency of a single feature, the detection of a difference between the frequencies of an ensemble of features and the attribution of the source of a text. These three abstract tasks suffice to solve problems in a wide variety of settings. Furthermore, the techniques described in this thesis can be extended to provide a wide range of additional tests beyond the ones described here.
A variety of applications for these methods are examined in detail. These applications are drawn from the area of text analysis and genetic sequence analysis. The textually oriented tasks include finding interesting collocations and cooccurent phrases, language identification, and information retrieval. The biologically oriented tasks include species identification and the discovery of previously unreported long range structure in genes. In the applications reported here where direct comparison is possible, the performance of these new methods substantially exceeds the state of the art.
Overall, the methods described here provide new and effective ways to analyse text and other symbolic sequences. Their particular strength is that they deal well with situations where relatively little data are available. Since these methods are abstract in nature, they can be applied in novel situations with relative ease.
Measuring vagueness and subjectivity in texts: from symbolic to neural VAGO
Institut Jean-Nicod(让·尼科研究所);CNRS(法国国家科学研究中心);ENS-PSL(巴黎高等师范学院-巴黎文理研究大学);EHESS(法国高等社会科学研究院);IRISA-CNRS(法国国家信息与自动化研究所-法国国家科学研究中心);Direction Générale de l’Armement(法国军备总局);Mondeca(蒙代卡公司)
CodeSteer: Symbolic-Augmented Language Models via Code/Text Guidance
Massachusetts Institute of Technology, Boston, MA, USA(麻省理工学院);Harvard University, Boston, MA, USA(哈佛大学);MIT-IBM Watson AI Lab, Boston, MA, USA(MIT-IBM Watson AI实验室);University of Illinois Urbana-Champaign, Urbana, IL, USA(伊利诺伊大学厄巴纳-香槟分校)
Microsoft Research Asia(微软亚洲研究院);Dalian University of Technology(大连理工大学);Zhejiang University(浙江大学);Shanghai Jiao Tong University(上海交通大学);Nanjing University(南京大学)
A symbolic Perl algorithm for the unification of Nahuatl word spellings
ICTT, Avignon Université(阿维尼翁大学ICTT);Inst. de Investigaciones en Educación, Universidad Veracruzana(韦拉克鲁斯大学教育研究所);GIL, Universidad Nacional Autónoma de México(墨西哥国立自治大学GIL);Avignon, France(法国阿维尼翁)
Digestion Algorithm in Hierarchical Symbolic Forests: A Fast Text Normalization Algorithm and Semantic Parsing Framework for Specific Scenarios and Lightweight Deployment