NLP practitioners often want to take existing trained models and apply them to data from new domains. While fine-tuning or few-shot learning can be used to adapt a base model, there is no single recipe for making these techniques work; moreover, one may not have access to the original model weights if it is deployed as a black box. We study how to improve a black box model's performance on a new domain by leveraging explanations of the model's behavior. Our approach first extracts a set of features combining human intuition about the task with model attributions generated by black box interpretation techniques, then uses a simple calibrator, in the form of a classifier, to predict whether the base model was correct or not. We experiment with our method on two tasks, extractive question answering and natural language inference, covering adaptation from several pairs of domains with limited target-domain data. The experimental results across all the domain pairs show that explanations are useful for calibrating these models, boosting accuracy when predictions do not have to be returned on every example. We further show that the calibration model transfers to some extent between tasks.
PRISM: Calibrating Black-Box Judges with Counterfactual Probes
Massachusetts Institute of Technology(麻省理工学院);Sakana AI;Lab for Information and Decision Systems(信息与决策系统实验室);Statistics and Data Science Center(统计与数据科学中心);EECS(电气工程与计算机科学系)
LLM-Generated Black-box Explanations Can Be Adversarially Helpful
University of Toronto(多伦多大学);Vector Institute(矢量研究所);UMass Amherst(马萨诸塞大学阿默斯特分校);Stevens Institute of Technology(史蒂文斯理工学院);Dalhousie University(达尔豪斯大学)
An Evaluation of Explanation Methods for Black-Box Detectors of Machine-Generated Text
Faculty of Computer Science, University of Vienna(维也纳大学计算机科学系);UniVie Doctoral School Computer Science, University of Vienna(维也纳大学计算机科学博士学院);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学文学与文化研究系)
Explaining Black-box Language Models with Knowledge Probing Systems: A Post-hoc Explanation Perspective
School of Computer and Information Technology, Shanxi University, China(山西大学计算机与信息学院);Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, China(教育部计算智能与中文信息处理重点实验室);Institute for Infocomm Research, A*Star, Singapore(A*Star信息与通信研究所)
Look at the Variance! Efficient Black-box Explanations with Sobol-based Sensitivity Analysis
Carney Institute for Brain Science, Brown University(布朗大学卡尼脑科学研究所);Artificial and Natural Intelligence Toulouse Institute, Université de Toulouse(图卢兹大学图卢兹人工与自然智能研究所);Institut de Recherche Technologique Saint-Exupery(圣埃克苏佩里技术研究所);Sorbonne Université, CNRS(索邦大学、法国国家科学研究中心)
Gnothi Seauton: Empowering Faithful Self-Interpretability in Black-Box Transformers
School of Artificial Intelligence, Shanghai Jiao Tong University(上海交通大学人工智能学院);Shanghai Jiao Tong University(上海交通大学);Sichuan University(四川大学);Big Data and AI Lab, ICBC(中国工商银行大数据与人工智能实验室);Hong Kong University of Science and Technology, Guangzhou(香港科技大学广州校区)