您好，欢迎访问中国农业科学院 机构知识库！

全部

  • 全部
  • 标题
  • 学者
  • 机构
  • 关键词

Inference to the Best Explanation in Large Language Models

文献类型： 会议论文

第一作者： Dhairya Dalal

作者： Dhairya Dalal 1 ; Marco Valentino 2 ; Andre Freitas 3 ; Paul Buitelaar 4 ;

作者机构： 1.SFI Centre for Research and Training in Artificial Intelligence, University of Galway, Ireland

2.Idiap Research Institute, Switzerland

3.Idiap Research Institute, Switzerland##Department of Computer Science, University of Manchester, UK##National Biomarker Centre, CRUK-MI, University of Manchester, UK

4.SFI Centre for Research and Training in Artificial Intelligence, University of Galway, Ireland##Insight SFI Research Centre for Data Analytics, University of Galway, Ireland

会议名称： [ "Annual Meeting of the Association for Computational Linguistics" , "Annual meeting of the Association for Computational Linguistics"]

主办单位：

页码： 217-235

摘要： While Large Language Models (LLMs) have found success in real-world applications, their underlying explanatory process is still poorly understood. This paper proposes IBE-Eval, a framework inspired by philosophical accounts on Inference to the Best Explanation (IBE) to advance the interpretation and evaluation of LLM explanations. IBE-Eval estimates the plausibility of natural language explanations through a combination of explicit logical and linguistic features including: consistency, parsimony, coherence, and uncertainty. Extensive experiments are conducted on Causal Question Answering (CQA), where IBE-Eval is tasked to select the most plausible causal explanation amongst competing ones generated by the LLM (e.g. GPT 3.5 or LLaMA 2). The experiments reveal that IBE-Eval can successfully identify the best explanation with up to 77% accuracy (27% above random), improving upon a GPT 3.5-as-a-judge baseline ( +17%) while being intrinsically more efficient and in-terpretable. Additional analysis suggests that, despite LLM-specific variances, generated explanations tend to conform to IBE criteria and that IBE-Eval is significantly correlated with human judgment, opening up opportunities for future development of automated explanation verification tools.

分类号： tp391

  • 相关文献
作者其他论文 更多>>
置顶
购物车
反馈

© 京ICP备09089781号-29 主办单位：中国农业科学院

学术资源： 中国农业科学院 农业专业知识服务系统 农科联盟资源共建共享平台 中国农业科技文献与信息服务平台 中国农业期刊集成服务平台