History Reuse and Bag-of-Words Loss for Long Summary Generation
文献类型: 外文期刊
第一作者: Liu, Qing
作者: Liu, Qing;Chen, Lei;Yuan, Yuan;Liu, Qing;Wu, Huarui
作者机构:
关键词: History; Decoding; Training; Predictive models; Postal services; Computational modeling; Vocabulary; Abstractive summarization; long summaries; history reuse; bag-of-words; word order deviation
期刊名称:IEEE-ACM TRANSACTIONS ON AUDIO SPEECH AND LANGUAGE PROCESSING ( 影响因子:3.919; 五年影响因子:3.906 )
ISSN: 2329-9290
年卷期: 2021 年 29 卷
页码:
收录情况: SCI
摘要: Recurrent Neural Network (RNN) based abstractive text summarization models have made great progress over the past few years, largely triggered by the encoder-decoder architecture. However, there has been little work improving the generation of relatively long summaries. In this paper, we concentrate on two prominent problems in long summary generation. First, although significant efforts have been made to assist the encoder in handling long sequences, the decoder struggles with long sequences owing to the limited storage capacity of RNN. We propose a simple and effective approach called history reuse, which first mines critical information from the history summary sequence and then transmits the information to the decoder. Second, since encoder-decoder models are typically trained to produce exactly the same summary as the target summary, certain word order deviations between the predicted summary and target summary are excessively punished. Accordingly, we introduce a fully differentiable loss called bag-of-words (BoW) loss, which takes advantage of the feature of BoW discarding word order information in texts, and computes the difference between the two summaries at the BoW space. Experiments on two benchmark datasets, CNN/Daily Mail and Pubmed, demonstrate that our methods significantly improve the baseline.
分类号:
- 相关文献
作者其他论文 更多>>
-
Greenhouse light and CO2 regulation considering cost and photosynthesis rate using i-nsGA II
作者:Gao, Pan;Lu, Miao;Yang, Yongxia;Hu, Jin;Gao, Pan;Lu, Miao;Hu, Jin;Wu, Huarui;Mao, Hanping
关键词:Multi-objective optimization; Photosynthesis rate; Curvature theory; Environmental regulation; Regulation cost
-
The genetic basis of grain protein content in rice by genome-wide association analysis
作者:Chen, Pingli;Lou, Guangming;Wang, Yufu;He, Yuqing;Chen, Pingli;Lou, Guangming;Wang, Yufu;He, Yuqing;Chen, Pingli;Chen, Wengfeng;Fan, Zhilan;Liu, Qing;Sun, Bingrui;Mao, Xingxue;Yu, Hang;Jiang, Liqun;Zhang, Jing;Lv, Shuwei;Xing, Junlian;Pan, Dajian;Li, Chen;Chen, Pingli;Chen, Wengfeng;Fan, Zhilan;Liu, Qing;Sun, Bingrui;Mao, Xingxue;Yu, Hang;Jiang, Liqun;Zhang, Jing;Lv, Shuwei;Xing, Junlian;Pan, Dajian;Li, Chen;He, Yuqing;Chen, Pingli;Chen, Wengfeng;Fan, Zhilan;Liu, Qing;Sun, Bingrui;Mao, Xingxue;Yu, Hang;Jiang, Liqun;Zhang, Jing;Lv, Shuwei;Xing, Junlian;Pan, Dajian;Li, Chen;He, Yuqing;Chen, Pingli;Chen, Wengfeng;Fan, Zhilan;Liu, Qing;Sun, Bingrui;Mao, Xingxue;Yu, Hang;Jiang, Liqun;Zhang, Jing;Lv, Shuwei;Xing, Junlian;Pan, Dajian;Li, Chen;He, Yuqing;Chen, Junxiao
关键词:Oryza sativa; Grain protein content; GWAS; Flo5; Grain quality; CRISPR; Cas9
-
Improving crop-livestock integration in China using numerical experiments at catchment and regional scales
作者:Chen, Lei;Wang, Yiwen;Yang, Nian;Zhu, Kaihang;Yan, Xiaoman;Shen, Zhenyao;Bai, Zhaohai;Zhai, Limei
关键词:Agriculture; Crop -livestock integration; Fertilizer; Crop yield; Nonpoint source pollution
-
Manganese-based coordination framework derived manganese sulfide nanoparticles integrated with carbon sheets for application in supercapacitor
作者:Xi, Yanjie;Zeng, Xia;Chen, Jing;Ma, Lin;Zhan, Zhihua;Chen, Chen;Yuan, Yuan;Liao, Lusheng;Peng, Ziyun;Zheng, Liyi;Huang, Yilin;Xu, Limei;Ma, Lin;Liao, Lusheng
关键词:Manganese sulfide; Coordination framework; Pseudocapacitance; Supercapacitor; Carbon nanomaterials
-
Ulva prolifera polysaccharide enhances the root colonisation by Bacillus amyloliquefaciens strain Cas02
作者:Chu, Depeng;Zheng, Yanfen;You, Xiangwei;Jing, Changliang;Li, Yiqiang;Yuan, Yuan;Wang, Xiaoqiang;Liang, Jinchang;Fan, Jiajun;Clark, James;Peng, Lijuan;Wang, Rui
关键词:Root colonisation; Bacillus amyloliquefaciens; Ulva prolifera polysaccharide; Biofilm; Rhizosphere microbiome
-
OsGLP3-7 positively regulates rice immune response by activating hydrogen peroxide, jasmonic acid, and phytoalexin metabolic pathways
作者:Sun, Bingrui;Ma, Yamei;Ding, Jierong;Yu, Hang;Jiang, Liqun;Zhang, Jing;Lv, Shuwei;Liu, Bin;Liu, Qing;Li, Wenyan;Yu, Ting;Huang, Wenjie;Yan, Shijuan;Yang, Jianyuan
关键词:disease resistance; germin-like protein; H2O2; jasmonic acid; phytoalexin; rice
-
Creation and gene expression analysis of a giant embryo rice mutant with high GABA content
作者:Chen, Pingli;Liu, Qing;Sun, Bingrui;Lv, Shuwei;Jiang, Liqun;Zhang, Jing;Mao, Xingxue;Yu, Hang;Chen, Yangyang;Chen, Wenfeng;Fan, Zhilan;Pan, Dajian;Li, Chen
关键词:Gamma-amino butyric acid (GABA); Giant embryo rice; CRISPR/Cas9; RNA-seq; Grain development