Semantic Role Labeling Based on Highway-BiLSTM-CRF Model
  IJCTT-book-cover
 		          
 
© 2021 by IJCTT Journal
Volume-69 Issue-10
Year of Publication : 2021
Authors : Xinxin Li, Xiangzhong Pu
DOI :  10.14445/22312803/IJCTT-V69I10P103

How to Cite?

Xinxin Li, Xiangzhong Pu, "Semantic Role Labeling Based on Highway-BiLSTM-CRF Model," International Journal of Computer Trends and Technology, vol. 69, no. 10, pp. 23-27, 2021. Crossref, https://doi.org/10.14445/22312803/IJCTT-V69I10P103

Abstract
Semantic Role Labeling is an important task in natural language processing. At present, the main approach of Semantic role labeling is based on BiLSTM. However, the BiLSTM network may have training difficulties and vanishing gradient problems with increased network depth. This paper proposes a Highway-BiLSTM-CRF model to solve this problem, which connects BiLSTM layers with highway networks. In the input layer, dependency relations, the distance between predicate and arguments are added to improve the experimental effect. Finally, the CRF layer is used to obtain the optimal tagging sequence. Experimental results of Chinese PropBank show that Chinese semantic role labeling achieves the best performance when BiLSTM depth is 8 layers, in which F1 value reaches 80.15%.

Keywords
Semantic role labeling, BiLSTM-CRF, Highway network, The dependency relation.

Reference

[1] Luheng He, Kenton Lee, Mike Lewis, Luke Zettlemoyer, Deep Semantic Role Labeling: What Works and What’s Next. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (2017) 473-483.
[2] Wang Zhen, Jiang Tingsong, Chang Baobao, et al. Chinese semantic role labeling with the bidirectional recurrent neural network. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. (2015) 1626-1631?
[3] Yu Zhang, Guoguo Chen, Dong Yu, Kaisheng Yaco, Sanjeev Khudanpur, and James Glass. High-way long short-term memory rnns for distant speech recognition. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP),(2016)5755–5759.
[4] Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation, 9(8)(1997) 1735-1780.
[5] Xue Nianwen. Labeling Chinese predicates with semantic roles. Computational Linguistics, 34(2)( 2008) 225-255.
[6] Sun Weiwei, Sui Zhifang, Wang Meng, et al. Chinese semantic role labeling with shallow parsing. Proceedings of the EMNLP. (2009) 1475-1483.
[7] Yang Haitong, Zong Chengqing. Multi-predicate semantic role labeling, Proceedings of the 2014 Conference on empirical Methods in Natural Language Processing. (2014) 363-373?
[8] Collobert R, Weston J. A unified architecture for natural language processing: Deep neural networks with multitask learning, Proceedings of the 25th ICML. (2008)160-167.
[9] Collobert R, Weston J, Bottou L, et al. Natural language-processing (almost) from scratch. The Journal of Machine Learning Research. 12(1) (2011) 2493-2537.
[10] Niu Yilin, Xie Ruobong, Liu Zhiyuan, et al. Improved Word Representation Learnong with Sememes. Proceedings of ACL. (2017) 2049-2058.
[11] Li Tianshi, Li Qi, Chang Bbaobao. Improving Chinese Semantic Role Labeling with English Proposition Bank. China National Conference on Chinese Computational Linguistics. (2016) 3-11.
[12] Xia Qiaolin, Chang Baobao, Sui Zhifang. A Progressive Learning Approach to Chinese SRL Using Heterogeneous Data. Proceedings off the 55th ACL. (2017) 2069-2077.
[13] Sha Lei, Jiang Tingsong, Li Sujian, et al. Capturing argument relationships for Chinese semantic role labeling. Proceedings of the Conference on Empirical Methods in Natural Language Processing. (2016) 2011-2016.
[14] Guo Jiang?Che Wanxiang?Wang Haifeng? et al. ? A unified architecture for semantic role labeling and relation classi-fication[C]// Proceedings of the 26th International Conference on Computational Linguistics. (2016) 1264-1274.
[15] Wang Y, Johnson M, Wan S, et al. How to best use syntax in semantic role labeling.Proceedings of the 57th Conference of the Association for Computational Linguistics. (2019)5338-5343.
[16] Peters, Matthew E., Neumann, Mark, Iyyer, Mohit and Gardner, Matt and Clark, Christopher and Lee, Kenton and Zettlemoyer, Luke. Deep contextualized word representations. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. (2018) 2227-2237.
[17] Danqi Chen and Christopher D Manning. A Fast and Accurate Dependency Parser using Neural Networks. Proceedings of EMNLP (2014) 740-750.
[18] Mikolov, Tomas and Sutskever, Ilya and Chen, Kai and Corrado, Greg and Dean, Jeffrey. Distributed Representations of Words and Phrases and Their Compositionality. Proceedings of the 26th International Conference on Neural Information Processing Systems. (2013) 3111–3119.
[19] Jeffrey Pennington, Richard Socher, and Christopher D. Manning. GloVe: Global Vectors for Word Representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. (2014) 1532–1543.
[20] Zhang Miaomiao, Zhang Yujie, Liu Mingtong, Xu Jin-an, Chen Yufeng. Chinese Semantic Role Labeling Based on Gated Mechanism and Bi-LSTM-CRF. Computer and Modernization. 4(1) (2018) 254-263.