Probabilistic Chinese word segmentation with non-local information and stochastic training | |
Sun, Xu ; Zhang, Yaozhong ; Matsuzaki, Takuya ; Tsuruoka, Yoshimasa ; Tsujii, Jun&apos ; ichi | |
刊名 | information processing management
![]() |
2013 | |
关键词 | Word segmentation Natural language processing Conditional random fields Latent conditional random fields Online training CONDITIONAL RANDOM-FIELDS |
DOI | 10.1016/j.ipm.2012.12.003 |
英文摘要 | In this article, we focus on Chinese word segmentation by systematically incorporating non-local information based on latent variables and word-level features. Differing from previous work which captures non-local information by using semi-Markov models, we propose an alternative method for modeling non-local information: a latent variable word segmenter employing word-level features. In order to reduce computational complexity of learning non-local information, we further present an improved online training method, which can arrive the same objective optimum with a significantly accelerated training speed. We find that the proposed method can help the learning of long range dependencies and improve the segmentation quality of long words (for example, complicated named entities). Experimental results demonstrate that the proposed method is effective. With this improvement, evaluations on the data of the second SIGHAN CWS bakeoff show that our system is competitive with the state-of-the-art systems. (C) 2012 Elsevier Ltd. All rights reserved.; Computer Science, Information Systems; Information Science & Library Science; SCI(E); EI; SSCI; 1; ARTICLE; 3; 626-636; 49 |
语种 | 英语 |
内容类型 | 期刊论文 |
源URL | [http://ir.pku.edu.cn/handle/20.500.11897/224251] ![]() |
专题 | 信息科学技术学院 |
推荐引用方式 GB/T 7714 | Sun, Xu,Zhang, Yaozhong,Matsuzaki, Takuya,et al. Probabilistic Chinese word segmentation with non-local information and stochastic training[J]. information processing management,2013. |
APA | Sun, Xu,Zhang, Yaozhong,Matsuzaki, Takuya,Tsuruoka, Yoshimasa,Tsujii, Jun&apos,&ichi.(2013).Probabilistic Chinese word segmentation with non-local information and stochastic training.information processing management. |
MLA | Sun, Xu,et al."Probabilistic Chinese word segmentation with non-local information and stochastic training".information processing management (2013). |
个性服务 |
查看访问统计 |
相关权益政策 |
暂无数据 |
收藏/分享 |
除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。
修改评论