Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

Jinshuo Liu; Yusen Chen; Juan Deng; Donghong Ji; Jeff Pan

doi:10.1007/978-3-319-69005-6_19

Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

Jinshuo Liu, Yusen Chen, Juan Deng^*, Donghong Ji, Jeff Pan

^*Corresponding author for this work

Computing Science

Wuhan University

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Abstract

One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).

Original language	English
Title of host publication	Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings
Editors	Maosong Sun, Baobao Chang, Xiaojie Wang, Deyi Xiong
Publisher	Springer-Verlag
Pages	224-236
Number of pages	13
ISBN (Electronic)	9783319690056
ISBN (Print)	9783319690049
DOIs	https://doi.org/10.1007/978-3-319-69005-6_19
Publication status	Published - 7 Oct 2017
Event	16th China National Conference on Computational Linguistics, CCL 2017 and 5th International Symposium on Natural Language Processing Based on Naturally Annotated Big Data, NLP-NABD 2017 - Nanjing, China Duration: 13 Oct 2017 → 15 Oct 2017

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10565 LNAI
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	16th China National Conference on Computational Linguistics, CCL 2017 and 5th International Symposium on Natural Language Processing Based on Naturally Annotated Big Data, NLP-NABD 2017
Country/Territory	China
City	Nanjing
Period	13/10/17 → 15/10/17

Keywords

Chinese abbreviation
Collaborative recovery
Improved CRF

Access to Document

10.1007/978-3-319-69005-6_19

Cite this

Liu, J., Chen, Y., Deng, J., Ji, D., & Pan, J. (2017). Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. In M. Sun, B. Chang, X. Wang, & D. Xiong (Eds.), Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings (pp. 224-236). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10565 LNAI). Springer-Verlag. https://doi.org/10.1007/978-3-319-69005-6_19

Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. / Liu, Jinshuo; Chen, Yusen; Deng, Juan et al.
Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings. ed. / Maosong Sun; Baobao Chang; Xiaojie Wang; Deyi Xiong. Springer-Verlag, 2017. p. 224-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10565 LNAI).

Research output: Chapter in Book/Report/Conference proceeding › Published conference contribution

Liu, J, Chen, Y, Deng, J, Ji, D & Pan, J 2017, Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. in M Sun, B Chang, X Wang & D Xiong (eds), Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10565 LNAI, Springer-Verlag, pp. 224-236, 16th China National Conference on Computational Linguistics, CCL 2017 and 5th International Symposium on Natural Language Processing Based on Naturally Annotated Big Data, NLP-NABD 2017, Nanjing, China, 13/10/17. https://doi.org/10.1007/978-3-319-69005-6_19

Liu J, Chen Y, Deng J, Ji D, Pan J. Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. In Sun M, Chang B, Wang X, Xiong D, editors, Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings. Springer-Verlag. 2017. p. 224-236. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-69005-6_19

Liu, Jinshuo ; Chen, Yusen ; Deng, Juan et al. / Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation. Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings. editor / Maosong Sun ; Baobao Chang ; Xiaojie Wang ; Deyi Xiong. Springer-Verlag, 2017. pp. 224-236 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{099b924d7e91483088f872ff92986d6e,

title = "Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation",

abstract = "One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).",

keywords = "Chinese abbreviation, Collaborative recovery, Improved CRF",

author = "Jinshuo Liu and Yusen Chen and Juan Deng and Donghong Ji and Jeff Pan",

year = "2017",

month = oct,

day = "7",

doi = "10.1007/978-3-319-69005-6_19",

language = "English",

isbn = "9783319690049",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer-Verlag",

pages = "224--236",

editor = "Maosong Sun and Baobao Chang and Xiaojie Wang and Deyi Xiong",

booktitle = "Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings",

note = "16th China National Conference on Computational Linguistics, CCL 2017 and 5th International Symposium on Natural Language Processing Based on Naturally Annotated Big Data, NLP-NABD 2017 ; Conference date: 13-10-2017 Through 15-10-2017",

}

TY - GEN

T1 - Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

AU - Liu, Jinshuo

AU - Chen, Yusen

AU - Deng, Juan

AU - Ji, Donghong

AU - Pan, Jeff

PY - 2017/10/7

Y1 - 2017/10/7

N2 - One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).

AB - One of the important works of Information Content Security is evaluating the theme words of the text. Because of the variety of the Chinese expression, especially of the abbreviation, the supervision of the theme words becomes harder. The goal of this paper is to quickly and accurately discover the intercept abbreviations from the text crawled at the short time period. The paper firstly segments the target texts, and then utilizes the Supported Vector Machine (SVM) to recognize the abbreviations from the wrongly segmented texts as the candidates. Secondly, this paper presents the collaborative methods: Improve the Conditional Random Fields (CRF) to predict the corresponding word to each character of the abbreviation; To solve the problems of the 1:n relationship, collaboratively merge the ranking list from the predict steps with the matched results of the thesaurus of abbreviations. The experiments demonstrate that our method at the recognizing stage is 76.5% of the accuracy and 77.8% of the recall rate. At the recovery step, the accuracy is 62.1%, which is 20.8% higher than the method based on Hidden Markov Model (HMM).

KW - Chinese abbreviation

KW - Collaborative recovery

KW - Improved CRF

UR - http://www.scopus.com/inward/record.url?scp=85031411561&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-69005-6_19

DO - 10.1007/978-3-319-69005-6_19

M3 - Published conference contribution

AN - SCOPUS:85031411561

SN - 9783319690049

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 224

EP - 236

BT - Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data - 16th China National Conference, CCL 2017 and 5th International Symposium, NLP-NABD 2017, Proceedings

A2 - Sun, Maosong

A2 - Chang, Baobao

A2 - Wang, Xiaojie

A2 - Xiong, Deyi

PB - Springer-Verlag

T2 - 16th China National Conference on Computational Linguistics, CCL 2017 and 5th International Symposium on Natural Language Processing Based on Naturally Annotated Big Data, NLP-NABD 2017

Y2 - 13 October 2017 through 15 October 2017

ER -

Collaborative Recognition and Recovery of the Chinese Intercept Abbreviation

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this