Research on Named Entity Recognition Method Based on Language Pre-Training Model
DOI: 10.23977/acss.2023.070111 | Downloads: 14 | Views: 256
Jiurong Fan 1
1 School of Mechanical and Electrical Engineering, Yunnan Open University, Kunming, China
Corresponding AuthorJiurong Fan
Aiming at the problem that the existing named entity recognition models have insufficient ability to recognize common unknown words in data, this paper proposes a text vectorization representation method based on language pre-training model. The program can't understand the text directly, and it can only be understood by the program after the text is converted into a numerical value. Firstly, this paper introduces the methods of word vector representation, including discrete representation and distributed representation. The traditional word vector representation method can't deal with the problem of polysemy and can't fully express semantic features. Aiming at the defects of word vector method, this paper proposes a text vectorization method based on language pre-training model. The idea of fine-tune is introduced, and the pre-training model, which completed training on massive data sets, is transferred to the People's Daily data set, and the parameters are optimized. Finally, this paper designs a comparative experiment on the People's Daily data set, compares it with the traditional word embedding methods using CBOW, Skip-gram and GloVe, analyzes the results, and verifies the effectiveness of the proposed method.
KEYWORDSNatural language processing, language pre-training model, transfer learning
CITE THIS PAPER
Jiurong Fan. Research on Named Entity Recognition Method Based on Language Pre-Training Model. Advances in Computer, Signals and Systems (2023) Vol. 7: 82-90. DOI: http://dx.doi.org/10.23977/acss.2023.070111.
 Goelz S E, Hamilton S R, Vogelstein B. Purification of DNA from formaldehyde fixed and paraffin embedded human tissue [J]. Biochem Biophys Res Commun, 1985, 130(1):118-126.
 Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017:1-1.
 Bengio Y, Schwenk H, Jean-Sébastien Sené ca l, et al. Neural Probabilistic Language Models[J]. The Journal of Machine Learning Research, 2003, 3(6):1137-1155.
 Mateusz Szczepański, Pawlicki M, Kozik R, et al. The Application of Deep Learning Imputation and Other Advanced Methods for Handling Missing Values in Network Intrusion Detection [J]. Vietnam Journal of Computer Science, 2023, 10(01):1-23.
 Buczak A, Guven E. A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection [J]. IEEE Communications Surveys & Tutorials, 2017, 18(2):1153-1176.
 Pi X, Iijima B A, Lu W. Effects of Ionospheric Scintillation on GNSS‐Based Positioning [J]. Navigation, 2017, 64(1):3-22.
 Chandrashekar S, Bashel B, Balasubramanya H, et al. UALCAN: A Portal for Facilitating Tumor Subgroup Gene Expression and Survival Analyses [J]. Neoplasia (New York, N.Y.), 2017, 19(8):649-658.
 Du J, Cheng K, Yu Y, et al. Panchromatic Image Super-Resolution Via Self Attention-Augmented Wasserstein Generative Adversarial Network [J]. Sensors, 2021, 21(6):2158.
 Markley J L, R Brüschweiler, Edison A S, et al. The future of NMR-based metabolomics [J]. Current Opinion in Biotechnology, 2017, 43:34-40.