Author(s)

Jiurong Fan ¹

Affiliation(s)

¹ School of Mechanical and Electrical Engineering, Yunnan Open University, Kunming, China

Corresponding Author

Jiurong Fan

ABSTRACT

Aiming at the problem that the existing named entity recognition models have insufficient ability to recognize common unknown words in data, this paper proposes a text vectorization representation method based on language pre-training model. The program can't understand the text directly, and it can only be understood by the program after the text is converted into a numerical value. Firstly, this paper introduces the methods of word vector representation, including discrete representation and distributed representation. The traditional word vector representation method can't deal with the problem of polysemy and can't fully express semantic features. Aiming at the defects of word vector method, this paper proposes a text vectorization method based on language pre-training model. The idea of fine-tune is introduced, and the pre-training model, which completed training on massive data sets, is transferred to the People's Daily data set, and the parameters are optimized. Finally, this paper designs a comparative experiment on the People's Daily data set, compares it with the traditional word embedding methods using CBOW, Skip-gram and GloVe, analyzes the results, and verifies the effectiveness of the proposed method.

KEYWORDS

Natural language processing, language pre-training model, transfer learning

CITE THIS PAPER

Jiurong Fan. Research on Named Entity Recognition Method Based on Language Pre-Training Model. Advances in Computer, Signals and Systems (2023) Vol. 7: 82-90. DOI: http://dx.doi.org/10.23977/acss.2023.070111.

REFERENCES

[1] Goelz S E, Hamilton S R, Vogelstein B. Purification of DNA from formaldehyde fixed and paraffin embedded human tissue [J]. Biochem Biophys Res Commun, 1985, 130(1):118-126.
[2] Kendall A, Cipolla R. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017:1-1.
[3] Bengio Y, Schwenk H, Jean-Sébastien Sené ca l, et al. Neural Probabilistic Language Models[J]. The Journal of Machine Learning Research, 2003, 3(6):1137-1155.
[4] Mateusz Szczepański, Pawlicki M, Kozik R, et al. The Application of Deep Learning Imputation and Other Advanced Methods for Handling Missing Values in Network Intrusion Detection [J]. Vietnam Journal of Computer Science, 2023, 10(01):1-23.
[5] Buczak A, Guven E. A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection [J]. IEEE Communications Surveys & Tutorials, 2017, 18(2):1153-1176.
[6] Pi X, Iijima B A, Lu W. Effects of Ionospheric Scintillation on GNSS‐Based Positioning [J]. Navigation, 2017, 64(1):3-22.
[7] Chandrashekar S, Bashel B, Balasubramanya H, et al. UALCAN: A Portal for Facilitating Tumor Subgroup Gene Expression and Survival Analyses [J]. Neoplasia (New York, N.Y.), 2017, 19(8):649-658.
[8] Du J, Cheng K, Yu Y, et al. Panchromatic Image Super-Resolution Via Self Attention-Augmented Wasserstein Generative Adversarial Network [J]. Sensors, 2021, 21(6):2158.
[9] Markley J L, R Brüschweiler, Edison A S, et al. The future of NMR-based metabolomics [J]. Current Opinion in Biotechnology, 2017, 43:34-40.

Sponsors, Associates, and Links

---

• Power Systems Computation
  
  
• Internet of Things (IoT) and Engineering Applications
  
  
• Computing, Performance and Communication Systems
  
  
• Journal of Artificial Intelligence Practice
  
  
• Journal of Network Computing and Applications
  
  
• Journal of Web Systems and Applications
  
  
• Journal of Electrotechnology, Electrical Engineering and Management
  
  
• Journal of Wireless Sensors and Sensor Networks
  
  
• Journal of Image Processing Theory and Applications
  
  
• Mobile Computing and Networking
  
  
• Vehicle Power and Propulsion
  
  
• Frontiers in Computer Vision and Pattern Recognition
  
  
• Knowledge Discovery and Data Mining Letters
  
  
• Big Data Analysis and Cloud Computing
  
  
• Electrical Insulation and Dielectrics
  
  
• Crypto and Information Security
  
  
• Journal of Neural Information Processing
  
  
• Collaborative and Social Computing
  
  
• International Journal of Network and Communication Technology
  
  
• File and Storage Technologies
  
  
• Frontiers in Genetic and Evolutionary Computation
  
  
• Optical Network Design and Modeling
  
  
• Journal of Virtual Reality and Artificial Intelligence
  
  
• Natural Language Processing and Speech Recognition
  
  
• Journal of High-Voltage
  
  
• Programming Languages and Operating Systems
  
  
• Visual Communications and Image Processing
  
  
• Journal of Systems Analysis and Integration
  
  
• Knowledge Representation and Automated Reasoning
  
  
• Review of Information Display Techniques
  
  
• Data and Knowledge Engineering
  
  
• Journal of Database Systems
  
  
• Journal of Cluster and Grid Computing
  
  
• Cloud and Service-Oriented Computing
  
  
• Journal of Networking, Architecture and Storage
  
  
• Journal of Software Engineering and Metrics
  
  
• Visualization Techniques
  
  
• Journal of Parallel and Distributed Processing
  
  
• Journal of Modeling, Analysis and Simulation
  
  
• Journal of Privacy, Trust and Security
  
  
• Journal of Cognitive Informatics and Cognitive Computing
  
  
• Lecture Notes on Wireless Networks and Communications
  
  
• International Journal of Computer and Communications Security
  
  
• Journal of Multimedia Techniques
  
  
• Automation and Machine Learning
  
  
• Computational Linguistics Letters
  
  
• Journal of Computer Architecture and Design
  
  
• Journal of Ubiquitous and Future Networks