Analysing the Current State of Development of Bert-Based Text Classification in Natural Language Processing

College of Sciences, Nanjing Agricultural University, Nanjing, Jiangsu, 210008, China
College of Economics and Management, Nanjing Forestry University, Nanjing, Jiangsu, 210037, China

^* 731224143@njfu.edu.cn

Abstract

Text classification is an important task in natural language processing, and traditional text classification methods have certain limitations, such as low accuracy, low flexibility and high disadvantages in terms of computational resources and time cost. With the development of deep learning, the BERT text classification model shows what advantages can compensate for the limitations of traditional methods. This paper aims to explore the development of BERT for text classification from three aspects: the birth of BERT, the optimisation and improvement of BERT, and the architectural innovation and application extension. This paper concludes that BERT has brought text classification to a new stage of 'pre-training + fine-tuning'. Its variants are more effective than other methods for binary sentiment classification, Alzheimer's disease detection and power audit text classification. However, the BERT model suffers from high computational resource requirements, high efficiency compromised by computational complexity, insufficient model interpretability, and low adaptability to specific domains and small datasets. In the future, we can promote the development of efficient model architectures and training methods, focus on interpretable model architectures and tools, and improve the adaptability of BERT in specific domains.