收藏切换
Automatic sentence segmentation and word segmentation for Liye Qin Bamboo manuscripts based on CRF model
收藏切换
PDF
Science & Technology Review | 2024, 42(23) : 135 - 144
Less
收藏切换
Science & Technology Review | 2024, 42(23): 135-144
Papers
Automatic sentence segmentation and word segmentation for Liye Qin Bamboo manuscripts based on CRF model
Full
FENG Huimin1,2, GUO Shuaishuai2, LIU Ming2
Affiliations
    1. Department of Basic Courses, Shandong Agricultural Engineering University, Jinan 250100, China;
    2. Institute for Advanced Study in History of Science, Northwest University, Xi'an 710127, China
Published: 2024-12-13 doi: 10.3981/j.issn.1000-7857.2023.05.00812
Outline
收藏切换
Information processing of ancient Chinese seldom uses unearthed documents as corpus to carry out relevant research. The number of Liye Qin bamboo manuscripts reached ten times that of all the Qin slips unearthed before, which can fill many gaps in the historical records of the Qin Dynasty. In this paper, we used them as experimental corpus and explored the automatic sentence segmentation and word segmentation of unearthed documents based on the CRF model. We combined the actual characteristics of the corpus and set up different feature templates to verify the generalization ability of model sequence labeling on different tasks. We set up a joint approach to sentence segmentation and word segmentation as comparative experiment to select a better performance processing plan. At the same time, a comparative experiment was designed between deep learning methods and pretrained models. The results proved that the overall performance of the joint approach in each task was improved and that the F1-score of automatic sentence segmentation and word segmentation reached 75.79% and 94.44%, respectively. Since it's faster and takes less time, this approach is more suitable for the Liye Qin bamboo slips. The research results can serve the proofreading work of the last three volumes of Liye Qin bamboo slips and the in-depth processing and construction of the corpus.
CRF model  /  Liye Qin bamboo manuscripts  /  automatic sentence segmentation  /  automatic word segmentation
FENG Huimin, GUO Shuaishuai, LIU Ming. Automatic sentence segmentation and word segmentation for Liye Qin Bamboo manuscripts based on CRF model[J]. Science & Technology Review, 2024 , 42 (23) : 135 -144 . DOI: 10.3981/j.issn.1000-7857.2023.05.00812
Year 2024 volume 42 Issue 23
PDF
720
241
Cite this Article
BibTeX
Article Info
doi: 10.3981/j.issn.1000-7857.2023.05.00812
  • Receive Date:2023-05-29
  • Online Date:2025-01-06
  • Published:2024-12-13
Article Data
Affiliations
History
  • Received:2023-05-29
  • Revised:2023-10-30
Affiliations
References
Share
https://castjournals.cast.org.cn/joweb/kjdb/EN/10.3981/j.issn.1000-7857.2023.05.00812
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数

Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)

Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT