Article(id=1242134504840704125, tenantId=1146029695717560320, journalId=1146031591421210625, issueId=1242134390566887892, articleNumber=null, orderNo=54, doi=10.3981/j.issn.1000-7857.2016.2.048, pmid=null, cstr=null, oa=null, hot=null, price=null, onlineType=0, articleFormat=0, articleType=null, articleTypeStr=null, receivedDate=1429718400000, receivedDateStr=2015-04-23, revisedDate=1433692800000, revisedDateStr=2015-06-08, acceptedDate=null, acceptedDateStr=null, onlineDate=1454555982870, onlineDateStr=2016-02-04, pubDate=1453910400000, pubDateStr=2016-01-28, doiRegisterDate=null, doiRegisterDateStr=null, onlineIssueDate=1454555982870, onlineIssueDateStr=2016-02-04, onlineJustAcceptDate=null, onlineJustAcceptDateStr=null, onlineFirstDate=null, onlineFirstDateStr=null, sourceXml=null, magXml=null, createTime=1774077543688, creator=sys-migrate, updateTime=1774077543688, updator=sys-migrate, issue=Issue{id=1242134390566887892, tenantId=1146029695717560320, journalId=1146031591421210625, year='2016', volume='34', issue='2', pageStart='1', pageEnd='332', issueExtLink='null', onlineDate='null', pubDate='1453910400000', pubDateStr='2016-01-28', beforeIssueId=null, nextIssueId=null, price=null, status=1, issueComplete=1, articleOrder=3, issueType=-1, specialIssue=null, createTime=1774077516449, creator='sys-migrate', updateTime=1774077516449, updator='sys-migrate', preIssue=null, nextIssue=null, articleTotal=null, ext=null, issueFiles=null, downloadFileDto=null}, startPage=282, endPage=286, ext={EN=ArticleExt(id=1242134508284223609, articleId=1242134504840704125, tenantId=1146029695717560320, journalId=1146031591421210625, language=EN, title=Weibo topic detection based on improved TF-IDF algorithm, columnId=1242116810380743325, journalTitle=Science & Technology Review, columnName=Reviews, runingTitle=null, highlight=null, articleAbstract=The topic detection and tracking (TDT) is an issue of natural language processing, which concerns with solving the problem of information explosion. The Weibo TDT is a central issue in recent years. A bad performance is usually achieved for Weibo with a short text, while the topic detection of a long text is widely used in the industry with better results. Weibo's features of short text and not very clear meaning make the clustering algorithms' effect not ideal in topic detection. So this paper focuses on finding a new way to improve the effect of clustering for Weibo. Weibo features fast renewal and strong timeliness. Hot topics produced by Weibo show burstiness, and their representative words increase in a great extent. With this feature in mind, improving the representative word's weight to a certain degree is a good way to give a prominence to the feature of short text. The burstiness of the words is a thing to consider, similar to the kinetic theory of the object. The formula of the kinetic energy theorem is used in this paper. Then an improved feature extraction algorithm named the TFIDF-KE (term frequency-inverse document frequency-kinetic energy) is proposed. The new algorithm consists of the kinetic energy and the TF-IDF (term frequency-inverse document frequency). The formula of the kinetic energy theorem is used to evaluate the burstiness of the words and add the value to the formula. Then, the weight of some important words can be improved when extracting features. Finally, the implementation of the CURE (clustering using representatives) algorithm completes the Weibo topic detection task. The method presented in this paper describes burstiness of text and feature and solves the problem that the feature of bursty hot topics is not obvious, when clustering in a certain extent. The experimental results show that the method can effectively improve the effect of topic detection in some degree and a better accuracy rate P can be achieved, as well as the R and F values of the recall rate. So TF-IDF-KE is an effective optimization method and can well be used for the task of the TDT., authors=CHEN Shuoying1 , JIN Zhensheng2 , authorsList=CHEN Shuoying, JIN Zhensheng, authorCompany=1. Department of Network Information Center, Beijing Institute of Technology, Beijing 100081, China;1 , 金镇晟2 , authorsList=陈朔鹰, 金镇晟, authorCompany=1. 北京理工大学网络信息中心, 北京 100081;
科技导报
|综述文章
2016
, 34
(2) :
282
-286
基于改进的TF-IDF算法的微博话题检测
全屏
陈朔鹰1 , 金镇晟2
作者信息
1. 北京理工大学网络信息中心, 北京 100081;
作者简介:
陈朔鹰,副教授,研究方向为计算机网络、数据挖掘、物联网,电子信箱:chensy@bit.edu.cn
Weibo topic detection based on improved TF-IDF algorithm
CHEN Shuoying1 , JIN Zhensheng2
Affiliations
1. Department of Network Information Center, Beijing Institute of Technology, Beijing 100081, China;
出版时间: 2016-01-28
doi: 10.3981/j.issn.1000-7857.2016.2.048
文章导航
中文微博具有更新快、时效性强等特点,产生的热点话题均具有一定的突发性,与此同时文本中有代表性的特征词也会随之激增。利用这一特性,在传统的TF-IDF(term frequency-inverse document frequency)基础上提出一种改进的特征权重算法,称之为TF-IDF-KE(term frequency-inverse document frequency-kinetic energy),用以解决突发性热点话题在聚类时特征不明显的问题。该算法结合物体的动能原理,将特征项的突发值用动能的概念进行描述,加入权值计算,提高突发性特征项的权重,最后使用CURE(clustering using representatives)算法,实现微博的话题检测。该方法描述了文本和特征项所具有的动态属性,实验结果表明,该方法能够有效地提高话题检测的效果。
微博
/
TF-IDF
/
话题检测
/
TDT
/
文本聚类
The topic detection and tracking (TDT) is an issue of natural language processing, which concerns with solving the problem of information explosion. The Weibo TDT is a central issue in recent years. A bad performance is usually achieved for Weibo with a short text, while the topic detection of a long text is widely used in the industry with better results. Weibo's features of short text and not very clear meaning make the clustering algorithms' effect not ideal in topic detection. So this paper focuses on finding a new way to improve the effect of clustering for Weibo. Weibo features fast renewal and strong timeliness. Hot topics produced by Weibo show burstiness, and their representative words increase in a great extent. With this feature in mind, improving the representative word's weight to a certain degree is a good way to give a prominence to the feature of short text. The burstiness of the words is a thing to consider, similar to the kinetic theory of the object. The formula of the kinetic energy theorem is used in this paper. Then an improved feature extraction algorithm named the TFIDF-KE (term frequency-inverse document frequency-kinetic energy) is proposed. The new algorithm consists of the kinetic energy and the TF-IDF (term frequency-inverse document frequency). The formula of the kinetic energy theorem is used to evaluate the burstiness of the words and add the value to the formula. Then, the weight of some important words can be improved when extracting features. Finally, the implementation of the CURE (clustering using representatives) algorithm completes the Weibo topic detection task. The method presented in this paper describes burstiness of text and feature and solves the problem that the feature of bursty hot topics is not obvious, when clustering in a certain extent. The experimental results show that the method can effectively improve the effect of topic detection in some degree and a better accuracy rate P can be achieved, as well as the R and F values of the recall rate. So TF-IDF-KE is an effective optimization method and can well be used for the task of the TDT.
Weibo
/
TF-IDF
/
topic detection
/
TDT
/
text clustering
陈朔鹰, 金镇晟.
基于改进的TF-IDF算法的微博话题检测.
科技导报,
2016
, 34
(2)
: 282
-286
.
DOI: 10.3981/j.issn.1000-7857.2016.2.048
CHEN Shuoying, JIN Zhensheng.
Weibo topic detection based on improved TF-IDF algorithm[J].
Science & Technology Review ,
2016
, 34
(2)
: 282
-286
.
DOI: 10.3981/j.issn.1000-7857.2016.2.048
2016年第34卷第2期
PDF下载
635
142
引用本文
BibTeX
文章信息
doi: 10.3981/j.issn.1000-7857.2016.2.048
接收时间：2015-04-23
首发时间：2016-02-04
出版时间：2016-01-28
收稿日期：2015-04-23
修回日期：2015-06-08
https://castjournals.cast.org.cn/joweb/kjdb/CN/10.3981/j.issn.1000-7857.2016.2.048
引用本文
BibTeX
2种不同金属材料的力学参数
科 属数 种数 占总种数比例 属 种数 占总种数比例 鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78 小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39 多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39 红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87 小菇属 Mycena 11 5.26 光柄菇属 Pluteus 5 2.39 红菇属 Russula 17 8.13 栓菌属 Trametes 5 2.39
关闭全屏
BibTeX
EndNote
RefWorks
TxT
PDF下载
全屏
635
142
关闭全屏
北京市海淀区学院南路86号
010-62199257（仅加盟咨询）
qkjq@cast.org.cn
