This dataset contains a collection of keyphraseness values for phrases extracted from Wikipedia articles. The keyphraseness value Q(s) of a phrase s is the probability that the phrase appears in a Wikipedia article as being anchor text. In total, 4,342,732
phrases are extracted from the English Wikipedia dump created on January 30, 2010. In this release, we remove the 184,979 phrases containing non-English characters. Among the remaining 4,157,753 phrases, about 1.9 million phrases have non-zero keyphraseness
values. This dataset contains one text file and a readme file in zip format (about 45MB in size). Each line is a mapping: [phrase],[keyphraseness value] (e.g., jackie_chan, 0.9509918319719953).
This dataset has been used in the following 3 papers. Please refer to the papers for more details about the dataset and how the keyphraseness values can be used in various tasks (All papers can be downloaded freely from ACM digital library using the links below).
This dataset is released solely for research purposes.
Please cite at least one of the following 3 papers if you use this dataset in your research.
·Chenliang Li, Aixin Sun, Jianshu Weng, Qi He. Exploiting hybrid contexts for Tweet segmentation. SIGIR 2013
·Chenliang Li, Aixin Sun, Anwitaman Datta. Twevent: segment-based event detection from tweets. CIKM 2012
·Chenliang Li, Jianshu Weng, Qi He, Yuxia Yao, Anwitaman Datta, Aixin Sun, Bu-Sung Lee. TwiNER: named entity recognition in targeted twitter stream SIGIR 2012.