Performance evaluation of Baidu, Google and Bing（搜索引擎的性能評估）
Link to Github:https://github.com/yip522364642/Evaluation-of-search-engines
1 Data preparation
- Compose these 3 queries:
query1='網絡信息檢索的性能評估'
query2='蘋果新品發佈會'
query3='2018乒乓球男子世界盃'
- Run these 3 queries on Baidu, Bing and Google search engines, and collect the top 20 documents returned by each SE. Totally, each query corresponds to 60 documents.
- Judge the relevance between query and documents. If relevant, mark 1 before the website of document. Otherwise, mark 0.
- Union related documents which marks 1 from three SEs as full relevant document set of the query.
2 Evaluation Procedure
For each query in 3 SEs, if the returned document is relevant to it, calculate the P/R value at this time and record the result.
In addition, calculate the P-value when the SE returns 20 documents so as to calculate MAP of each SE.
3 Result Analysis
Draw a P/R graph for each query, and consider x axis as Recall Rate, y axis as Precision Rate.
P/R value(query1='網絡信息檢索的性能評估')
P/R value(query2='蘋果新品發佈會')
P/R value(query3='2018乒乓球男子世界盃')
MAP value: {'Baidu': 0.7000000000000001, 'Bing': 0.65, 'Google': 0.7166666666666667}
The higher the MAP value is, the better average performance SE has. As is shown above, we can conclude the performance of these 3 SEs from high to low: Google, Baidu, and Bing. Meanwhile, the MAP value of Google and Baidu is closed, but Bing’s is obviously lower. Hence, the average performance of Google and Baidu is similar and Bing’s is worse.
Link to Github:https://github.com/yip522364642/Evaluation-of-search-engines