sklearn.metrics.ndcg_score¶
sklearn.metrics.ndcg_score(y_true, y_score, *, k=None, sample_weight=None, ignore_ties=False)
计算归一化折损累计增益。
在应用对数折扣后,将真实分数的总和按预测分数的诱导顺序进行排序。 然后除以最佳分数(理想DCG,获得完全排名),得到0到1之间的分数。
如果真实标签的y_score排名高,则排名度量值高。
参数 | 说明 |
---|---|
y_true | ndarray, shape (n_samples, n_labels) 多标签分类的真实目标,或要排名的实体的真实分数。 |
y_score | ndarray, shape (n_samples, n_labels) 目标分数可以是概率估计,置信度值或决策的非阈值度量(如某些分类器上的“ decision_function”所返回)。 |
k | int, optional (default=None) 仅考虑排名中最高的k分数。 如果为None,则使用所有输出。 |
sample_weight | ndarray, shape (n_samples,), optional (default=None) 样本权重。如果为None,则所有样本的权重都相同。 |
ignore_ties | bool, optional (default=False) 假设y_score在效率增益方面不存在关联(如果y_score是连续的,则可能是这种情况)。 |
返回值 | 说明 |
---|---|
normalized_discounted_cumulative_gain | float in [0., 1.] 所有样本的平均NDCG分数。 |
另见:
折损累计增益(未归一化)。
参考
Wikipedia entry for Discounted Cumulative Gain
Jarvelin, K., & Kekalainen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS), 20(4), 422-446.
Wang, Y., Wang, L., Li, Y., He, D., Chen, W., & Liu, T. Y. (2013, May). A theoretical analysis of NDCG ranking measures. In Proceedings of the 26th Annual Conference on Learning Theory (COLT 2013)
McSherry, F., & Najork, M. (2008, March). Computing information retrieval performance measures efficiently in the presence of tied scores. In European conference on information retrieval (pp. 414-421). Springer, Berlin, Heidelberg.
示例
>>> from sklearn.metrics import ndcg_score
>>> # we have groud-truth relevance of some answers to a query:
>>> true_relevance = np.asarray([[10, 0, 0, 1, 5]])
>>> # we predict some scores (relevance) for the answers
>>> scores = np.asarray([[.1, .2, .3, 4, 70]])
>>> ndcg_score(true_relevance, scores)
0.69...
>>> scores = np.asarray([[.05, 1.1, 1., .5, .0]])
>>> ndcg_score(true_relevance, scores)
0.49...
>>> # we can set k to truncate the sum; only top k answers contribute.
>>> ndcg_score(true_relevance, scores, k=4)
0.35...
>>> # the normalization takes k into account so a perfect answer
>>> # would still get 1.0
>>> ndcg_score(true_relevance, true_relevance, k=4)
1.0
>>> # now we have some ties in our prediction
>>> scores = np.asarray([[1, 0, 0, 0, 1]])
>>> # by default ties are averaged, so here we get the average (normalized)
>>> # true relevance of our top predictions: (10 / 10 + 5 / 10) / 2 = .75
>>> ndcg_score(true_relevance, scores, k=1)
0.75
>>> # we can choose to ignore ties for faster results, but only
>>> # if we know there aren't ties in our scores, otherwise we get
>>> # wrong results:
>>> ndcg_score(true_relevance,
... scores, k=1, ignore_ties=True)
0.5