why does scikitlearn says F1 score is ill-defined with FN bigger than 0?

Question

I run a python program that calls sklearn.metrics's methods to calculate precision and F1 score. Here is the output when there is no predicted sample:

/xxx/py2-scikit-learn/0.15.2-comp6/lib/python2.6/site-packages/sklearn/metr\

ics/metrics.py:1771: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 due to no predicted samples. 'precision', 'predicted', average, warn_for)

/xxx/py2-scikit-learn/0.15.2-comp6/lib/python2.6/site-packages/sklearn/metr\

ics/metrics.py:1771: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 due to no predicted samples.

  'precision', 'predicted', average, warn_for)

When there is no predicted sample, it means that TP+FP is 0, so

precision defined as

TP/(TP+FP)) is 0/0

not defined,

F1 score (defined as 2TP/(2TP+FP+FN)) is 0 if FN is not zero.

In my case, sklearn.metrics also returns the accuracy as 0.8, and recall as 0. So FN is not zero.

But why does scikilearn says F1 is ill-defined?

What is the definition of F1 used by Scikilearn?

Answer 1

You can take a look at this well defined Github Repository from Scikit learn developers. It has detailed information for evaluation metrics.

Generally

F1 = 2 * (precision * recall) / (precision + recall)

precision = TP/(TP+FP) 

if the predictor doesn't predict positive class overall, then precision is 0.

recall = TP/(TP+FN)

But if the predictor doesn't predict positive class - TP is 0 - recall is 0.

Finally, you are dividing 0/0.

Hope this answer helps.

To learn what is machine learning and applications of machine learning, visit the machine learning course by Intellipaat.