Explore Courses Blog Tutorials Interview Questions
0 votes
in Machine Learning by (19k points)
Is there a chance to get the value of information gain be negative? It is calculated according to the formula in the following paper. I cannot write the formula, because it includes some hard notations.


1 Answer

0 votes
by (33.1k points)
In your problem, IG(Y|X) = H(Y) - H(Y|X) >= 0 , since H(Y) >= H(Y|X) worst case is that X and Y are independent, thus H(Y|X)=H(Y).

You can also say that observing the random variable X taking some value, we either gain no or some information about Y.

Hope this answer helps you!

Browse Categories