Can the value of information gain be negative?

Question

Is there a chance to get the value of information gain be negative? It is calculated according to the formula in the following paper. I cannot write the formula, because it includes some hard notations.

Thanks!

Answer 1

In your problem, IG(Y|X) = H(Y) - H(Y|X) >= 0 , since H(Y) >= H(Y|X) worst case is that X and Y are independent, thus H(Y|X)=H(Y).

You can also say that observing the random variable X taking some value, we either gain no or some information about Y.

Hope this answer helps you!