Softmax as compared to standard normalization, it performs exponential normalization, that means its output directly depends upon the uniform distribution of input. While the output of normal distribution does not get affected until the ratio proportion is the same.
The formula for Softmax function:
The formula for standard deviation:
Example for softmax function:
>>> softmax([1,2]) # blurry image of a ferret
[0.26894142, 0.73105858]) # it is a cat perhaps !?
>>> softmax([10,20]) # crisp image of a cat
[0.0000453978687, 0.999954602]) # it is definitely a CAT !
Example for standard normalization:
>>> std_norm([1,2]) # blurry image of a ferret
[0.3333333333333333, 0.6666666666666666] # it is a cat perhaps
>>> std_norm([10,20]) # crisp image of a cat
[0.3333333333333333, 0.6666666666666666] # it is a cat perhaps !?
In the above example, softmax predicts more accurately when the image resolution is higher, but the standard normalization function predicted the same probability in lower and higher resolution of the image. That’s why Softmax is most commonly used in neural network.