Differences between numpy.random.rand vs numpy.random.randn in Python

Question

What are all the differences between numpy.random.rand and numpy.random.randn?

From the docs, I know that the only difference among them is from the probabilistic distribution each number is drawn from, but the overall structure (dimension) and data type used (float) are the same. I have a hard time debugging a neural network because of believing this.

Specifically, I am trying to re-implement the Neural Network provided in the Neural Network and Deep Learning book by Michael Nielson. The original code can be found here. My implementation was the same as the original one, except that I defined and initialized weights and biases with numpy.random.rand in init function, rather than numpy.random.randn as in the original.

However, my code that use random.rand to initialize weights and biases doesn't work because the network won't learn and the weights and biases are will not change.

What difference(s) among two random functions cause this weirdness?

Answer 1

When you will look at the documentation of numpy you will see that the numpy.random.randn generates samples from the normal distribution, while numpy.random.rand from uniform (in range [0,1)).

The main reason in this is an activation function, especially in your case where you use the sigmoid function. The plot of the sigmoid looks like the following:

To know more about this you can have a look at the following video tutorial:-