How to implement the ReLU function in Numpy

Question

I want to make a simple neural network and I wish to use the ReLU function. Can someone give me a clue of how can I implement the function using NumPy. Thanks for your time!

Answer 1

You can build ReLU function in NumPy easily using NumPy arrays and math functions together.

For example:

>>> x = np.random.random((3, 2)) - 0.5

>>> x

array([[-0.00590765,  0.18932873],

       [-0.32396051,  0.25586596],

       [ 0.22358098,  0.02217555]])

>>> np.maximum(x, 0)

array([[ 0.        , 0.18932873],

       [ 0.        , 0.25586596],

       [ 0.22358098,  0.02217555]])

>>> x * (x > 0)

array([[-0.        , 0.18932873],

       [-0.        , 0.25586596],

       [ 0.22358098,  0.02217555]])

>>> (abs(x) + x) / 2

array([[ 0.        , 0.18932873],

       [ 0.        , 0.25586596],

       [ 0.22358098,  0.02217555]])

If you want to timing of the following function, then you can use %%time magic function, which helps you to clear the problem more.

import numpy as np

x = np.random.random((5000, 5000)) - 0.5

print("max method:")

%timeit -n10 np.maximum(x, 0)

print("multiplication method:")

%timeit -n10 x * (x > 0)

print("abs method:")

%timeit -n10 (abs(x) + x) / 2

The output of the above code:

max method:

10 loops, best of 3: 239 ms per loop

multiplication method:

10 loops, best of 3: 145 ms per loop

abs method:

10 loops, best of 3: 288 ms per loop

Hope this answer helps.

To know more visit this Python NumPy Tutorial.