Simple multi layer neural network implementation

Question

some time ago I have started my adventure with machine learning (during last 2 years of my studies). I have read a lot of books and written a lot of code with machine learning algorithms EXCEPT neural networks, which were out of my scope. I'm very interested in this topic, but I have a huge problem: All the books I have read have two main issues:

Contain tones of maths equations. After lecture I'm quite familiar with them and by hand, on the paper I can do the calculations.

Contain big examples embedded in some complicated context (for example investigating internet shop sales rates) and to get inside neural networks implementation, I have to write lot of code to reproduce the context. What is missing - SIMPLE straightforward implementation without a lot of context and equations.

Could you please advise me, where I can find SIMPLE implementation of multi layer perception (neural network) ? I don't need theoretical knowledge, and don want also context-embedded examples. I prefer some scripting languages to save time and effort - 99% of my previous works were done in Python.

Here is the list of books I have read before (and not found what I wanted):

• Machine learning in action
• Programming Collective Intelligence
• Machine Learning: An Algorithmic Perspective
• Introduction to neural networks in Java
• Introduction to neural networks in C#

Answer 1

Here is an implementation of multi-layer neural network:

For example:

    import math

    import random

    BIAS = -1

       class Neuron:

        def __init__(self, n_inputs ):

            self.n_inputs = n_inputs

            self.set_weights( [random.uniform(0,1) for x in range(0,n_inputs+1)] ) 

        def sum(self, inputs ):

            # Does not include the bias

            return sum(val*self.weights[i] for i,val in enumerate(inputs))

        def set_weights(self, weights ):

            self.weights = weights

        def __str__(self):

            return 'Weights: %s, Bias: %s' % ( str(self.weights[:-1]),str(self.weights[-1]) )

    class NeuronLayer:

        def __init__(self, n_neurons, n_inputs):

            self.n_neurons = n_neurons

            self.neurons = [Neuron( n_inputs ) for _ in range(0,self.n_neurons)]

        def __str__(self):

            return 'Layer:\n\t'+'\n\t'.join([str(neuron) for neuron in self.neurons])+''

    class NeuralNetwork:

        def __init__(self, n_inputs, n_outputs, n_neurons_to_hl, n_hidden_layers):

            self.n_inputs = n_inputs

            self.n_outputs = n_outputs

            self.n_hidden_layers = n_hidden_layers

            self.n_neurons_to_hl = n_neurons_to_hl

            self._create_network()

            self._n_weights = None

        def _create_network(self):

            if self.n_hidden_layers>0:

                # create the first layer

                self.layers = [NeuronLayer( self.n_neurons_to_hl,self.n_inputs )]

                # create hidden layers

                self.layers += [NeuronLayer( self.n_neurons_to_hl,self.n_neurons_to_hl ) for _ in range(0,self.n_hidden_layers)]

                # hidden-to-output layer

                self.layers += [NeuronLayer( self.n_outputs,self.n_neurons_to_hl )]

            else:

                # If we don't require hidden layers

                self.layers = [NeuronLayer( self.n_outputs,self.n_inputs )]

        def get_weights(self):

            weights = []

            for layer in self.layers:

                for neuron in layer.neurons:

                    weights += neuron.weights

            return weights

             def n_weights(self):

            if not self._n_weights:

                self._n_weights = 0

                for layer in self.layers:

                    for neuron in layer.neurons:

                        self._n_weights += neuron.n_inputs+1 # +1 for bias weight

            return self._n_weights

        def set_weights(self, weights ):

            assert len(weights)==self.n_weights, "Incorrect amount of weights."

            stop = 0

            for layer in self.layers:

                for neuron in layer.neurons:

                    start, stop = stop, stop+(neuron.n_inputs+1)

                    neuron.set_weights( weights[start:stop] )

            return self

        def update(self, inputs ):

            assert len(inputs)==self.n_inputs, "Incorrect amount of inputs."

            for layer in self.layers:

                outputs = []

                for neuron in layer.neurons:

                    tot = neuron.sum(inputs) + neuron.weights[-1]*BIAS

                    outputs.append( self.sigmoid(tot) )

                inputs = outputs   

            return outputs

        def sigmoid(self, activation,response=1 ):

                      try:

                return 1/(1+math.e**(-activation/response))

            except OverflowError:

                return float("inf")

        def __str__(self):

            return '\n'.join([str(i+1)+' '+str(layer) for i,layer in enumerate(self.layers)])

Hope this answer helps.