2 views

some time ago I have started my adventure with machine learning (during last 2 years of my studies). I have read a lot of books and written a lot of code with machine learning algorithms EXCEPT neural networks, which were out of my scope. I'm very interested in this topic, but I have a huge problem: All the books I have read have two main issues:

Contain tones of maths equations. After lecture I'm quite familiar with them and by hand, on the paper I can do the calculations.

Contain big examples embedded in some complicated context (for example investigating internet shop sales rates) and to get inside neural networks implementation, I have to write lot of code to reproduce the context. What is missing - SIMPLE straightforward implementation without a lot of context and equations.

Could you please advise me, where I can find SIMPLE implementation of multi layer perception (neural network) ? I don't need theoretical knowledge, and don want also context-embedded examples. I prefer some scripting languages to save time and effort - 99% of my previous works were done in Python.

• Machine learning in action
• Programming Collective Intelligence
• Machine Learning: An Algorithmic Perspective
• Introduction to neural networks in Java
• Introduction to neural networks in C#

by (33.1k points)

Here is an implementation of multi-layer neural network:

For example:

import math

import random

BIAS = -1

class Neuron:

def __init__(self, n_inputs ):

self.n_inputs = n_inputs

self.set_weights( [random.uniform(0,1) for x in range(0,n_inputs+1)] )

def sum(self, inputs ):

# Does not include the bias

return sum(val*self.weights[i] for i,val in enumerate(inputs))

def set_weights(self, weights ):

self.weights = weights

def __str__(self):

return 'Weights: %s, Bias: %s' % ( str(self.weights[:-1]),str(self.weights[-1]) )

class NeuronLayer:

def __init__(self, n_neurons, n_inputs):

self.n_neurons = n_neurons

self.neurons = [Neuron( n_inputs ) for _ in range(0,self.n_neurons)]

def __str__(self):

return 'Layer:\n\t'+'\n\t'.join([str(neuron) for neuron in self.neurons])+''

class NeuralNetwork:

def __init__(self, n_inputs, n_outputs, n_neurons_to_hl, n_hidden_layers):

self.n_inputs = n_inputs

self.n_outputs = n_outputs

self.n_hidden_layers = n_hidden_layers

self.n_neurons_to_hl = n_neurons_to_hl

self._create_network()

self._n_weights = None

def _create_network(self):

if self.n_hidden_layers>0:

# create the first layer

self.layers = [NeuronLayer( self.n_neurons_to_hl,self.n_inputs )]

# create hidden layers

self.layers += [NeuronLayer( self.n_neurons_to_hl,self.n_neurons_to_hl ) for _ in range(0,self.n_hidden_layers)]

# hidden-to-output layer

self.layers += [NeuronLayer( self.n_outputs,self.n_neurons_to_hl )]

else:

# If we don't require hidden layers

self.layers = [NeuronLayer( self.n_outputs,self.n_inputs )]

def get_weights(self):

weights = []

for layer in self.layers:

for neuron in layer.neurons:

weights += neuron.weights

return weights

def n_weights(self):

if not self._n_weights:

self._n_weights = 0

for layer in self.layers:

for neuron in layer.neurons:

self._n_weights += neuron.n_inputs+1 # +1 for bias weight

return self._n_weights

def set_weights(self, weights ):

assert len(weights)==self.n_weights, "Incorrect amount of weights."

stop = 0

for layer in self.layers:

for neuron in layer.neurons:

start, stop = stop, stop+(neuron.n_inputs+1)

neuron.set_weights( weights[start:stop] )

return self

def update(self, inputs ):

assert len(inputs)==self.n_inputs, "Incorrect amount of inputs."

for layer in self.layers:

outputs = []

for neuron in layer.neurons:

tot = neuron.sum(inputs) + neuron.weights[-1]*BIAS

outputs.append( self.sigmoid(tot) )

inputs = outputs

return outputs

def sigmoid(self, activation,response=1 ):

try:

return 1/(1+math.e**(-activation/response))

except OverflowError:

return float("inf")

def __str__(self):

return '\n'.join([str(i+1)+' '+str(layer) for i,layer in enumerate(self.layers)])