Hi,
As of late I have been learning much about machine learning, basically trying to rehearse how computers learn and possibly use this method to gain an "achievement" with the lottery!!? lol - Thus far I wanted to present what I currently have figured out in terms of coding using python to predict a vector of 5 dimensions or a pick 5 game. What I have, is very simple - however complex if you're not familiar with programming and machine learning. I thought to share my code and see if there's any one here whom would like to contribute in this project and make it complete.
So far the following functions have been called;
One and only one input training sample for X (Multiple batches are needed to predict better), likewise, only one sample for y.
There are 5 inputs, 5 outputs and 4 hidden layers.
5 random weights per neuron connection wljk.
The sigmoid function is assigned to each activation layer.
And sigmoid prime which depicts the amount of error for the output y-hat.
import numpy as np
#Sample training input
X = np.array(([3,5,20,23,26]), dtype=float)
y = np.array(([3,20,25,28,30]), dtype=float)
X = X/np.amax(X, axis=0)
y = y/36 #Max number size is 36
class Neural_Network(object):
def __init__(self):
#define Hyperparameters
self.inputLayerSize = 5
self.outputLayerSize = 5
self.hiddenLayerSize_1 = 7
self.hiddenLayerSize_2 = 7
self.hiddenLayerSize_3 = 7
self.hiddenLayerSize_4 = 7
#weights (parameters)
self.W1 = np.random.randn(self.inputLayerSize, self.hiddenLayerSize_1)
self.W2 = np.random.randn(self.hiddenLayerSize_1, self.hiddenLayerSize_2)
self.W3 = np.random.randn(self.hiddenLayerSize_2, self.hiddenLayerSize_3)
self.W4 = np.random.randn(self.hiddenLayerSize_3, self.hiddenLayerSize_4)
self.W5 = np.random.randn(self.hiddenLayerSize_4, self.outputLayerSize)
def forward(self, X):
#propagate inputs through network
self.z2 = np.dot(X, self.W1)
self.a2 = self.sigmoid(self.z2)
self.z3 = np.dot(self.a2, self.W2)
self.a3 = self.sigmoid(self.z3)
self.z4 = np.dot(self.a3, self.W3)
self.a4 = self.sigmoid(self.z4)
self.z5 = np.dot(self.a4, self.W5)
yHat = self.sigmoid(self.z5)
return yHat
def sigmoid(z):
#Apply sigmoid activation function to scalar, vector or matrix
return 1/(1+np.exp(-z))
def sigmoidPrime(z):
#Derivative of sigmoid function
return np.exp(-z)/((1+np.exp(-z))**2)
NN = Neural_Network()
yHat = NN.forward(X)
print yHat
print y
Still, what's missing is adding;
Backpropagation,
Computing the cost function with respect to each derivatives weight,
Numerical gradient checking,
training the network,
Testing and overfitting.
Any one interested in adding information is welcome. This is a software to get a visual sense in how Neural Networks can perform predictions in the background.
