Implementation of Deep Neural Network able to recognize digits from MNIST database
numpy 1.17.4
save_network(self, directory) - saves network in files, which will be in directory
- directory - path to directory, which will contain saved files
load_network(self, directory) - loads network to model, from files located in directory
- directory - path to directory, which will contain saved files
add_input(self, size) - Adds input layer to model. It must be used before any add_* function. Size indicate to size of input data. Return True if adding layer has been completed with success, otherwise False.
- size - size of input data
add_fully_connected(self, neurons, activation='linear', left_range=-0.1, right_range=0.1) - Adds fully connected layer. Return True if adding layer has been completed with success, otherwise False.
- neurons - number of neurons in layer
- activation - name of activation function, which will be used in added layer
- left_range - left range used to generate weights values
- left_range - right range used to generate weights values
predict(self, input_data) - If passed input_data is valid to network structure it returns network answer, otherwise None.
- input_data - data used as input for network, which answer is returned
fit(self, input_data, expected_data, iterations=100, alpha=0.01, drop_percent=0.5, test_input=None, test_expected=None) - Return True if training has been completed with success, otherwise False.
- input_data - array of data, where input_data.shape = (number of input data, shape of single data)
- expected_data - the same structure as in input_data, but here we pass expected data
- iterations - number of iterations during training
- alpha - learning rate (0.0-1.0) indicates how fast network should learn
- drop_percent - drop (0.0-1.0) indicates how much of neurons should be 'dropped', it prevents network from overfitting.
- test_input - data used for debugging purposes, probably you should omit it
- test_expected - same issue like in test_input
To pass vectors and matrices use numpy structures
Neural Network consists of layers where we can distinguish three types of layers - input, hidden, output. Obviously there is only one input layer and one output, but we can use as many hidden layers as we want. Every layer contains some number of neurons and activation function (except input layer).
Input layer indicates how input data looks like. So if we have image of digit 28x28px, then we will transform it into vector with 784 elements and this number is a number of "neurons" in input layer.
As it was said before, we can use any number of hidden layers, and we put them between input and output layer. It is your job to figure out how many hidden layers and number of neurons use. There is additional thing to use - activation function, which says how data are manipulated. Few of them are ready to use in this implementation:
- linear
- relu
- tanh
- softmax (only for output)
The last one is output layer, which is similar to hidden. Number of neurons here is number of possible answers for our input. In case of digits we want to guess 0-9 digits, then our output will consists of 10 neurons, each corresponding to one digit. It is recommended to use linear or softmax function here.
Here we have built structure of network with where we use 2 hidden layers with relu functions and 16/32 neurons in hidden layers. So now we can start training and fit function is in charge of it where we are passing parameters as learning rate, drop percent or iterations, which helps us control training flow. But main issue here is passing input training data and expected data. After training, we can use our network to recognize digits from image.