9.3 KiB
Install required dependencies for matplotlib GUI frontend and all pip other packages for this project
sudo apt install python3-tk
python3.9 -m pip install -r requirements.txt
Neural network implementation using Python CLI to dynamically generate a resizable network
and then run a given number of learning cycles on the provided data set.
As an example, the IRIS dataset is used to classify flower types using petal measurements.
Input layer perceptron count can be adjusted with INPUTS positional parameter.
Hidden layer perceptron count can be adjusted with PERCEPTRONS positional parameter.
Output layer perceptron count can be adjusted with OUTPUTS positional parameter.
Hidden layers can be added or removed using--hidden-layers option setting.
Node bias can be initialized randomly or with provided data.
Perceptron edge weight bias can be initialized randomly or with provided data.
Threshold for perceptron fire can be initialized randomly or with provided data.
Setup instructions and output of neural-network.py -h-
python3.9 neural-network.py -h
usage: neural-network.py [-h] [--hidden-layers [HIDDEN_LAYERS]] [--cycles [CYCLES]] [--learn-rate [LEARNING_RATE]]
[--bias [INITIAL_BIAS]] [--weight [INITIAL_EDGE_WEIGHTS]] [--error-threshold [ERROR_THRESHOLD]]
[--fire-threshold [FIRE_THRESHOLD]] [--spacing [LAYER_SPACING]] [--horizontal] [--silent] [--verbose]
[--file [file_path]]
[INPUTS] [PERCEPTRONS] [OUTPUTS]
Neural network implementation
positional arguments:
INPUTS Number of inputs for the neural network
(default: '3')
PERCEPTRONS Number of perceptrons in each hidden layer
(default: '8')
OUTPUTS Number of outputs for the neural network
(default: '3')
optional arguments:
-h, --help show this help message and exit
--hidden-layers [HIDDEN_LAYERS], -l [HIDDEN_LAYERS]
Number of hidden layers
(default: '1')
--cycles [CYCLES], -c [CYCLES]
Number of cycles to run through the network
(default: '3')
--learn-rate [LEARNING_RATE]
Learning rate to use for the network.
Must be within range of 0.0 < rate <= 1.0
(default: '0.25')
--bias [INITIAL_BIAS], -b [INITIAL_BIAS]
The initial bias to use for perceptrons within the network.
Must be within range of -1.0 <= bias <= 1.0
If value is unset, bias will be initialized randomly
--weight [INITIAL_EDGE_WEIGHTS], -w [INITIAL_EDGE_WEIGHTS]
The initial edge weight to use for node connections in the network
If value is unset, edge weights will be initialized randomly
--error-threshold [ERROR_THRESHOLD], --error [ERROR_THRESHOLD]
The acceptable error threshold to use for training the network.
(default: '0.5')
--fire-threshold [FIRE_THRESHOLD], --fire [FIRE_THRESHOLD]
The fire threshold for perceptrons in the network.
If a perceptron's cumulative inputs reach this value, the perceptron fires
(default: '0.25')
--spacing [LAYER_SPACING]
Distance between origin of network layers within visualization
(default: '2.0')
--horizontal, --flip The network visualization will flow left-to-right
(default: 'False')
--silent Do not show the network visualization, only print output to console
(default: 'False')
--verbose, -v When this flag is set, error rate and change in weight will be output for each calculation
(default: 'False')
--file [file_path], -f [file_path]
Optionally provide a json file to configure any option available through the cli
json keys match --long version of each option, where --long-split option key is "long_split" in json
Input and output layers are sized by the length of a single input sequence and the number of possible label classifications. If the length of an input sequence does not match the number of input nodes requested, a warning will show. If the length of possible label classifications does not match the number of output nodes requested, a warning will show. In both cases, the program corrects the node count to match the input data / labels, and not the requested node count.
The total number of output labels provided must match the total number of input sequences.
Running NN program uses IRIS data set by default. Warnings will be shown if input and output node count is changed without providing new input.
python3.9 neural-network.py --file input.json --silent
Warning: Input sequences each contain 3 entries but 5 input nodes were requested.
Using 3 input nodes instead of 5
Warning: Output labels contain 3 possible classifications but 8 output were nodes requested.
Using 3 output nodes instead of 8
Creating a single layer neural network:
Total input nodes: 3
Number of perceptrons in each hidden layer: 8
Total output nodes: 3
Number of hidden layers: 3
Fire threshold: 0.25
Error threshold: 0.5
Learn rate: 0.25
Initial bias: Random
Initial edge weights: Random
Network visualization settings:
Graph visualization is enabled: False
Graph visualization is horizontal: True
Graph visualization is vertical: False
Graph visualization layer spacing: 2.0
Test data input count: 150
inputs layer: [0, 1, 2]
hidden layer: [[3, 4, 5, 6, 7, 8, 9, 10], [11, 12, 13, 14, 15, 16, 17, 18], [19, 20, 21, 22, 23, 24, 25, 26]]
outputs layer: [27, 28, 29]
[Cycle 1] Accuracy: 92.6667% [139 / 11]
[Cycle 2] Accuracy: 95.3333% [286 / 14]
[Cycle 3] Accuracy: 96.2222% [433 / 17]
[Cycle 4] Accuracy: 96.6667% [580 / 20]
[Cycle 5] Accuracy: 96.9333% [727 / 23]
[Cycle 6] Accuracy: 97.1111% [874 / 26]
[Cycle 7] Accuracy: 97.2381% [1021 / 29]
[Cycle 8] Accuracy: 97.3333% [1168 / 32]
[Cycle 9] Accuracy: 97.4074% [1315 / 35]
[Cycle 10] Accuracy: 97.4667% [1462 / 38]
Correct: 1462 Wrong: 38 Total: 1500
Cycle 1 accuracy: 92.6667% Cycle 10 accuracy: 97.4667%
4.8% change over 10 cycles 0.48% average change per cycle
Running NN program with garbage data in input-test.json to test resizing of input / output layers.
A single input sequence is [0, 1, 0, 1, 1, 1] which is length of 6, so 6 input nodes are created.
Within the output labels, there are 8 unique labels in the set, so 8 output nodes are created.
The length a single label must match the number of output nodes.
For 8 output nodes, the labels [1, 0, 0, 0, 0, 0, 0, 0] and [0, 1, 0, 0, 0, 0, 0, 0] are valid.
python3.9 neural-network.py --file ./input-test.json --silent
Warning: Output labels contain 8 possible classifications but 10 output were nodes requested.
Using 8 output nodes instead of 10
Creating a single layer neural network:
Total input nodes: 6
Number of perceptrons in each hidden layer: 8
Total output nodes: 8
Number of hidden layers: 3
Fire threshold: 0.25
Error threshold: 0.5
Learn rate: 0.25
Initial bias: Random
Initial edge weights: Random
Network visualization settings:
Graph visualization is enabled: False
Graph visualization is horizontal: True
Graph visualization is vertical: False
Graph visualization layer spacing: 2.0
Test data input count: 14
inputs layer: [0, 1, 2, 3, 4, 5]
hidden layer: [[6, 7, 8, 9, 10, 11, 12, 13], [14, 15, 16, 17, 18, 19, 20, 21], [22, 23, 24, 25, 26, 27, 28, 29]]
outputs layer: [30, 31, 32, 33, 34, 35, 36, 37]
[Cycle 1] Accuracy: 35.7143% [5 / 9]
[Cycle 2] Accuracy: 39.2857% [11 / 17]
[Cycle 3] Accuracy: 40.4762% [17 / 25]
[Cycle 4] Accuracy: 41.0714% [23 / 33]
[Cycle 5] Accuracy: 41.4286% [29 / 41]
[Cycle 6] Accuracy: 41.6667% [35 / 49]
[Cycle 7] Accuracy: 41.8367% [41 / 57]
[Cycle 8] Accuracy: 41.9643% [47 / 65]
[Cycle 9] Accuracy: 42.0635% [53 / 73]
[Cycle 10] Accuracy: 42.1429% [59 / 81]
Correct: 59 Wrong: 81 Total: 140
Cycle 1 accuracy: 35.7143% Cycle 10 accuracy: 42.1429%
6.4286% change over 10 cycles 0.6429% average change per cycle
By default, the following network and visualization will be generated
python3.9 neural-network.py
# Output removed for GUI example
