Neural Networks: Understanding Using Visual Basic
Warning: Hazardous Grammar ahead. This tutorial was written years ago when I was just beginning to learn good English writing.Introduction
Neural Networks as a subject was the most difficult one to learn when I started taking interest in AI. Although, Internet was full of NN tutorials but they all seemed cryptic and too much mathematical. Anything which was available was hard to digest for a beginner in this field.I swept through tons of code and tutorials just to understand what the hell, much hyped, NNs were. Then, one fine day, Eureka!
I finally understood them and their practical applications. I immediately decided to write code in Visual Basic(my fav lang). After an hour of writing code and few more for tweaking it, I finally produced something which I had only dreamt of. It was one of the the best moments of my life. Then I finally uploaded .........history is very boring. The real thing: There isn't a single NN tutorial available for Visual Basic Programmers. That is why I decided to write this tutorial.
Introduction to Neural Networks
Neural Network or more appropriately Artificial Neural Network is basically a mathematical model of what goes in our mind(or brain). The brain of all the advanced living creatures consists of neurons, a basic cell, which when interconnected produces what we call Neural Network. The sole purpose of a Neuron is to receive electrical signals, accumulate them and see further if they are strong enough to pass forward.So simple in its basic functionality but the interconnections of these produces beings(me, u and others) capable of writing about them. phew! The real thing lies not in neurons but the complex pattern in which they are interconnected. NNs are just like a game of chess, easy to learn but hard to master. As the moves of chess are simple, yet the succession of moves is what makes the game complex and fun to play. Imagine, a chess game in which you are allowed only one single move. Would that game be fun to play?
In the same way, a single neuron is useless. Well, practically useless. It is the complex connection between them and values attached with them(explained later) which makes brains capable of thinking and having a sense of consciousness(much debated).
Basic Working
As explained earlier, a neuron is basically a cell which accumulates electrical signals with different strengths. What it does more is that it compares the accumulated signal with one predefined value unique to every neuron. This value is called bias. Well, now I think is time to explain with an image. So here is it:
Image Source: http://www.interwet.psu.edu/f41.gif
The network is called multi layered because it consists of more than two layers. The neurons are arranged in a number of layers, generally three. They are input, hidden/middle and output layers. The names signify the function of the layer.
This network is feed-forward, means the values are propagated in one direction only. There are many other topologies in which values can be looped or move in both forward and backward direction. But, this network allows the movement of values only from input layer to output layer. The functions of various layers are explained below:
- Input layer: As it says, this layer takes the inputs(the values you pass) and forwards it to hidden layer. You can just imagine input layer as a group of neurons whose sole task is to pass the numeric inputs to the next level. Well it depends on the implementation but, in my implementation the numeric input has to be in range 0 and 1 (both inclusive). The larger the number greater its strength. E.g. 0.51 is stronger than 0.39 but 0.93412 is stronger still. But, the interpretation of this strength depends upon the implementation and the problem NN is required to solve. E.g. for an OCR you connect every pixel with its respective input neuron and darker the pixel, higher the signal/input strength. In short shades of gray would correspond to input strength. But at last again, remember: Input layer never processes data, it just hands over it.
- Middle layer: This layer is the real thing behind the network. Without
this layer, network would not be capable of solving complex problems. There
can be any number or middle or hidden layers. But, for most of the tasks,
one is sufficient. The number of neurons in this layer is crucial. There is
no formula for calculating the number, just hit and trial works.
This layer takes the input from input layer, does some calculations and forwards to the next layer, in most cases it is the output layer. - Output layer: This layer consists of neurons which output the result to you. This layer takes the value from the previous layer, does calculations and gives the final result. Basically, this layer is just like hidden layer but instead of passing values to the next layer, the values are treated as output.
- Dendrites: No! it is not some creature from X-Files, but a name given to straight lines joining two neurons of consecutive layers, which you can see in the image. They are just a passage(or method) through which values are passed from one layer to the next. There is a value attached with dendrite called weight. The weight associated with a dendrites basically determines the importance of incoming value. A weight with larger value determines that the value from that particular neuron is of higher significance. To achieve this what we do is multiply the incoming value with weight. So no matter how high the value is, if the weight is low the multiplication yields the final low value. I know it is hard to explain and understand, but take a look at code and you'll understand this concept more clearly.
'Don't forget to write option base 1 into the code ' or else this net will not work 'Coded by Paras Chopra 'paras_chopra@fastmail.fm 'http://naramcheez.paraschopra.com 'Please don't forget to give comments, credits and most important your VOTE! Option Base 1 Option Explicit Const e = 2.7183 'Mathematical const, used in sigmod function 'Dendrite connects one neuron to another and allows signal to pass from it Private Type Dendrite Weight As Double 'Weight it has End Type Private Type Neuron 'The main thing Dendrites() As Dendrite 'Array of Dendrites DendriteCount As Long 'Number of dendrites Bias As Double 'The bias Value As Double 'The value to be passed to next layer of neurons Delta As Double 'The delta of neuron (used while learning) End Type Private Type Layer 'Layer containing number of neurons Neurons() As Neuron 'Neurons in the layer NeuronCount As Long 'Number of neurons End Type Private Type NeuralNetwork Layers() As Layer 'Layers in the network LayerCount As Long 'Number of layers LearningRate As Double 'The learning rate of the network End Type Dim Network As NeuralNetwork ' Our main network
The code above is pretty self explanatory. And for the things you haven't understood,
don't worry that belongs to belong to the training section, which I would be explaining
a little later on. The only thing that you need to understand now is bias.
Bias is just another value or parameter associated with a neuron. In my and most
of the implementations this value is added to the accumulated incoming value.
You will better understand this when we see code for running the NN. For now see
the code for creating the net:
'0 = Unsuccessful and 1 = Successful
Function CreateNet(LearningRate As Double, ArrayOfLayers As Variant) As Integer
Dim i, j, k As Integer
Network.LayerCount = UBound(ArrayOfLayers) 'Init number of layers
If Network.LayerCount < 2 Then 'Input and output layers must be there
CreateNet = 0 'Unsuccessful
Exit Function
End If
Network.LearningRate = LearningRate 'The learning rate
ReDim Network.Layers(Network.LayerCount) As Layer 'Redim the layers variable
For i = 1 To UBound(ArrayOfLayers) ' Initialize all layers
DoEvents
Network.Layers(i).NeuronCount = ArrayOfLayers(i)
ReDim Network.Layers(i).Neurons(Network.Layers(i).NeuronCount) As Neuron
For j = 1 To ArrayOfLayers(i) 'Initialize all neurons
DoEvents
'We will not init dendrites for it because output layers doesn't have any
If i = UBound(ArrayOfLayers) Then
Network.Layers(i).Neurons(j).Bias = GetRand 'Set the bias to random value
Network.Layers(i).Neurons(j).DendriteCount = ArrayOfLayers(i - 1)
ReDim Network.Layers(i).Neurons(j).Dendrites(Network.Layers(i).Neurons(j).DendriteCount) As Dendrite
'Redim the dendrite var
For k = 1 To ArrayOfLayers(i - 1)
DoEvents
Network.Layers(i).Neurons(j).Dendrites(k).Weight = GetRand 'Set the weight of each dendrite
Next k
ElseIf i = 1 Then 'Only init dendrites not bias
DoEvents 'Do nothing coz it is input layer
Else
Network.Layers(i).Neurons(j).Bias = GetRand 'Set the bias to random value
Network.Layers(i).Neurons(j).DendriteCount = ArrayOfLayers(i - 1)
ReDim Network.Layers(i).Neurons(j).Dendrites(Network.Layers(i).Neurons(j).DendriteCount) As Dendrite
'Redim the dendrite var
For k = 1 To ArrayOfLayers(i - 1)
DoEvents
Network.Layers(i).Neurons(j).Dendrites(k).Weight = GetRand 'Set the weight of each dendrite
Next k
End If
Next j
Next i
CreateNet = 1
End Function
Basically, what we are doing above is:
- Creating an array of Neurons and dendrites.
- Looping through all the neurons and assigning their respective baises a random value.
- Looping through all the dendrites and assigning their respective weights a random value.
Function Run(ArrayOfInputs As Variant) As Variant 'It returns the output in form of array Dim i, j, k As Integer If UBound(ArrayOfInputs) <> Network.Layers(1).NeuronCount Then Run = 0 Exit Function End If For i = 1 To Network.LayerCount DoEvents For j = 1 To Network.Layers(i).NeuronCount DoEvents If i = 1 Then Network.Layers(i).Neurons(j).Value = ArrayOfInputs(j) 'Set the value of input layer Else Network.Layers(i).Neurons(j).Value = 0 'First set the value to zero For k = 1 To Network.Layers(i - 1).NeuronCount DoEvents 'Calculating the value Network.Layers(i).Neurons(j).Value = Network.Layers(i).Neurons(j).Value + (contd on next line) + Network.Layers(i - 1).Neurons(k).Value * Network.Layers(i).Neurons(j).Dendrites(k).Weight Next k Network.Layers(i).Neurons(j).Value = Activation(Network.Layers(i).Neurons(j).Value + (contd on next line) + Network.Layers(i).Neurons(j).Bias) 'Calculating the real value of neuron End If Next j Next i ReDim OutputResult(Network.Layers(Network.LayerCount).NeuronCount) As Double For i = 1 To (Network.Layers(Network.LayerCount).NeuronCount) DoEvents OutputResult(i) = (Network.Layers(Network.LayerCount).Neurons(i).Value) 'The array of output result Next i Run = OutputResult End Function
What we have done above may be summed up as:
- Assigning all the input neurons the values provided in the 'ArrayOfInputs' array. For every other type of neuron, the value assigned is 0 because we have to calculate that value.
- Now for every hidden layer (also for output layer), the value is sum of values of previous layers multiplied with the weight of dendrites.
- To the resulting value, add up the bias value.
- Pass the resulting value through Activation function.
- Give back the values of output neurons as final results.
Private Function Activation(Value As Double) 'To crunch a number between 0 and 1 Activation = (1 / (1 + Exp(Value * -1))) End Function
I will suppose you wouldn't like to get into the mathematical details(hurray!). In case you like,
you can easily google it and find out.
Training
Training is the most important part of a neural network and the one consisting of the most mathematics. We'll be using Backpropagation method for training the NN. I would be explaining only the basic idea how it is done and not all the details. And you need not worry, just apply the formulae written in the code and you can get the training up and ready. Always remember, it is not necessary to know the details of any system in order to get something practical and working out of it. The best example illustrating this principle is Charles Darwin(what?). Yes, at the time when he wrote 'On the Origin of Species', DNA was not known. So, he propounded the evolution without even knowing the method of how it is done i.e. how traits are passed on from parents to offspring.Back to NN training, first lets see the code:
Function SupervisedTrain(inputdata As Variant, outputdata As Variant) As Integer '0=unsuccessful and 1 = successful Dim i, j, k As Integer If UBound(inputdata) <> Network.Layers(1).NeuronCount Then 'Check if correct amount of input is given SupervisedTrain = 0 Exit Function End If If UBound(outputdata) <> Network.Layers(Network.LayerCount).NeuronCount Then 'Check if correct amount of output is given SupervisedTrain = 0 Exit Function End If Call Run(inputdata) 'Calculate values of all neurons and set the input 'Calculate delta's For i = 1 To Network.Layers(Network.LayerCount).NeuronCount DoEvents 'Deltas of Output layer Network.Layers(Network.LayerCount).Neurons(i).Delta = Network.Layers(Network.LayerCount).Neurons(i).Value * (contd) * (1 - Network.Layers(Network.LayerCount).Neurons(i).Value) * (contd) * (outputdata(i) - Network.Layers(Network.LayerCount).Neurons(i).Value) For j = Network.LayerCount - 1 To 2 Step -1 DoEvents For k = 1 To Network.Layers(j).NeuronCount DoEvents Network.Layers(j).Neurons(k).Delta = Network.Layers(j).Neurons(k).Value * (contd) * (1 - Network.Layers(j).Neurons(k).Value) * Network.Layers(j + 1).Neurons(i).Dendrites(k).Weight * (contd) * Network.Layers(j + 1).Neurons(i).Delta 'Deltas of Hidden Layers Next k Next j Next i For i = Network.LayerCount To 2 Step -1 DoEvents For j = 1 To Network.Layers(i).NeuronCount DoEvents Network.Layers(i).Neurons(j).Bias = Network.Layers(i).Neurons(j).Bias + (contd) + (Network.LearningRate * 1 * Network.Layers(i).Neurons(j).Delta) 'Calculate new bias For k = 1 To Network.Layers(i).Neurons(j).DendriteCount DoEvents 'Calculate new weights Network.Layers(i).Neurons(j).Dendrites(k).Weight = Network.Layers(i).Neurons(j).Dendrites(k).Weight + (contd) (Network.LearningRate * Network.Layers(i - 1).Neurons(k).Value * Network.Layers(i).Neurons(j).Delta) Next k Next j Next i SupervisedTrain = 1 End Function
So lets see what we have here in arguments, inputdata a sample input array and
outputdata, its corresponding output which we require the NN to produce.
- Run the NN using inputdata as inputs. Now, we have all the values of individual neurons.
- Loop through last layer to first layer.
- Calculate the difference between output values desired and output values produced.
- Using that difference, adjust the values of bias and weights accordingly.
Yes, I know I haven't explained the training part in details(well practically zero). You see, I had written this code 1-2 years back and was not actively involved with NNs. During that period I have forgotten all the mathematics behind Backpropagation. But anyway, If you want to delve into thedetails, Google is to your rescue.
Practical Applications
There are numerous applications of NNs limited only by your imagination. Innovation is key to success, so dude use NNs to create something which will revolutionize the world!For the sake of writing :) a few applications are given below:
- Optical Character Recognization
- Stock market prediction
- Creating new art forms
- Modeling human behavior
- Loan risk analysis
- Classification of patterns, Image analysis, music generation, etc, etc.
Further reading
There is more theory in this world about NNs than you can read in your lifetime. There are Hopfield networks, Recurrent networks, NeuroGenetics, etc. They are for people who really want to do serious research in this field. But in case, you want to read further, you can. Internet is a great free information highway. You can get anything you want on the internet if you know how to search effectively. Even if you don't, train a NN to fetch NN tutorials for you :-)Thanks for reading this basic tutorial, Hope you enjoyed the ride.
Code
Click here to download the accompanying code.
Комментариев нет:
Отправить комментарий