SKEDSOFT
Introduction:-A technical neural network consists of simple processing units, the neurons, and directed, weighted connections between those neurons.The strength of a connection (or the connecting weight) be reactions of the neurons to the input values depend on this activation state. Activation state indicates the extent of a neuron’s activation and is often shortly referred to as activation.
Definition 1 (Activation state / activationin general). Let j be a neuron. The activation state aj, in short activation, is explicitly assigned to j, indicates the extent of the neuron’s activity and results from the activation function.
Components of neural networks:-A technical neural network consists of simpleprocessing units, the neurons, anddirected, weighted connections between those neurons. Here, the strength of a connection (or the connecting weight) be- tween two neurons i and j is referred to as wi,j.
Definition 2 (Neural network). A neural network is a sorted triple (N, V,w) with two sets N, V and a function w, where N is the set of neurons and V a set {(i, j)|i, j 2 N} whose elements are called connections between neuron i and neuron j. The function w :V ! R definesn. network= neurons weighted connectionthe weights, where w((i, j)), the weight of the connection between neuron i and neuron j, is shortened to wi,j . Depending on wi,j the point of view it is either undefined or 0 for connections that do not exist in the network.
Note: In some of the cited literature i and j could be interchanged in wi,j . Here, a consistent standard does not exist. But in this text I try to usethe notation I found more frequently and in the more significant citations.
So the weights can be implemented in a square weight matrix W or, optionally, in a weight vector W with the row number of the matrix indicating where the connection begins, and the column number of the matrix indicating, which neuron is the target. Indeed, in this case the numeric 0 marks a non-existing connection. This matrix representation is also called Hinton diagram.
Connections carry information that is processed by neurons:-Data are transferred between neurons via connections with the connecting weight being either excitatory or inhibitory.
The propagation function converts vector inputs to scalar network inputs:-Looking at a neuron j, we will usually find a lot of neurons with a connection to j, i.e.which transfer their output to j.
Note that, here again, in some of the cited literature axes and rows could be interchanged. The published literature is not consistent here, as well.
Data processing of a neuron. Theactivation function of a neuron implies thethreshold value.For a neuron j the propagation functionreceives the outputs oi1, . . . , o in of other neurons i1, i2, . . . , n (which are connectedto j), and transforms them in con- manages side ratio n of the connecting weights wi,j inputs into the network input net j that can be furtherprocessed by the activation function.Thus, the network input is the result ofthe propagation function.
Definition 3 (Propagation function and network input). Let I = {i1, i2. . . in} be the set of neurons, such that 8z 2 {1, . . . , n} : 9wiz,j . Then the network input of j, called netj, is calculated by the propagation function fprop as follows:
netj= fprop(oi1 , . . . , oin,wi1,j , . . . ,win,j)
Here the weighted sum is very popular: The multiplication of the output of each neuron i by wi,j, and the summation of result
Definition 4(Activation state / activationin general). Let j be a neuron. Theactivation state aj , in short activation, isexplicitly assigned to j, indicates the extentof the neuron’s activity and resultsfrom the activation function.
It is possible to get and get activationstates of neurons by using the methodsgetActivation or setActivation inthe class NeuralNetwork.
Neurons get activated if the network input exceeds their threshold value near the threshold value:-The activationfunction of a neuron reacts particularlysensitive. From the biological point ofview the threshold value represents thethreshold at which a neuron starts firing.The threshold value is also mostly highestpoint ofsensationincluded in the definition of the activationfunction, but generally the definition is thefollowing:
Definition 5(Threshold value in general).Let j be a neuron. The thresholdvalue _j is uniquely assigned to j and_I marks the position of the maximum gradientvalue of the activation function.3.2.5 The activation functiondetermines he activation of aneuron dependent on networkinput and threshold valueAt a certain time – as we have alreadylearned – the activation aj of a neuron jdepends on the previous3 activation stateof the neuron and the external input.
Definition 6(Activation function andActivation). Let j be a neuron. The ac- calculates activation function is defined as activationaj(t) = fact(netj(t), aj(t − ),_j). (3.3)It transforms the network input netj , as well as the previous activation state Jfactaj(t − 1) into a new activation state j(t),with the threshold value _ playing an importantrole, as already mentioned.Unlike the other variables within the neuralnetwork (particularly unlike the onesdefined so far) the activation function isoften defined globally for all neurons orat least for a set of neurons and only thethreshold values are different for each neuron.We should also keep in mind thatthe threshold values can be changed, forexample by a learning procedure. So itcan in particular become necessary to relatethe threshold value to the time and towrite, for instance _j as _j(t) (but for reasonsof clarity, I omitted this here). Theactivation function is also called transferfunction.The previous activation is not always relevant forthe current – we will see examples for both variants.
