[Solved] Strange Output of Neural Net

Started by
1 comment, last by brwarner 18 years, 1 month ago
I am using C# and created a Neural Net like the one on AI-Junkie using the Sigma function. I input 4 variables (current x,y + x,y of target) and it spits out 2 outputs. In the sigmoid function I set the value that determines the limit to 1.Then the outputs are added to x and y values of an object. Yet the objects values sometimes hit infinity, how is this possible?? Here is the source code in C# for the NeuralNet. I may have done somthing wrong, this is my first time using a NeuralNet and this was a test.

using System;
using System.Collections.Generic;
using System.Text;

namespace NeuralNet_Test
{
    public struct Neuron
    {
        public List<double> weights;
        public int NumberOfInputs;

        public Neuron(int numInputs)
        {
            NumberOfInputs = numInputs;
            weights = new List<double>();
            for (int i = 0; i < NumberOfInputs; i++)
                weights.Add(RandomDouble(10));
        }
        private double RandomDouble(int max)
        {
            Random r = new Random();
            return ((r.NextDouble() - 0.5) * 2 * max);
        }
    }

    public struct NeuralLayer
    {
        public List<Neuron> Neurons;
        public int NumberOfNeurons;
        public int InputsPerNeuron;

        public NeuralLayer(int numOfNeurons,int numInputsPerNeuron)
        {
            NumberOfNeurons = numOfNeurons;
            InputsPerNeuron = numInputsPerNeuron;
            Neurons = new List<Neuron>();
            for (int i = 0; i < NumberOfNeurons; i++)
                Neurons.Add(new Neuron(InputsPerNeuron));
        }
    }

    public class NeuralNet
    {
        private List<NeuralLayer> layers;
        private int NeuronsPerHiddenLayer;
        private int HiddenLayers;
        private int NumberOfInputs,
            NumberOfOutputs;


        public NeuralNet(int numInputs, int numOutputs, int numHiddenLayers, int numNeuronsPerLayer)
        {
            //Assign Instance Data
            NumberOfInputs = numInputs;
            NumberOfOutputs = numOutputs;
            HiddenLayers = numHiddenLayers;
            NeuronsPerHiddenLayer = numNeuronsPerLayer;
            layers = new List<NeuralLayer>();
            //Create the Neural Net
            CreateNet();
        }

        private void CreateNet()
        {
            //Create the NeuralNet
            //Usually called by the constructor

            //Create the first layer if any
            if (HiddenLayers > 0)
                layers.Add(new NeuralLayer(NeuronsPerHiddenLayer, NumberOfInputs));
            //Create each additional hidden layer
            for (int i = 0; i < HiddenLayers - 1; i++)
            {
                layers.Add(new NeuralLayer(NeuronsPerHiddenLayer,NeuronsPerHiddenLayer));
            }
            //Create Output layer
            layers.Add(new NeuralLayer(NumberOfOutputs,NeuronsPerHiddenLayer));
            //That's all folks
        }

        public List<double> UpdateNet(List<double> input)
        {
            //Number of inputs read this time around
            int weight = 0;
            //Net input this round
            double netInput = 0;
            //Output
            List<double> Output = new List<double>();
            //For each Layer
            for (int i = 0; i < HiddenLayers; i++)
            {
                //If this is not the first layer the input is last
                //layers output
                if (i > 0)
                    input = Output;

                //Clear Output from previous layer
                Output.Clear();

                //For each Neuron
                for (int j = 0; j < layers.NumberOfNeurons; j++)
                {
                    //Reset counter
                    weight = 0;

                    //For each weight
                    for (int k = 0; k < layers.Neurons[j].NumberOfInputs; k++)
                    {
                        netInput += layers.Neurons[j].weights[k] * 
                            input[weight];
                    }

                    //Finally add the output to the table
                    Output.Add(Activation(netInput,1));

                    //Clear variables
                    netInput = 0;
                    weight = 0;
                }
            }
            return Output;
        }

        private double Activation(double output, double response)
        {
            return (1 / 1 + Math.Exp(-output / response));
        }
    }
}


[Edited by - brwarner on March 23, 2006 3:54:58 PM]
Advertisement
Sorry, don't know C# op precedence but if it's similar to C,

Quote:Original post by brwarner

return (1 / 1 + Math.Exp(-output / response));



should be:

return ( 1 / ( 1 + Math.Exp( -output / response ) ) );

See: http://mathworld.wolfram.com/SigmoidFunction.html
Thanks :)!! Also I found that I had to add one to i because it was missing the Output layer.

This topic is closed to new replies.

Advertisement