Jump to content
  • Advertisement
Sign in to follow this  
nir143

basic neural network

This topic is 2637 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

[font=Arial,]

after reading some articles about neural network(back-propagation) i try to write a simple neural network by myself.

ive decided XOR neural-network, my problem is when i am trying to train the network, if i use only one example to train the network,lets say 1,1,0(as input1,input2,targetOutput). after 500 trains +- the network answer 0.05. but if im trying more then one example (lets say 2 different or all the 4 possibilities) the network aims to 0.5 as output :( i searched in google for my mistakes with no results :S ill try to give as much details as i can to help find what wrong:

-ive tried networks with 2,2,1 and 2,4,1 (inputlayer,hiddenlayer,outputlayer).

-the output for every neural defined by:

[color=#00008B]

double[color=#000000]

input [color=#000000]

=[color=#000000]

[color=#800000]

0.0[color=#000000]

;[color=#000000]


[color=#00008B]

for[color=#000000]

[color=#000000]

([color=#00008B]

int[color=#000000]

n [color=#000000]

=[color=#000000]

[color=#800000]

0[color=#000000]

;[color=#000000]

n [color=#000000]

<[color=#000000]

layers[color=#000000]

[[color=#000000]

i[color=#000000]

].[color=#2B91AF]

Count[color=#000000]

;[color=#000000]

n[color=#000000]

++)[color=#000000]


input [color=#000000]

+=[color=#000000]

layers[color=#000000]

[[color=#000000]

i[color=#000000]

][[color=#000000]

n[color=#000000]

].[color=#2B91AF]

Output[color=#000000]

[color=#000000]

*[color=#000000]

weights[color=#000000]

[[color=#000000]

n[color=#000000]

];[color=#000000]


while 'i' is the current layer and weight are all the weights from the previous layer.

-the last layer(output layer) error is defined by:

[color=#000000]

value[color=#000000]

*([color=#800000]

1[color=#000000]

-[color=#000000]

value[color=#000000]

)*([color=#000000]

targetvalue[color=#000000]

-[color=#000000]

value[color=#000000]

);[color=#000000]


while 'value' is the neural output and 'targetvalue' is the target output for the current neural.

-the error for the others neurals define by:

[color=#00008B]

foreach[color=#000000]

neural [color=#00008B]

in[color=#000000]

the nextlayer
sum[color=#000000]

+=[color=#000000]

neural[color=#000000]

.[color=#000000]

value[color=#000000]

*[color=#000000]

currentneural[color=#000000]

.[color=#000000]

weights[color=#000000]

[[color=#000000]

neural[color=#000000]

];[color=#000000]


-all the weights in the network are adapt by this formula(the weight from neural -> neural 2)

[color=#000000]

weight[color=#000000]

+=[color=#2B91AF]

LearnRate[color=#000000]

*[color=#000000]

neural[color=#000000]

.[color=#000000]

myvalue[color=#000000]

*[color=#000000]

neural2[color=#000000]

.[color=#000000]

error[color=#000000]

;[color=#000000]


while LearnRate is the nework learning rate(defined 0.25 at my network). -the biasweight for each neural is defined by:

[color=#000000]

bias[color=#000000]

+=[color=#2B91AF]

LearnRate[color=#000000]

*[color=#000000]

neural[color=#000000]

.[color=#000000]

myerror[color=#000000]

*[color=#000000]

neural[color=#000000]

.[color=#2B91AF]

Bias[color=#000000]

;[color=#000000]


bias is const value=1.

that pretty much all i can detail, as i said the output aim to be 0.5 with different training examples :(

thank you very very much for your help ^_^.

[/font]

Share this post


Link to post
Share on other sites
Advertisement
Debugging your problem is hard to do (especially without access to the code itself), and I am not willing to do it for you. But I can tell you how I would go about doing it.

Do you understand why the update rules are what they are? They are supposed to be taking one little step in gradient descent. The specific formulas you use are ways of computing the derivative of the error function (typically the square of the difference between the output and the desired output) with respect to each weight. You can try to change the weights a tiny bit, measure the change in the error function and see if that matches your computation.

Share this post


Link to post
Share on other sites
ive try some tests,as i said if im using only one training example everything works fine(atleast aim to 0.04 mistake after 600 trains).
i dont understand what you advice me to do,
change the weights little bit?:S(that created randomally),i think i misunderstood you,
the 4-5 formulas i worte here are ok fine right?

tyvm for your help:)

Share this post


Link to post
Share on other sites

ive try some tests,as i said if im using only one training example everything works fine(atleast aim to 0.04 mistake after 600 trains).
i dont understand what you advice me to do,
change the weights little bit?:S(that created randomally),i think i misunderstood you,
the 4-5 formulas i worte here are ok fine right?

tyvm for your help:)


This is what I mean by changing the weights a little bit. After you evaluate the network, pick a weight, add 0.001 to it and evaluate it again. If you consider the error function as a function of the value of the weight, you can now compute (f(w+0.001)-f(w))/0.001 , which should be close to the derivative. I don't know if the formulas you posted are correct, but the way I would figure out if they are correct is by trying to understand them as a gradient-descent step, which involves computing that derivative.

Share this post


Link to post
Share on other sites
im not strong in this kind of math:S
but ive followed few guides about the formulas and about the theory of course and as i checked the derivative function is correct,
i try to give as much details as i can to make the formulas\code understandable:)
I went through the code few times and still dont understand where the problem:S

Share this post


Link to post
Share on other sites
ive try also to use genetic algorithm instead of back-propagation and they output aim to be 0.5:S
that means the problem should be in the feed-forward formulas?

Share this post


Link to post
Share on other sites
Try using 0.1 instead of 0, and 0.9 instead of 1, for your inputs and outputs. What is your activation function?Also, try removing the bias nodes for now to see if that makes a difference.

Share this post


Link to post
Share on other sites
im using logistic activation function-sigmoid curve.

ive try what you said with 0.9\0.1 instead of 1\0 and removing the bias but the output still aim to be 0.5:S

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!