• Advertisement

Archived

This topic is now archived and is closed to further replies.

Neural Problems

This topic is 6056 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi. I''ve been looking at different places on the internet for information about neural nets. I programmed one that is supposed to be able to learn simple math. The problem I am having is that it only answers with 1, 2, or 82, and one time 4. Sometimes it can memorize a math problem(As long as it''s answer is 0 or 1), but when I try to teach it something else it gets the previous one wrong, and eventually after several wrong answers it will start giving 82 for everything. Do I just not know what I''m doing? Do I need to lower/raise the amount the weights change? or is the whole concept of a neural net learning simple math just beyond it''s capabilities? I can send my source code to anyone who is willing to help me. Thanks in advance! EAX

Share this post


Link to post
Share on other sites
Advertisement
Guest Anonymous Poster
What''s your e-mail address?

Share this post


Link to post
Share on other sites
My e-mail address is aaronsj@hotmail.com

EAX

Share this post


Link to post
Share on other sites
When you say that when you teach it something, it gets the previous one wrong it makes me think that when you train the network to learn something new you''re not including the training it got before. When you introduce a new math problem, you have train the network by looping through all the previous math problems in addition to the new problem.

Also, a neural net can only output a value between -1 and 1, if you want the program to output an answer you have to scale the number.
ex. output: 0.82
output * 100 = 82.

You might also want to make sure your using a network capable of memorizing the math problems, and that the network is structured properly. www.generation5.org has some very good information on neural networks.

Digital Radiation

Share this post


Link to post
Share on other sites
I''ve looked over most of the things people have suggested I look over, and most of it has things I don''t understand. I also keep messing with my neural net to try to make it better, and sometimes I do get a little improvement, but still it forgets things whenever it learns something else. I''m beginning to think there is no hope for my neural net.

Thanks to everyone who has tried to help.

EAX

Share this post


Link to post
Share on other sites
Perhaps you could tell us what you''ve actually implemented and (in case it''s different) what you intended to implement. i.e. what was your learning algorithm? How did you present the training data? How did you test the network to see if it had learnt? What topology of network did you use?
I''m expecting a feedfoward, trained by backprop answer here, but until we know, none of us can help you.
Post some code and what your intentions were.

Mike

Share this post


Link to post
Share on other sites
Ok.
I start with 24 nodes, for the numbers 0-9... +, -, *, /... and another 0-9, then all of those are connected to 81 other nodes which are connected to 81 others which are connected to the answers (81 because 9*9=81 is the highest problem).

float FirstLayer[24][82];
float SecondLayer[82][82];

for (i=0;i<24;i++)
{
for (j=0;j<82;j++)
{
FirstLayer[j] = .5;
}
}
for (i=0;i<81;i++)
{
for (j=0;j<82;j++)
{
SecondLayer[i][j] = .5;
}
}

The code above is supposed to set all the weights to .5

int InitFirst(float FirstLayer[24][82], float SecondLayer[82][82], char MProb[])
{
float Neuron[82];
int i;
int sign;
if (MProb[1] == ''+'')
sign = 10;
if (MProb[1] == ''-'')
sign = 11;
if (MProb[1] == ''*'')
sign = 12;
if (MProb[1] == ''/'')
sign = 13;
for (i=0;i<82;i++)
{
Neuron[i] = 1 * FirstLayer[(MProb[0] - 48)][i];
Neuron[i] += 1 * FirstLayer[sign][i];
Neuron[i] += 1 * FirstLayer[(MProb[2] - 35)][i];
}
return(InitSecond(SecondLayer, Neuron));
}

int InitSecond(float SecondLayer[82][82], float Fire[])
{
float Neuron[82];
int i, j;

for(i=0;i<82;i++)
{
Neuron[i] = 0;
}

for(i=0;i<82;i++)
{
if (Fire[i] >= 1)
{
for(j=0;j<82;j++)
{
Neuron[j] += 1 * SecondLayer[i][j];
}
}
}
return(FinalAnswer(Neuron));
}

int FinalAnswer(float Neuron[])
{
int i;
int Highest = 0;

for(i=0;i<82;i++)
{
if (Neuron[i] > Neuron[Highest])
Highest = i;
}
return(Highest);
}

These 3 functions are what I use to determin what answer the neural network is giving, I probably did something wrong here, but if I did, I''m sure someone will point it out.
the variable MProb is a char array that contains the first and second numbers and the mathmatical sign used(+, -, *, or /)


void Strengthen(float FirstLayer[24][82], float SecondLayer[82][82], int answer, char MProb[])
{
int sign;
int First;
int Second;
int i;
float Neuron[82];

if (MProb[1] == ''+'')
sign = 10;
if (MProb[1] == ''-'')
sign = 11;
if (MProb[1] == ''*'')
sign = 12;
if (MProb[1] == ''/'')
sign = 13;

First = MProb[0] - 48;
Second = MProb[2] - 35;

for (i=0;i<82;i++)
{
Neuron[i] = 1 * FirstLayer[First][i];
FirstLayer[First][i] += ((float)(random(0, 10)))/100;
Neuron[i] += 1 * FirstLayer[sign][i];
FirstLayer[sign][i] += ((float)(random(0, 10)))/100;
Neuron[i] += 1 * FirstLayer[Second][i];
FirstLayer[Second][i] += ((float)(random(0, 10)))/100;
}

for(i=0;i<82;i++)
{
if (Neuron[i] >= 1)
{
SecondLayer[i][answer] += ((float)(random(0, 2)))/100;
}
}
}

void Weaken(float FirstLayer[24][82], float SecondLayer[82][82], int answer, char MProb[])
{
int sign;
int First;
int Second;
int i;
float Neuron[82];

if (MProb[1] == ''+'')
sign = 10;
if (MProb[1] == ''-'')
sign = 11;
if (MProb[1] == ''*'')
sign = 12;
if (MProb[1] == ''/'')
sign = 13;

First = MProb[0] - 48;
Second = MProb[2] - 35;

for (i=0;i<82;i++)
{
Neuron[i] = 1 * FirstLayer[First][i];
FirstLayer[First][i] -= ((float)(random(0, 10)))/100;
Neuron[i] += 1 * FirstLayer[sign][i];
FirstLayer[sign][i] -= ((float)(random(0, 10)))/100;
Neuron[i] += 1 * FirstLayer[Second][i];
FirstLayer[Second][i] -= ((float)(random(0, 10)))/100;
}

for(i=0;i<82;i++)
{
if (Neuron[i] >= 1)
{
SecondLayer[i][answer] -= ((float)(random(0, 2)))/100;
}
}
if (sign == 10)
Strengthen(FirstLayer, SecondLayer, (First + (Second - 13)), MProb);
if (sign == 11)
Strengthen(FirstLayer, SecondLayer, (First - (Second - 13)), MProb);
if (sign == 12)
Strengthen(FirstLayer, SecondLayer, (First * (Second - 13)), MProb);
if (sign == 13)
Strengthen(FirstLayer, SecondLayer, (First / (Second - 13)), MProb);
}

These are my functions to strengthen and weaken the connections.

When I run the program it asks for a simple math problem, I put in a math problem and it gives me the answer it thinks it is. Then it asks me if it was right and if I say yes it uses the strengthen function and if I say no it uses the weaken function.
The weaken function determins the right answer and uses the strengthen function with the correct answer passed to it. If I ask the same problem a few times in a row it will get it right but then when I make it memorize another problem and I go back to the previous one it gets it wrong.

I hope this explains things a little better.

Thanks,

EAX

Share this post


Link to post
Share on other sites
First thing you should always do is comment pretty much every line of code. That way I know what various lines like

First = MProb[0] - 48;
Second = MProb[2] - 35;

are meant to do and you can compare logical design with implementation.
I''m not really sure how your code is meant to work, it''s not any algorithm I''ve seen for network training and the way you adjust weights seems almost random. Then again I probably don''t understand the algorithm.
Next up, what you should do is scrap having so many neurons and connections, there''s no way you need that many. What you should do is have the output neuron give a value between 0 and 1 and scale that value to fit the output you want. i.e. if you want the output to be between 0 and 82 simply multiply the output by 83 and round down.
Have you tried implementing something like backpropagation, that might be a good place to start, it''s not complex, is well understood and easier to debug by other people.
Sorry I can''t directly point to your problem.

Mike

Share this post


Link to post
Share on other sites
Ok thanks, I''ll revise my code and comment more. The way I was trying to do it was randomly change the weights. I''m not sure how to make 1 node give values between 0 and 1, all I know about is adjusting weights and I''m not very good at it. I''ve looked over several tutorials on neural nets and this source code was my way of trying to implement what I read about. I''ll take your advice and post another message with my results or any questions.

Thanks,

EAX

Share this post


Link to post
Share on other sites
Ok,
I''ve got my neural net able to memorize 1*1, 2*2, 3*3, 4*4, 5*5, 6*6, 7*7, 8*8, and 9*9 all at the same time, but it has trouble memorizing 5+6 and 6+5 at the same time, I have to tell it 5+5 and 6+5 several times before it gets them both right.
I''ve adjusted my weights to not got over 1 or below 0, and the weight is adjusted by hw high it is(It will go down more if it is a bigger number and less if it is a smaller number.) but it still uses randoms, even though it is more controlled now.

The reason I''m thinking it has a hard time with 5+5 and 6+5 is that they look so similar, it''s hard for it to distinguish the 2, but I''m not sure. I still haven''t fisgured out how to use just 1 node for output.

So, now my question is, should I keep working on my neural net until it can easily memorize 5+5 and 6+5 at the same time, or is it expected for a neural net to have a hard time with that?

Thanks again,

EAX

Share this post


Link to post
Share on other sites

  • Advertisement