Jump to content

  • Log In with Google      Sign In   
  • Create Account


math required for Adaline neural network


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
8 replies to this topic

#1 rrlangly   Members   -  Reputation: 110

Like
0Likes
Like

Posted 20 January 2013 - 04:37 PM

Hi,

 

I've implemented an adaline neural network by looking at example code, but at times it seems buggy and I really need to understand it better to know exactly what my problem is.

 

Eventually, I'd like to implement other types of networks, but just to start off with and for adalines only ...

 

What is the minimim math I need to understand this. My current plan is to buy some math books just to cover what I need and dig in.

 

Any suggestions/recommendations much appreciated.



Sponsor:

#2 Álvaro   Crossbones+   -  Reputation: 11896

Like
0Likes
Like

Posted 20 January 2013 - 05:34 PM

You need to understand what a gradient is. How much math do you know? Do you understand what a derivative is?

#3 rrlangly   Members   -  Reputation: 110

Like
0Likes
Like

Posted 20 January 2013 - 07:04 PM

I understand this conceptually from reading high-level descriptions, but I think I need to get to the point where I can work out a problem and see it through. So the answer is no.



#4 Álvaro   Crossbones+   -  Reputation: 11896

Like
0Likes
Like

Posted 20 January 2013 - 08:25 PM

Do you know how to compute the derivative of 1/(1+exp(-x))?

#5 rrlangly   Members   -  Reputation: 110

Like
0Likes
Like

Posted 21 January 2013 - 04:08 PM

I'd have to say no.



#6 Álvaro   Crossbones+   -  Reputation: 11896

Like
0Likes
Like

Posted 21 January 2013 - 05:17 PM

I would say if you understand the concept of gradient descent and how to compute the gradient of the error function as a function of the weights, that's about all you need to know.

#7 rrlangly   Members   -  Reputation: 110

Like
0Likes
Like

Posted 27 January 2013 - 03:33 PM

And what areas/courses would this mostly be taught under so I can find the books to study in-depth?



#8 Álvaro   Crossbones+   -  Reputation: 11896

Like
0Likes
Like

Posted 27 January 2013 - 07:27 PM

The area is called multivariable calculus, but you only need the very basics.

#9 Selenaut   Members   -  Reputation: 102

Like
1Likes
Like

Posted 28 January 2013 - 03:14 PM

Honestly, I can't help you with your application, but I can tell you how the simple calculus works (which I'm assuming is what Alvaro is referring to).

 

To give an extremely simple introduction, a derivative is the instantaneous rate of change at a specific point on a line. The derivative of a function f(x) (which is notated f'(x)) is found by using the difference quotient:

42cf4f4861ae1266b13104c4115e7b5d.png

 

(This picture's from the Wikipedia article on derivatives; just replace a with x.)

 

So if f(x) = x2, f'(x) = {the limit as h approaches 0 of} ([x+h]2 - x2)/h. This may seem like it would equal 0; however, if the h variable is left in during expansion, you get:

 

{the limit as h approaches 0 of}(x2 + 2xh + h2 - x2)/h  --> {the limit as h approaches 0 of}(2xh + h2)/h --> {the limit as h approaches 0 of} 2x + h = 2x.

 

Therefore, the derivative of x2 is equal to 2x.

 

There are a lot of rules for derivatives that make derivation a lot easier, e.g. the power rule. This states that the derivative of a polynomial function's derivative can be found by taking each term, multiplying it by its power, and then reducing the power by 1. Thus the derivative of x3 = 3x2, and the derivative of 2x2 + 3x = 4x + 3, etc.

 

Now, for Alvaro's function, (1+e-x)-1, there are a few other rules involved; its derivative (I'm pretty sure) is -(1+e-x)-2-e-x.

 

Hope this helps,

 

Selenaut






Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS