Archived

This topic is now archived and is closed to further replies.

Neural Networks Creation

This topic is 5570 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

What would a graph look like of the learning of a Neural Network? How fast would data be learned? Would it be expontential or would it start out fast and then slow? I would like to create my own Neural Network AI that is text based which would get it''s input and learn from the internet. Given today''s regular desktop and hard drive do I have enough power to pull this off. -Thanks for the advice http://abyssgames.cjb.net

Share this post


Link to post
Share on other sites
This would depend on the type of network and training algorithm. A feedforword network and backprop typically shows some "exponential decay" in error and ends up on some level and from there you have the change that it start going down again. So the total curve looks like stairs. Note that you have a change that you get stuck and have to restart training using different initial weights. More advanced training algorithm (2nd order) show less staircase behavior, they get there fast or they get stuck very fast.
I do not know what you try do do with the text from the internet but you should check websom. This uses unsupervised learning to cluster newsgroup article, so that you can browse them by jumping to articles that are "close by".

[edited by - smilydon on September 10, 2002 2:14:07 PM]

Share this post


Link to post
Share on other sites
I am not a neural networks expert, so if someone comes along and contradicts me, they''re probably right.

But the standard way would look something similar to a logarithmic curve, with the gradient decreasing with time. With each step the error is calculated, and that value is used to create new values. Since the error will decrease, the change in new values will also decrease, and the rate of learning will also decrease.

Technically with most networks you could get it to a point where the learning has almost stopped completely because the output is so accurate, but often this is not what you want as you nearly always want a neural network to be able to generalise.

[ MSVC Fixes | STL | SDL | Game AI | Sockets | C++ Faq Lite | Boost | Asking Questions | Organising code files | My stuff ]

Share this post


Link to post
Share on other sites
The shape of the graph would be as Kylotan described. (Assuming the network learns from a gradient descent algorithm.)

I''d need to know more about your project before I comment on your other question. What are its inputs/outputs, and what do you want it to learn?




ai-junkie.com

Share this post


Link to post
Share on other sites
quote:
Original post by Armitage
Actually it is possible for the error to reach a minimum, and then decrease again. It is important to stop ''at the right time'' with the learning if you want the best outcome.


That''s a point about over-fitting the data. As several other posters pointed out, the general form of the learning rate using gradient descent is logarithmic. What should also be noted is that the gradient of the learning curve depends on a) the specific learning algorithm parameter values; and, b) the geometry of the underlying manifold that the network exists in.

There was a great invited talk at PRICAI''00 on ''Information Geometry of Neural Networks'' (by Shun-ichi Amari). While it is a bit beyond what most people in this forum would consider ''fun reading material'', it does highlight some very interesting points about learning in neural networks.

Cheers,

Timkin

Share this post


Link to post
Share on other sites
I would like to create a text based AI so that it would understand the english language and be able to talk to you. However, now that I think about it I''m not sure how I could get it to work. Would the inputs be letters and their relationship and proximity to each other?(thus making the words and sentences) Then the output would be the answer? So the neural network would look something like a triangle where you start at the top part with your input value and work down towards the goal?

-Thanks for the help.

http://abyssgames.cjb.net

Share this post


Link to post
Share on other sites
quote:
Original post by firefly2442
Would the inputs be letters and their relationship and proximity to each other?(thus making the words and sentences) Then the output would be the answer?


At its very heart an ANN is a form of optimization (mathematical, not speed). What is it that you are trying to optimize? How does your output space dependent on your input space.

quote:
Original post by firefly2442
So the neural network would look something like a triangle where you start at the top part with your input value and work down towards the goal?


No, that would be some kind of wierd network structure expert system. It sounds as though (and correct me if I am wrong) you are trying to strip text off web pages and feed it to an ANN; which would then be able to answer questions based on what you have fed it. If that is correct then I forsee a few problems:

1. ANNs really only map an input space to an output space by minimizing the error (as best they can) between the two.

2. If you fed the ANN the text from this page what would be the desired output? The same text?

3. How do you plan to encode characters / variable length strings as input?

- mongrelprogrammer

Share this post


Link to post
Share on other sites
From what I understand of your description, ANNs are not what you should be considering.

You can use ANNs for some types of text processing though. ANNs have been used reasonably successfully in spam filters for example.




ai-junkie.com

Share this post


Link to post
Share on other sites
The problem you are considering is a very current topic in AI research. Typically there are several tasks involved in document classification and none of them are trivial. A popular approach to this problem involves probabilistic clustering (classification). While ANNs can be used for classification tasks, it would require finding a suitable mapping from the document/text space into an input space suitable for ANN classification. As yet, I have not heard of anyone doing this successfully. I would suggest that you do some scanning of the web - in particular university comp. sci. departments - to find more information on this subject.

Cheers,

Timkin

[edited by - Timkin on September 11, 2002 10:03:30 PM]

Share this post


Link to post
Share on other sites