ANN Question

quote:

---

Original post by The Reindeer Effect
So I have been reading up on AI lately, and from what I can tell ANN''s are what I need (recognizing trends and predicting based on a large dataset).

---

Aaaarrrrrggggggggggghhhhhhhhhhhhhhhhhhhh!!!! *tears out some more hair*

Why is this the common conclusion that people come to when looking at data mining??? Did the popular media of the 80s so deeply pervert the perception of AI that the only thing people see when they look at AI is ANNs?

The tasks you describe are sub-problems within data mining. ANNs are certainly NOT the only tool for this task and typically speaking, aren''t the best tool for the task either! Don''t limit your capabilities by restricting your toolbox to just one tool. Learn about other techniques: regression (linear and nonlinear), causal modelling, Bayesian Inference, clustering/classification, etc. Once you''ve got these tools in your toolbox, you''ll be in a good position to try different tools on the job and find the one that works the best for the specific problem you have... and typically speaking, evaluating best is difficult... different techniques will generally give you subtley different results that you''ll have difficulty identifying the ''correct'' result within (because typically the correct result is inaccessible... or you wouldn''t need to do the data mining in the first place)!

Timkin

Well timkin beat me to it, but I can add a little. What you have is indeed a data mining problem, so my AI professors homepage may have some valuable information for you (data mining is his specialty).

Try this out:
Statistical Data Mining Tutorials

While it is a lot of reading, it contains tutorials for most of the tools you would need in your "toolbox" for data mining, including neural nets. I recommend reading the Decision Trees tutorial first, as it is fairly simple and might possibly be good enough for your application.

quote:

---

Original post by Timkin
Why is this the common conclusion that people come to when looking at data mining??? Did the popular media of the 80s so deeply pervert the perception of AI that the only thing people see when they look at AI is ANNs?

---

Well, gosh, Timkin, doesn''t AI stand for Artificial Intelligence? And isn''t a neural network really just an exact computer replica of a human brain, intelligence and everything? seems like that would be the perfect way to do EVERYTHING in AI EVER!

---

"Sneftel is correct, if rather vulgar." --Flarelocke

quote:

---

Original post by Sneftel

quote:

---

Original post by Timkin
Well, gosh, Timkin, doesn''t AI stand for Artificial Intelligence? And isn''t a neural network really just an exact computer replica of a human brain, intelligence and everything? seems like that would be the perfect way to do EVERYTHING in AI EVER!

---

A neural network may be close to the human brain, but not an ARTIFICIAL neural network (ANN) ANNs are a very stripped down version of the way we think the brain works. Saying that they are the exact same thing isn''t accurate IMHO.

In any case, he didn''t say ANNs aren''t AI. He said that for data mining, there are many tools, but people only seem to consider ANNs a lot of the time.

<SPAN CLASS=smallfont>quote:

---

Original post by The Reindeer Effect
So I have been reading up on AI lately, and from what I can tell ANN''s are what I need (recognizing trends and predicting based on a large dataset). My biggest question, which from the tutorial or two that I read was not mentioned, is the actual TOPOLOGY of the network. Pretty much the generic explanation is "input layer -> hidden layer -> output". What exactly is this? How do I design a network, how do I know which neurons should link to which? What does adding more neurons do to the network, and what does refactoring the network topology (just changing links) do?

---

</SPAN>

The most commonly implemented artificial neural networks are multilayer perceptrons (MLP). Typically, the input "layer" is simply the model inputs (the independent variables). The hidden and output layers are collections of nodes, which calculate weighted sums of the values coming from the previous layer and run them through (usually) sigmoid transfer functions. These nonlinear nodes are essentially basis functions which are assembled to produce an approximation to the training data. Most often (by far), each node receives data from all nodes in the previous layer, in addition to a bias value (equivalent to the intercept in a linear regression). Sometimes, "jump" connections are included, which allow information to flow between non-contiguous layers. Also, some implementations utilize only partial connections between layers. Increasing the number of hidden nodes increases the maximum possible complexity of the function being approximated (which may lead to overfitting if not used carefully).

I recommend at least browsing through the Usenet comp.ai.neural-nets FAQ, which one may find here:

ftp://ftp.sas.com/pub/neural/FAQ.html

Additionally, I suggest "Computer Systems That Learn", by Weiss and Kulikowski, as an excellent introduction to important issues surround theuse of neural networks and other empirical modeling algorithms.


-Predictor
Data Mining in MATLAB


[Edited by - Predictor on February 13, 2009 4:06:33 PM]

my goodness. What does it take to close a <sarcasm> tag around here?

---

"Sneftel is correct, if rather vulgar." --Flarelocke

<SPAN CLASS=smallfont>quote:

---

Original post by Timkin
The tasks you describe are sub-problems within data mining. ANNs are certainly NOT the only tool for this task and typically speaking, aren't the best tool for the task either!

---

</SPAN>

Do you mean that neural networks are never the best tools for such tasks, that neural networks are usually not the best or that neural networks are not always the best, or something else altogether?

-Predictor
Data Mining in MATLAB




[Edited by - Predictor on February 13, 2009 4:11:58 PM]

quote:

---

Original post by Predictor
Do you mean that neural networks are never the best tools for such tasks, or that neural networks are not always the best, or something else altogether?

---

I think my AI professor put it best: "ANNs are the second best way to do anything". It''s a useful techique, and the generality with which it may be applied is very attractive. But it''s a technique that cannot exploit the invariants and heuristics of a particular problem, and thus is usually doomed to be less effective than a custom-engineered AI solution.

---

"Sneftel is correct, if rather vulgar." --Flarelocke

quote:

---

Original post by Sneftel
I think my AI professor put it best: "ANNs are the second best way to do anything". It''s a useful techique, and the generality with which it may be applied is very attractive. But it''s a technique that cannot exploit the invariants and heuristics of a particular problem, and thus is usually doomed to be less effective than a custom-engineered AI solution.

---

Can you give an example of an invariant or heuristic that neural networks cannot exploit, which some other technique could?

-Predictor
http://will.dwinnell.com

quote:

---

Original post by Predictor
Do you mean that neural networks are never the best tools for such tasks, that neural networks are usually not the best or that neural networks are not always the best, or something else altogether?

---

Lock in ''b'' please! What I meant above is that ANNs are usually not the best task for the job. Why do I think that? Because, generally speaking, the best tool is job dependent... ANNs are a broad spectrum solution, usually tunable to a given task. That broad spectrum applicability comes at the cost of accuracy. Even in tasks where the ANN is more accurate than some other method, this accuracy comes at a higher price due to the requirement of data pre-processing, extensive tuning of network architecture and extensive training. I admit to a slight bias, but I believe that techniques such as MML classification are, while maintaining broad spectrum applicability, also typically more accurate than ANNs on the same problems. Personally, I''m a big fan of causal modelling.

Of course, I''m by no means an expert on ANNs. My beliefs arise from limited experience with ANNs (although a lot of experience with other techniques) and reading lots of literature, so if you can offer a counter claim based on your experiences, I''m all ears!

Cheers,

Timkin

<SPAN CLASS=smallfont>quote:

---

Original post by Timkin

<SPAN CLASS=smallfont>quote:

---

Original post by Predictor
Do you mean that neural networks are never the best tools for such tasks, that neural networks are usually not the best or that neural networks are not always the best, or something else altogether?

---

</SPAN>

Lock in ''b'' please! What I meant above is that ANNs are usually not the best task for the job. Why do I think that? Because, generally speaking, the best tool is job dependent... ANNs are a broad spectrum solution, usually tunable to a given task. That broad spectrum applicability comes at the cost of accuracy. Even in tasks where the ANN is more accurate than some other method, this accuracy comes at a higher price due to the requirement of data pre-processing, extensive tuning of network architecture and extensive training. I admit to a slight bias, but I believe that techniques such as MML classification are, while maintaining broad spectrum applicability, also typically more accurate than ANNs on the same problems. Personally, I''m a big fan of causal modelling.

Of course, I''m by no means an expert on ANNs. My beliefs arise from limited experience with ANNs (although a lot of experience with other techniques) and reading lots of literature, so if you can offer a counter claim based on your experiences, I''m all ears!

---

</SPAN>

I''d say there is a domain-knowledge spectrum along which problems lie. Some problems are well-studied and understood and have known, closed-form solutions.

The less that is known about a problem, especially regarding the functional form of its solution, the more useful empirical modeling becomes (assuming the availability of historical data of sufficient quality and quantity, of course). This is particularly true of nonparametric modeling, such as neural networks.

Assuming that a problem is a good candidate for empirical modeling, I know of no reason that neural networks (and remember that term covers many different techniques) should be considered, a priori, any better or worse than any other modeling technique.

-Predictor
Data Mining in MATLAB


[Edited by - Predictor on February 13, 2009 4:15:28 PM]

quote:

---

Original post by Predictor
Assuming that a problem is a good candidate for empirical modeling, I know of no reason that neural networks should be considered, a priori, any better or worse than any other modeling technique.

---

That''s a fair comment and one I would expect from someone who has made a living from data mining! I think one of the big detractions for ANNs though is that techniques for unsupervised learning of network topology are still in their infancy, even though ANNs have been widely used for more than 20 years in commercial practice. This results in the need for trained, specialised and experienced users who can craft networks by hand and tune them to produce answers. Those answers are then necessarily correlated with the practices of the user, meaning that one must believe in the quality of the user before they can believe in the quality of the results.

Cheers,

Timkin

<SPAN CLASS=smallfont>quote:

---

Original post by Timkin

<SPAN CLASS=smallfont>quote:

---

Original post by Predictor
Assuming that a problem is a good candidate for empirical modeling, I know of no reason that neural networks should be considered, a priori, any better or worse than any other modeling technique.

---

</SPAN>

That''s a fair comment and one I would expect from someone who has made a living from data mining! I think one of the big detractions for ANNs though is that techniques for unsupervised learning of network topology are still in their infancy, even though ANNs have been widely used for more than 20 years in commercial practice. This results in the need for trained, specialised and experienced users who can craft networks by hand and tune them to produce answers. Those answers are then necessarily correlated with the practices of the user, meaning that one must believe in the quality of the user before they can believe in the quality of the results.

---

</SPAN>

Regarding MLPs in particular, my feeling is that too much is made about discovering the "optimal" architecture. My experience has been that MLPs with some variety of sigmoid transfer function, a single hidden layer, and no fanciness (no jump connections, recurrent connections, node pruning, etc.) can be adequately be trained using early stopping to prevent overfitting.

While I agree that this work is best performed by a qualified analyst, I think that appropriate tools will make short work of neural network training. Contrary to the stories of neural network newcomers which I read on Usenet and forums such as this, my neural networks rarely require hundreds of hidden neurons or days to train. I suppose this leaves us in at least partial agreement that neural networks (MLPs, anyway) can be demanding to implement.

I do disagree with your last point, however. No one ever need take my models on faith, or guess as to how well they will work in practice. I test my models soundly using error resampling and (ignoring model drift) can tell you how well they will work using the performance metric of your choice. Good practice demands this sort of model assessment. Whatever idiosyncracies exist in my work, good or bad, will show up in the validation.

-Predictor
Data Mining in MATLAB

[Edited by - Predictor on February 13, 2009 4:54:09 PM]

quote:

---

Original post by Predictor
I do disagree with your last point, however. No one ever need take my models on faith, or guess as to how well they will work in practice. I test my models soundly using error resampling and (ignoring model drift) can tell you how well they will work using the performance metric of your choice.

---

That''s not quite what I meant. Certainly, any given model, or set of models, can be assessed for quality using an objective measure. However, the choice of the original model is still subject to the users experiences, or their ability to tune a given model. See my point? Certainly, an experienced user is going to hit the nail on the head more often than not and select a reasonable model... but a lay person is not. A lay person knowing that their model is inaccurate according to some measure doesn''t typically help them to discern a better model.

Cheers,

Timkin

<SPAN CLASS=smallfont>quote:

---

Original post by Timkin
Certainly, any given model, or set of models, can be assessed for quality using an objective measure. However, the choice of the original model is still subject to the users experiences, or their ability to tune a given model. See my point? Certainly, an experienced user is going to hit the nail on the head more often than not and select a reasonable model... but a lay person is not. A lay person knowing that their model is inaccurate according to some measure doesn''t typically help them to discern a better model.

---

</SPAN>

Mmm... I suppose MLPs can be more subtle than competing modeling technologies (tree induction, rule induction, k-NN, etc.), but I don''t know how much difference this makes in terms of needed skill, given the skill necessary to deal with all the issues outside of actual modeling (sampling, data pre-processing, data post-procesing, etc.) which will be needed for any modeling project. I have seen people wreck their comparatively simple linear and logistic regression analyses on the rocks of inadequate sample size, questionable data collection procedures and poor target variable definition.

-Predictor
Data Mining in MATLAB

[Edited by - Predictor on February 13, 2009 4:20:10 PM]

Yes, I''d agree that other techniques are not ''straightforward'' either. All data-mining algorithms that I have encountered required some level of prior knowledge of the technique and its applicability to arrive at reasonable results, even those based on causal modelling (although very little information was required in this case and only because the software was research grade, not commercial grade). I would suspect that preference for one technique over another, or rating as to the optimality of one technique over others, will always be subjective based on ones experiences, unless someone does develop the perfectly idiot-proof data-mining tool!

It''s been an interesting discussion. Thanks!

Timkin