The problem is that an ANN tends to have too many parameters, with the consequent complications in training and risk of overfitting the data. There is a class of ANNs that I find very useful: They have a single neuron, and are more commonly called multilinear regression (if the activation function is linear) or logistic regression (if the activation function is a sigmoid). Beyond that, things get messy very quickly, at least with traditional back-propagation networks.

The problem is that an ANN tends to have too many parameters, with the consequent risk of overfitting the data. There is a class of ANNs that I find very useful: They have a single neuron, and are more commonly called multilinear regression (if the activation function is linear) or logistic regression (if the activation function is a sigmoid). Beyond that, things get messy very quickly, at least with traditional back-propagation networks.