A trained neural net is a weighted equation. The two are identical. The difference is in how the weighted equation is developed.

Hello

[quote name='Dave Weinstein' timestamp='1313167942' post='4848303']
A trained neural net is a weighted equation. The two are identical. The difference is in how the weighted equation is developed.

I've been trying to get that into people's heads for years but the schools keep cranking out people who think just saying "neural network" is sexy. "Weighted sums" doesn't impress people enough.
[/quote]

For a single layer net such as adaline or perceptron, with the activation function=identity , so we can say that the output is just a 'weighted sum' of the inputs
What about multi-layer networks ?
What about models that don't use the Mc Culloch & Pitts model, like spiking neural networks ? (LaPicque model for instance)

I think that 'weighted equation' is more accurate, at least a better shorcut

Hello

[quote name='IADaveMark' timestamp='1313170176' post='4848335']
[quote name='Dave Weinstein' timestamp='1313167942' post='4848303']
A trained neural net is a weighted equation. The two are identical. The difference is in how the weighted equation is developed.

I've been trying to get that into people's heads for years but the schools keep cranking out people who think just saying "neural network" is sexy. "Weighted sums" doesn't impress people enough.
[/quote]

For a single layer net such as adaline or perceptron, with the activation function=identity , so we can say that the output is just a 'weighted sum' of the inputs
What about multi-layer networks ?
What about models that don't use the Mc Culloch & Pitts model, like spiking neural networks ? (LaPicque model for instance)

I think that 'weighted equation' is more accurate, at least a better shorcut
[/quote]

I refer to them as "functions with parameters", or "functions with too many parameters for their own good".

"functions with too many parameters for their own good"
[/quote]

What do you mean ? I don't see the point

"functions with too many parameters for their own good"

What do you mean ? I don't see the point
[/quote]

An ANN consisting of a single neuron with linear activation function is multiple linear regression, and it is well understood since the times of Gauss. An ANN consisting of a single neuron with sigmoid activation function is logistic regression, and it's also well understood. A multi-layer perceptron has so many parameters that training becomes really hard (e.g., it's hard to avoid getting stuck in local minima), and trying to understand what each parameter does becomes hopeless. That's what I mean by "too many parameters for their own good".

An ANN consisting of a single neuron with linear activation function is multiple linear regression, and it is well understood since the times of Gauss. An ANN consisting of a single neuron with sigmoid activation function is logistic regression, and it's also well understood. A multi-layer perceptron has so many parameters that training becomes really hard (e.g., it's hard to avoid getting stuck in local minima), and trying to understand what each parameter does becomes hopeless. That's what I mean by "too many parameters for their own good".

Ah ok I understand what you meant, thanks

EDIT :
"avoid getting stuck in local minima" -> that's why a momentum is often added in the delta rule (but it doesn't eliminate the risk, and it's done by adding other parameters .... )

"understand what each parameter does becomes hopeless" -> we generally don't care about what the weight values are (and as you say it's hopeless anyway in a 'big' network)

I can think of two problems. One, even if we had sufficient processing power, there's the fact that the simulation would have to split time up in frames. That adds a certain amount of error, that would make the rules of that world fundamentally different from those of our world. Two, it presumes that we actually know more about the universe than we really do. We don't have perfect knowledge of physics yet. So we'd make assumptions, and the rules of the simulated world would drift even more.

And a third one that just occurred to me: Say you have preposterous processing power and perfect knowledge of the laws of physics. So you build your world (by the way, are you simulating just the world, or the sun as well? That's a whole other bunch of atoms to think about, and it's not like the sun doesn't have some small impact on our world, so four problems really), and you get your intelligence up and running. And now THEY want to make a computer to simulate the universe. So your computer has to simulate a world with a computer trying to simulate the world. And of course they are successful, so the intelligence that emerges in the simulated computer tries to simulate another world.

So now your computer is simulating a world that contains a computer that is simulating the simulated world, which will contain a computer that can simulate the simulated simulated world. And so on the recursion goes. And you want this to run in real time (in fact, millions of times faster than real time). You're essentially asking the computer to simulate the processing power of infinite computers just like itself. This is a logical impossibility.