Raeldor

Members
  • Content count

    862
  • Joined

  • Last visited

Community Reputation

254 Neutral

About Raeldor

  • Rank
    Advanced Member
  1. Oh, and also the error sum from the network seems to go down AND up during training. That can't be right, right? Surely if it's adjusting using an error adjustment factor, it must always go up?
  2. Hi All, I've been trying to adapt the code to make it compatible with more than one hidden layer, but I think on P306 after the line '// and now we calculate the error', after calculating the error it should then assign the error value to the neuron, should it not? My network is kind-of functioning, but behaving a little oddly. The adapted code looks like below if anyone would care to have a look and see if they can spot anything odd. Thanks Rael [code] -(bool)trainNetworkOnceWithInputSet2:(double*)inInputSet ofSize:(int)inSetSize andOutputSet:(double*)inOutputSet { // cumulative error for training set errorSum = 0.0f; // run each input pattern through the network, get outputs, calculate error and adjust weights for (int s=0; s < inSetSize; s++) { // first run input pattern and get outputs double *outputs = [self updateWithInputs:&inInputSet[s*inputCount]]; // save layers for readability NeuronLayer *outputLayer = [layers objectAtIndex:hiddenLayerCount]; // for each output neuron, calculate the error and adjust the weights for (int o=0; o < outputCount; o++) { // first calculate the error value double err = (inOutputSet[s*outputCount+o] - outputs[o]) * outputs[o] * (1.0f - outputs[o]); // update the error total (when this becomes lower than threshold, training is success) errorSum += (inOutputSet[s*outputCount+o] - outputs[o]) * (inOutputSet[s*outputCount+o] - outputs[o]); // keep a record of the error value Neuron *thisOutputNeuron = [outputLayer.neurons objectAtIndex:o]; thisOutputNeuron.errorValue = err; // for each weight for this output neuron except the bias int hiddenNeuronIndex = 0; NeuronLayer *hiddenLayer = [layers objectAtIndex:hiddenLayerCount-1]; for (int w=0; w < neuronsPerHiddenLayer; w++) { // calculate the new weight based on the backprop rules Neuron *thisHiddenNeuron = [hiddenLayer.neurons objectAtIndex:hiddenNeuronIndex]; thisOutputNeuron.weights[w] += err * learningRate * thisHiddenNeuron.activation; // go to next weight and next hidden neuron hiddenNeuronIndex++; } // also adust the bias thisOutputNeuron.weights[neuronsPerHiddenLayer] += err * learningRate * bias; } // loop through hidden layers for (int h=hiddenLayerCount-1; h >= 0; h--) { // get this hidden layer NeuronLayer *hiddenLayer = [layers objectAtIndex:h]; NeuronLayer *layerAbove = [layers objectAtIndex:h+1]; // for each neuron in the hidden layer, calc and adjust the weights for (int n=0; n < neuronsPerHiddenLayer; n++) { double err = 0; // to calculate the error for this neuron we need to loop through the output neurons and sum the errors * weights int myOutputCount = (h+1 == hiddenLayerCount ? outputCount : neuronsPerHiddenLayer); for (int o=0; o < myOutputCount; o++) { Neuron *thisOutputNeuron = [layerAbove.neurons objectAtIndex:o]; err += thisOutputNeuron.errorValue * thisOutputNeuron.weights[n]; } // now we can calculate the error Neuron *thisHiddenNeuron = [hiddenLayer.neurons objectAtIndex:n]; err *= thisHiddenNeuron.activation * (1.0f - thisHiddenNeuron.activation); // save error value to this hidden neuron thisHiddenNeuron.errorValue = err; // for each weight in this neuron (hidden) calculate the new weight based on the error signal and learning rate int myInputCount = (h == 0 ? inputCount : neuronsPerHiddenLayer); for (int w=0; w < myInputCount; w++) { if (h == 0) thisHiddenNeuron.weights[w] += err * learningRate * inInputSet[s*inputCount+w]; else { NeuronLayer *layerBelow = [layers objectAtIndex:h-1]; Neuron *inputNeuron = [layerBelow.neurons objectAtIndex:w]; thisHiddenNeuron.weights[w] += err * learningRate * inputNeuron.activation; } } // calculate the bias thisHiddenNeuron.weights[myInputCount] += err * learningRate * bias; } } // free outputs array free(outputs); } // make error sum not dependent on set size or output count errorSum /= (float)inSetSize*(float)outputCount; // all went ok return YES; }[/code]
  3. Oh, and interestingly, if I 'normalize' the inputs to make them fit the range 0 to 1 closer, the network seems to function better. So, I'm wondering more specifically why doesn't the network pick up on the finer details? Is it more a question of increasing the training until the error rate drops lower? Thanks Ray
  4. Hi All, I've code up a network identical to that in the 'AI techniques for game programming' book chapter 9, which is a supervised training network with a single hidden layer. I tested it using the XOR problem which works fine, but when I increase the inputs and outputs and try and train it on data that's a little more 'subtle' it doesn't function too well. When I say 'subtle', what I mean is that instead of 1's and 0's as input I have floating values between 1 and 0, but the differences between the training sets for any one input usually involve changes of maybe 0.1 in value. The outputs (of which there are 10) are either 1 or 0 and only one of them is '1' for any one training set (a classification problem). I guess my first question is, should this type of network suit itself well to such data, or is it only really designed to accept 1 and 0 as inputs? Thanks Ray
  5. Quick Math Angle From Vectors Question

    That's awesome, thank you. It looks a lot more elegant than my code! :P. I'll give it a shot and then try and break it down and see how it's working from a geometry point of view. OOI, what would be your preferred method of storage for rotations? Quaternions? Thanks Rael
  6. Quick Math Angle From Vectors Question

    Yes, you are right on the money with your assumptions. As for why I chose euler angles, well I guess because it's easier to visualize when debugging. Also, yes I do not really have camera roll ability (at least not yet). I am not familiar with cartesian-to-spherical coordinate conversion. It sounds like it somehow treats the co-ordinate as a point on a sphere? I would love some more information on this if you have it. Thanks!
  7. Quick Math Angle From Vectors Question

    Quote:Original post by jyk Quote:Original post by Raeldor Hi All, How can I get 3 independent euler angles from two vectors? I know the dot product of the normalized vectors will give me the difference in angle, but I need to the difference in angle in each of the x, y, z planes so I can build a rotation matrix from it.I didn't look at your code examples too carefully, but I am curious as to why you think you need an Euler angle triple in order to build a rotation matrix. I'm also wondering what you mean exactly by getting angles 'from two vectors'. Do you mean you want a rotation that rotates from one vector to the other? If your current code is working as you want it to then I suppose there's no need to mess with it, but I kind of suspect that Euler angles are not the optimal (or most conceptually appropriate) solution to the problem you're trying to solve. Thanks for the reply. I basically have a base camera class that works using position (vector3) and rotation (vector3), same as any other class based on Node. I sub-classed this camera to provide a 'TrackCamera' class, which can follow and/or track (point to) an object. I have calculated the camera position and the target position, but I now need to work out the rotation of the camera to point the camera towards the target as the 3 euler angles required by the base class. The rotation of the camera is based off a base vector of (0,0,1), ie looking straight down the z axis. Hope this clarifies a little; sorry for the confusion.
  8. Quick Math Angle From Vectors Question

    Hmm, it seems I got it to work, though I'm not sure why. Maybe someone could explain? I inserted the code... // try rotate direction by y direction=NWVectorTransform(NWMatrixRotateY(-m_rotation.y), direction); between the two calculations. I assume I'm somehow lining up the vector into the zy dimension? I can't really visualize it, so I'm not sure exactly why it is working.
  9. Quick Math Angle From Vectors Question

    Oh, as an addendum, I did get the angle for the rotation around 'y', by doing... // calculate y rotation Vector2 directionZx=Vector2(direction.z, direction.x); directionZx.Normalize(); m_rotation.y=acos(Vector2(1.0f, 0.0f).Dot(directionZx)); if (direction.x &lt; 0.0f) m_rotation.y=-m_rotation.y; But when I try and get the rotation around x the same way... // calculate x rotation Vector2 directionZy=Vector2(abs(direction.z), direction.y); directionZy.Normalize(); m_rotation.x=-acos(Vector2(1.0f, 0.0f).Dot(directionZy)); if (direction.y &lt; 0.0f) m_rotation.x=-m_rotation.x; Is has strange results as 'z' passes through zero when 'y' is close to zero because they seem to move closer together which confuses the dot product (if that makes sense?).
  10. Hi All, How can I get 3 independent euler angles from two vectors? I know the dot product of the normalized vectors will give me the difference in angle, but I need to the difference in angle in each of the x, y, z planes so I can build a rotation matrix from it. Is this possible? Thanks! Rael
  11. Hi All, Just a quick question. At the moment I am holding a local transform matrix for my objects and dealing with this to change the position, rotation etc., but it seems it would make the code more readable to have seperate position+rotation+scale and then just calculate the matrix when I'm running through my graph to calculate the world matrix. Is there a downside to doing this other than a small performance hit? What's the recommended approach? Thanks! Rael
  12. Hi All, I have a node class which is the base class of all nodes in the scene graph. At the moment it has such methods as UpdateWorldTransform(), UpdateControllers(), Draw(), and QueueToRenderer() for example. Some of these like UpdateWorldTransform() are recursive to the children and others like Draw() just draw the single node (as they are intended to be called from another graph class such as QuadTree). However, I am wondering if I should simplify this class since not all nodes will need controllers updating, or even drawing and rendering (ie, a camera technically doesn't need drawing). Is it good design to have some objects that have null implementations of base class methods? Is this acceptable, or should I move functions such as Draw() out into a RenderableNode class? I assume I would then have to rely on my QuadTree to make the calls or not based on the node-type. I'm conflicted as to whether null implementations of these base functions or a case statement based on the node type is the cleanest solution. Interested in your thoughts. Thanks! Rael
  13. Thanks for the link, I'll definitely read up on this. And it's not frowned upon to have helper functions sitting outside of classes in C++? Ie, Vector3 TransformVector(Vector3, Matrix)?
  14. Hi Guys, Thanks for the responses. I'm kind of shocked, I guess being new to C++, I expected it would be cleaner to implement everything inside classes, but I guess that's not the case. What about where I have the method inline Vector3 Cross(Vector3 in_right) {return Vector3(y*in_right.z-z*in_right.y, z*in_right.x-x*in_right.z, x*in_right.y-y*in_right.x);} Should this also be defined as a seperate math function passing in both the left and the right as opposed to a method? What is the 'Single Responsibility Principle', and where does this term come from? Is there a book describing these kinds of principles and best practices for C++ programming? Thanks! Rael [Edited by - Raeldor on December 18, 2006 12:10:25 PM]
  15. Hi All, I have two classes that have functions that are dependent on each other. The classes are a Matrix class and a Vector3 class. At the moment to resolve this I forward declare one of them and have the function declared in the CPP file not in the header file which seems to work. Problem is, I would like to make the function inline. If I declare the function in the header file still but outside of the class definition (ie, later in the module) can the compiler still put it inline? Thanks! Rael