Sign in to follow this  
daniel_i_l

writing to a dat file

Recommended Posts

I have a NN that I train and after training it I want to save it so I dont have to train it all the time. The NN header looks like this:
typedef vector<double> iovector;




//-------------------------------------------------------------------
//	define neuron struct
//-------------------------------------------------------------------
struct SNeuron
{
	//the number of inputs into the neuron
	int				      m_iNumInputs;

	//the weights for each input
	vector<double>	m_vecWeight;

  //the previous timesteps weight update used 
  //to add momentum
  vector<double>  m_vecPrevUpdate;

  //the activation of this neuron
  double          m_dActivation;

  //the error value
  double          m_dError;

	//ctor
	SNeuron(int NumInputs);
};


//---------------------------------------------------------------------
//	struct to hold a layer of neurons.
//---------------------------------------------------------------------

struct SNeuronLayer
{
	//the number of neurons in this layer
	int					      m_iNumNeurons;

	//the layer of neurons
	vector<SNeuron>		m_vecNeurons;

	SNeuronLayer(int NumNeurons, 
				       int NumInputsPerNeuron);
};


//----------------------------------------------------------------------
//	neural net class
//----------------------------------------------------------------------

class CNeuralNet
{
	
private:
	
	int					m_iNumInputs;

	int					m_iNumOutputs;

	int					m_iNumHiddenLayers;

	int					m_iNeuronsPerHiddenLyr;

  //we must specify a learning rate for backprop
  double      m_dLearningRate;

  //cumulative error for the network (sum (outputs - expected))
  double      m_dErrorSum;

  //true if the network has been trained
  bool        m_bTrained;

  //epoch counter
  int         m_iNumEpochs;

	//storage for each layer of neurons including the output layer
	vector<SNeuronLayer>	m_vecLayers;

  //given a training set this method performs one iteration of the
  //backpropagation algorithm. The training sets comprise of series
  //of vector inputs and a series of expected vector outputs. Returns
  //false if there is a problem.
  bool            NetworkTrainingEpoch(vector<iovector> &SetIn,
                                       vector<iovector> &SetOut);

  void            CreateNet();

  //sets all the weights to small random values
  void            InitializeNetwork();
  
	//sigmoid response curve
	inline double	  Sigmoid(double activation, double response);


public:


  CNeuralNet::CNeuralNet(int    NumInputs,
                         int    NumOutputs,
                         int    HiddenNeurons,
                         double LearningRate);


	//calculates the outputs from a set of inputs
	vector<double>	Update(vector<double> inputs);

  //trains the network given a training set. Returns false if
  //there is an error with the data sets
  bool            Train(CData* data);

  //accessor methods
  bool            Trained()const{return m_bTrained;}
  double          Error()const  {return m_dErrorSum;}
  int             Epoch()const  {return m_iNumEpochs;}

};

I have a Control class with a pointer to an instance of the NN like this:
class CController 
{

private:

  //the neural network
  CNeuralNet*     m_pNet;

  //this class holds all the training data
  CData*          m_pData;
  
  //the user mouse gesture paths - raw and smoothed
  vector<POINTS> m_vecPath;
  vector<POINTS> m_vecSmoothPath;

  //the smoothed path transformed into vectors
  vector<double> m_vecVectors;

  //true if user is gesturing
  bool    m_bDrawing;

  //the highest output the net produces. This is the most
  //likely candidate for a matched gesture.
  double  m_dHighestOutput;

  //the best match for a gesture based on m_dHighestOutput
  int     m_iBestMatch;

  //if the network has found a pattern this is the match
  int     m_iMatch;

  //the raw mouse data is smoothed to this number of points
  int     m_iNumSmoothPoints;

  //the number of patterns in the database;
  int     m_iNumValidPatterns;

  //the current state of the program
  mode    m_Mode;

  //local copy of the application handle
  //HWND    m_hwnd;

  //clears the mouse data vectors
  void    Clear();
  
  //given a series of points whis method creates a path of 
  //normalized vectors
  void    CreateVectors();

  //preprocesses the mouse data into a fixed number of points
  bool    Smooth();

  //tests for a match with a prelearnt gesture
  bool    TestForMatch();
  
  //dialog box procedure. A dialog box is spawned when the user
  //enters a new gesture.
  //static BOOL CALLBACK DialogProc(HWND   hwnd,
  //                                UINT   msg,
  //                                WPARAM wParam,
  //                                LPARAM lParam);

  //this temporarily holds any newly created pattern names
  //static string m_sPatternName;


public:

  CController();
  
  ~CController();


  //call this to train the network using backprop with the current data
  //set
  bool TrainNetwork();

  //renders the mouse gestures and relevant data such as the number
  //of training epochs and training error
  void Render();

  //returns whether or not the mouse is currently drawing
  bool Drawing()const{return m_bDrawing;}

  //this is called whenever the user depresses or releases the right
  //mouse button.
  //If val is true then the right mouse button has been depressed so all
  //mouse data is cleared ready for the next gesture. If val is false a
  //gesture has just been completed. The gesture is then either added to
  //the current data set or it is tested to see if it matches an existing
  //pattern.
  //The hInstance is required so a dialog box can be created as a child
  //window of the main app instance. The dialog box is used to grab the
  //name of any user defined gesture
  bool Drawing(bool val);

  void LearningMode();

  //call this to add a point to the mouse path
  void AddPoint(POINTS p)
  {
    m_vecPath.push_back(p);
  }  
  
  void SaveNet();
  void LoadNet();
};

the Save and Load look like this:
void CController::SaveNet()
{
   ofstream fout("file2.dat", ios::binary);
   fout.write((char *)(&m_pNet), sizeof(m_pNet));
   fout.close();
}

void CController::LoadNet()
{
   m_Mode = TRAINING;
   ifstream fin("file2.dat", ios::binary);
   fin.read((char *)(&m_pNet), sizeof(m_pNet));
   fin.close();
   m_Mode = ACTIVE;
}

Now after training the NN I told it to save, and then the next time I told it to load. I dont think that the save worked cause after saving file2.dat had only 1KB. But also after loading it didn't do anything and gave nonsense answers. What did I do wrong? (is it something with the pointers? Thanks.

Share this post


Link to post
Share on other sites
You can't just write the object into the file that easily. Since your class contains pointers, writing the raw object will simply write the values of those pointers - i.e. memory addresses - and not the data they point to. The net result is that, when you load the file back up, you're pointing to an address that was valid from the last run of the program, but is probably now bogus empty space. That's where your results are coming from.

What you need to do is manually write the fields of each object into the file, and store information like neuron counts and such, so that when you reload the objects you can easily reconstruct the original data.



You might also check into boost::serialization which can help make some of that detail easier to manage.

Share this post


Link to post
Share on other sites
Ok, since the only thing I really have to save are the weights I decided to do this:
I added these methods to the NN class:

vector<SNeuronLayer> GetNeurons(){return m_vecLayers;}
void PutNeurons(vector<SNeuronLayer> &n){m_vecLayers = n;}



and changed the save/load:

void CController::SaveNet()
{
vector<SNeuronLayer> netdata = m_pNet->GetNeurons();
ofstream fout("file2.dat", ios::binary);
fout.write((char *)(&netdata), sizeof(netdata));
fout.close();
}

void CController::LoadNet()
{
vector<SNeuronLayer> netdata;
m_Mode = TRAINING;
ifstream fin("file2.dat", ios::binary);
fin.read((char *)(&netdata), sizeof(netdata));
fin.close();
m_pNet->PutNeurons(netdata);
m_Mode = ACTIVE;
}


but the weights still don't get loaded into the NN? Why doesn't it work now?
Thanks!

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
You can't downcast a vector like that. You will have to save each element of the vector in turn, and read it likewise. I would recommend writing a count of elements first (eases read procedure)

Share this post


Link to post
Share on other sites
Whats a count? and is there a faster/easier way? I mean, if I write out every single weight then whats the advantage of the dat format over txt?
Thanks.

Share this post


Link to post
Share on other sites
there is no advantage. The dat format is just a file extension. you can write the same thing to a TXT file. you can even write al your data to a JPG file if you want. but that wouldn't make sense. I recommend making your own extension so you know what sort of data is in each file with a specific extension.

ex.
for your neuron data:

filename.neu

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this