Jump to content

  • Log In with Google      Sign In   
  • Create Account


Storing data


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
3 replies to this topic

#1 Jaap85   Members   -  Reputation: 241

Like
0Likes
Like

Posted 14 August 2012 - 02:35 PM

Hello everybody,

Once again i am struggling with a question i hope to find some answers for.

In my project, i work with a lot of classes, mainly containing other classes, which contain mostly strings and integers. The size of my data is approximately as follows:

x events
in each event, approximately 8 markets
in each market, approximately 100 measures
in each measure, approximately 20 variables (integers and strings).

Since both events and markets also contain a few other variables, the approximate number of variables to store for each class is about 20.000. I intend to store up to a maximum of 1.000 events, therefore storing at maximum about 20 million values.

I wonder what would be the most efficient way to be able to store this data in one session and then load them back to use them in another session. The method i thought of myself was to create a XML file for each event and store the data in that file. Unfortunately i dont have any experience with data storing or using XML whatsoever, so i just wondered if this would be a reasonably effective way to store data. Any insights in this problem would really help me.

Since i am storing the data locally and mainly for myself, data access or protection is absolutely not an issue. Storing and reading speed is also not a really large issue, as long as it takes under a minute. The most important thing to consider is that i am a really unexperienced coder who is looking for a relatively simple solution. Also it should be a flexible method which allows me to add (for example) some measures to a market or remove some variables to a measure.

Thank you in advance for your ideas and opinions.

By the way, i work in C# using Visual Studio.

Sponsor:

#2 Goran Milovanovic   Members   -  Reputation: 1104

Like
0Likes
Like

Posted 14 August 2012 - 06:31 PM

A little more context would help. I mean, what kind of simulation are you trying to build? What are you actually trying to do?

Without more context, it's hard to give good advice, but generally: If you have a lot of data to store, and you want to store it "efficiently", you probably want to use a properly structured binary format, rather than a "readable" format like XML, which requires more memory to store, and more processing power to parse.

+---------------------------------------------------------------------+

| Game Dev video tutorials  ->   http://www.youtube.com/goranmilovano |
+---------------------------------------------------------------------+

#3 Jaap85   Members   -  Reputation: 241

Like
0Likes
Like

Posted 15 August 2012 - 11:51 AM

Thank you very much for your reply. I am trying to build a simulation game to bet on sports events, like the website Betfair, therefore i try to capture the data from the website and store it to use in the game. The capturing of the data is not a problem, i already figured out how to do that, but now i need a way to store it properly.

I am not familiar with 'a structured binary format' but i will definitely google into that. If anybody has any other ideas or suggestions, i am happy to hear them.

#4 The_Neverending_Loop   Members   -  Reputation: 603

Like
0Likes
Like

Posted 15 August 2012 - 12:50 PM

I would also recommend just writing to a binary file, binary basically just means you are writing the actual byte information to a file. So say you were storing the integer X with value 256 to a file. If you were writing it as text you can open it up with notepad and you will see 256 written, but if you are writing it as binary you would just see some funny looking symbols that represent the binary equivalent of 256.

Since integers are 4 bytes long, then your file will also be 4 bytes long regardless of how big the integer is.

Now if i stored it as text the number 4,000,000 will take up more space then the number 1.

Just google BinaryWriter there should be some good examples to get you going.




Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS