• Advertisement
Sign in to follow this  

Storing data

This topic is 1987 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hello everybody,

Once again i am struggling with a question i hope to find some answers for.

In my project, i work with a lot of classes, mainly containing other classes, which contain mostly strings and integers. The size of my data is approximately as follows:

x events
in each event, approximately 8 markets
in each market, approximately 100 measures
in each measure, approximately 20 variables (integers and strings).

Since both events and markets also contain a few other variables, the approximate number of variables to store for each class is about 20.000. I intend to store up to a maximum of 1.000 events, therefore storing at maximum about 20 million values.

I wonder what would be the most efficient way to be able to store this data in one session and then load them back to use them in another session. The method i thought of myself was to create a XML file for each event and store the data in that file. Unfortunately i dont have any experience with data storing or using XML whatsoever, so i just wondered if this would be a reasonably effective way to store data. Any insights in this problem would really help me.

Since i am storing the data locally and mainly for myself, data access or protection is absolutely not an issue. Storing and reading speed is also not a really large issue, as long as it takes under a minute. The most important thing to consider is that i am a really unexperienced coder who is looking for a relatively simple solution. Also it should be a flexible method which allows me to add (for example) some measures to a market or remove some variables to a measure.

Thank you in advance for your ideas and opinions.

By the way, i work in C# using Visual Studio.

Share this post


Link to post
Share on other sites
Advertisement
A little more context would help. I mean, what kind of simulation are you trying to build? What are you actually trying to do?

Without more context, it's hard to give good advice, but generally: If you have a lot of data to store, and you want to store it "efficiently", you probably want to use a properly structured binary format, rather than a "readable" format like XML, which requires more memory to store, and more processing power to parse.

Share this post


Link to post
Share on other sites
Thank you very much for your reply. I am trying to build a simulation game to bet on sports events, like the website Betfair, therefore i try to capture the data from the website and store it to use in the game. The capturing of the data is not a problem, i already figured out how to do that, but now i need a way to store it properly.

I am not familiar with 'a structured binary format' but i will definitely google into that. If anybody has any other ideas or suggestions, i am happy to hear them.

Share this post


Link to post
Share on other sites
I would also recommend just writing to a binary file, binary basically just means you are writing the actual byte information to a file. So say you were storing the integer X with value 256 to a file. If you were writing it as text you can open it up with notepad and you will see 256 written, but if you are writing it as binary you would just see some funny looking symbols that represent the binary equivalent of 256.

Since integers are 4 bytes long, then your file will also be 4 bytes long regardless of how big the integer is.

Now if i stored it as text the number 4,000,000 will take up more space then the number 1.

Just google BinaryWriter there should be some good examples to get you going.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement