Database engine

Started by
10 comments, last by LorenzoGatti 14 years, 2 months ago
For a lot of standard searching tasks, particularly if you are only searching items by unique identifiers (such as ID's) then you can get pretty far, very quickly, by utilising the standard storage containers that your language provides. This is especially useful if you are able to load all your data into memory. I see that python supports dictionaries - these are ideal for this kind of operation.

This way you can do a variety of things cheaply in memory. The use case you mention, detecting when a team breaks a losing streak after 4-5 games, I would grab all games the team has played and then sort them chronologically (trivial in python). Then I would iterate and find losing streaks. (again trivial).

Given a large dataset which cannot be loaded into memory, this would not be performant, but for datasets which could be loaded completely into memory it would suffice.
Don't thank me, thank the moon's gravitation pull! Post in My Journal and help me to not procrastinate!
Advertisement
Standard questions:
- What data are you going to put in the database? How does it change?
- Will you have enough data to worry about performance?
- Why in a database and not just loaded at startup and kept in memory in plain data structures?
- What other applications use the data? For example, level editors or automated tests.
- Considering the functionality of your application, is it better to solve the object-relational mismatch by compromising the objects (possibly advanced SQL, open to different and generic client applications, very simple data structures) or by compromising the relational tables (natural object structures, possibly lower performance, very opaque)?

Regarding Zope, it is a heavyweight and unusually sophisticated web CMS (its objects are meant to be site content). Maybe you mean ZODB? I suggest you outgrow the shelve and pickle modules in the standard library before evaluating this and other possible replacements.

For a RDBMS, I am partial to Oracle; MySQL might be also good in the same category (and of course not ridiculously expensive), but if you don't need a remotely accessed DB server SQLite is easier to use at a modest performance cost.

Omae Wa Mou Shindeiru

This topic is closed to new replies.

Advertisement