Animatian data: disk versus memory

Started by
6 comments, last by cozzie 10 years, 2 months ago
Hi,

Say I have a ascii file holding around 20.000 lines with each 7 float values (camera animation in this case).
At this moment at startup I load them all (20.000) from the file in a vector of the right struct's for my animation. So when I need the information for the next frame, I retrieve it from the vector.

Now I was wondering, when will I ever need data from older frames or frames later then the current frame..
So I could also just open the file within my animation class, and everytime I need the next frame, just read it out of the file directly.

What do you think the advantages/ disadvantages would be for both approaches?

With todays common hardware my assumption is that saving say 7x 50.000 floats in memory should be a big problem. On the other hand, currently I don't have situations where I need other data then the current frame. Another sort of hybrid solution could be to load the next 100 or 1000 frames when 99 or 999 frames have past.

Crealysm game & engine development: http://www.crealysm.com

Looking for a passionate, disciplined and structured producer? PM me

Advertisement

7x 50,000 floats is only 1.4 MB. That might become a problem only if you'll have thousands of such animations.

Reading next frame from disk all the time is one huge disadvantage: disk IO is very slow, plus it puts extra wear on it.

What is the use case the OP has in mind? I assume a runtime of a game (i.e. not a tool, test, scripting, or similar; otherwise give more information).

1.) Using an ASCII file is not the wisest choice. Game data should be stored in binary files.

2.) 20.000 frames at 60 frames per second means an animation duration of 5.5 minutes. This is for sure also enough time to let some disk I/O operations happen without destroying the overall performance. Of course, reading the data in chunk is still meaningful. I don't assume the necessity for random access, of course.

3.) On the other hand: 5.5 minutes … what should that be, a game or a movie? Is it a really expected number?

4.) When baking the binary data, regression can be used to find smooth sections, so that key framing and interpolation are possible to reduce the amount of data.

5.) 7 times 50.000 floats, each float 4 bytes long, cover 1.4 million bytes. This is hardly a problem "with todays common hardware" (assuming that we're speaking of gamer desktop systems; if you mean something other you have to specify) by itself. Of course, you told us nothing about any other memory burden, so you again may give more informations.

Thanks both. A little more background;

- the assumption that 5.5 minutes will 'never' be reached is fully true

- it are basically camera movements/ rotations per frame for a ingame cutscene or a tech demo for example

(where 2 or 3 minutes already might be experienced as long :))

For now I'll keep going for loading the full animation in the vector of frames (having the 7 floats).

When I have more animations I'll load the next one on the fly, replacing the one before.

Up till now I've never went into binary saving/ loading (although probably no rocket science).

I'm using basic ofstream and ifstream operations now.

Crealysm game & engine development: http://www.crealysm.com

Looking for a passionate, disciplined and structured producer? PM me

Usually its best to treat disk io as the scary monster in the room, if anything its preferable to load as much as you feasibly can and only decide otherwise if it causes a performance problem or is unfeasible(i.e. streaming giant music tracks and things like that.)

But realistically you're still talking small scale here, loading 2 MB of anything doesn't take very long even with the evil of io, its when people do crap like reload 10 gb worth of assets in the middle of a game map where problems start to occur.

Usually its best to treat disk io as the scary monster in the room

Um, I've made a couple of games on Android and RAM consumption is constantly a problem. The solution is SQLite3, ie, querying a database file on disk/storing my game data on disk. Granted I've never used SQLite3 for querying individual animation frames & dont think I would ever use it for that purpose, but storing data on disk rather than in memory has been essential for my Smart Phone games.

For example; I have a dynamic 3d terrain made up of Google Static satellite images. These images use huge amounts of RAM and when the game camera is fully zoomed into a section of the terrain I actually delete areas of the terrain that are off camera (removing the images) and get some nice efficiency. And I store those images in a SQLite3 database. I do this so upon zooming out I re-generate those terrain tiles and grab the images from the SQLite3 db and I also do this for caching, ie, the next time the game loads it doesn't need to re-download those images.

Just sayin that, Disk IO isn't a scary monster but a life saver on Smart Phones or any device with small RAM (<600mb).

But realistically you're still talking small scale here, loading 2 MB of anything doesn't take very long even with the evil of io

Also 2mb for 1 animation in a Smart Phone is crazy especially in RAM, thats when I throw it to disk, ie, SQLite3. You have to consider on Smart Phones if the device has a total 600mb RAM that in no way means your app/game will ever be allowed to use anywhere near 600mb the OS will generally kill you around 150-200mb (but this differs greatly across devices and OS's).

I guess it all depends on the platform you are developing on; on Desktop you can afford to use RAM alot more. But on Smart Devices (especially not considering tablets) using the disk can greatly allow you to achieve efficiency and be a life saver.

Would be very interested to get others opinions on this, especially from the 'Platform Viewpoint'.


- it are basically camera movements/ rotations per frame for a ingame cutscene or a tech demo for example

Wait... if this is just for camera movement, do you really need that much data? Cameras tend to move smoothly from point to point. You should be able to only store information for certain keyframes, and then lerp between them. 1.4MB of data for 5 minutes of camera movements seems excessive to me.

Hi Phil_T.

The data is now 1.014KB :)

You're right, I should be able to save headings etc., but I'm not sure how I should handle skipping frames then. Although, maybe I do, but it seems more difficult that way. Checking how much frames have skipped with that heading and then going to the next etc.

Now I simply check the passed time and select the rigt frame, have an absolute position and rotation of the camera.

Crealysm game & engine development: http://www.crealysm.com

Looking for a passionate, disciplined and structured producer? PM me

This topic is closed to new replies.

Advertisement