Jump to content
  • Advertisement
Sign in to follow this  
gamervb

Loading text files

This topic is 914 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

I store 3D model information in plain text files. When the game starts the program uses fstream to read in the information and send them to vertex buffers. What is the standard way to do this process in the industry? Is it converting these text files into binary format and using memory map?

Share this post


Link to post
Share on other sites
Advertisement

convert to binary format - yes; smaller on disk and faster to parse
use memory mapped files - no, usually they read the data from the file and build meshes and/or create texture from its data, then close the file. There's no reason you couldn't use memory mapping, but I don't think it's practical because you shouldn't need its data for long (this made more sense in the days of software rendering)

Share this post


Link to post
Share on other sites

Memory mapping is meant for huge files that would be inconvenient to keep in RAM... the exact opposite of pretty much every 3D model in a game, which needs to fit in video memory for starters =P So no, just parse the file, store the values in memory and close the file as usual.

 

As for formats, it's pretty much up to you what you find easier. You can try looking at Assimp to have it take care of 3D models for you instead.

Share this post


Link to post
Share on other sites

Parsing should _not_ be required. Your engine's format should be as close to the in-memory representation as possible. Ideally, you get to the point where all internal references in the data are offset in their packed data. You literally just have to copy the contents into memory. Something like AssImp is useful for your content pipeline for conversions to your engine's native format but should not be used by the engine to directly load its assets.

When loading, you then have two options: read() the bytes into a memory location or mmap() the file and then memcpy() the bytes into a better location. Depending on the OS, the mmap+memcpy may be faster as it will involve fewer copies of the data being made. Whether it's so much faster as to be worth it is another question. I wouldn't bother and would just go for a read(), but it may be worth profiling and finding out what works best for your engine on your target platforms.

 

Either way, you want to pack all your assets into a single archive file. In that case, you can just keep a single handle (a mmap'd pointer or a file stream handle) open to that archive then copy/read assets out of it as required. A table packed into the archive and read into memory on load lets you map resource IDs to byte offsets and lengths in the archive. For loose assets, you can generate this table at runtime by scanning the loose asset folder, or just load on demand if your resource IDs are actually file paths (which has both some strong points and some weak points in itself).

Share this post


Link to post
Share on other sites

What is the standard way to do this process in the industry? Is it converting these text files into binary format and using memory map?

 

Noting that this is in For Beginners, what a beginner does is typically very different from what the industry does.

 

 

The industry, as mentioned above, will use a content pipeline that does all kinds of wonderful thing.  It manipulates all the data into a preferred format and bundles it up for fast load time. Generally there are content folders that are monitored at runtime for changes, and when an asset changes the resources are reloaded and replaced dynamically which aids in turnaround time. Generally there is at least one dedicated tools team that spends all their work days building useful things for the project to make everybody's lives easier.

 

The beginner generally does not. Usually a beginner does not start 3D graphics until they've learned the basics of programming, although some will jump in without that knowledge and face a steep learning curve. The beginner will usually hardcode simple models, cubes and squares until they either give up in frustration or figure out what is going on. Then the beginner will generally turn to an existing tool, such as Assimp or JMonkey or SDL or libGDX or similar, or use an engine like Unity or Unreal, and let the tools do the work.

 

Both Unity and Unreal have the benefit of being both moderately friendly to beginners and also professional-grade tools.  They are able to do much of the complex professional-grade neat things and let you focus on your actual game.

Edited by frob

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!