Sign in to follow this  
Husbj

Any standard way of embedding external data files in a compiled executable / dynamic link library?

Recommended Posts

Husbj    658

Like the title suggests I'm wondering if there is any standard way to go about this?

I could of course write a really simple program that reads an external file and generates a source file that sets up a byte buffer corresponding to it, but is there any other ways about this? Maybe something less ugly?

 

The reason I need to do this is that I'm developing a dll file that needs some external data, but it wouldn't do to clutter the plugin folder of the program where it will eventually end up with additional files.

 

 

Thanks for any suggestions, 

Husbjörn

Share this post


Link to post
Share on other sites
Hodgman    51223


I could of course write a really simple program that reads an external file and generates a source file that sets up a byte buffer corresponding to it, but is there any other ways about this?
The engine I'm working with at the moment does exactly this --

e.g. foo.bar  -> used to generate foo.h

 
foo.h looks like:

0x03040000, 0x03100004, 0xbeeb03ff, 0x00000008,
0x7e100280, 0xf0041f00, 0x00010206, 0xc0c00300,
0x0002011b, 0x03040000, 0x03100004, 0x00000000,
0x5362724f, 0x07726468, 0x00003853,

Then in the code somewhere:

const uint32_t foo_bar[] = {
#include "binary_files/foo.h"
};

Share this post


Link to post
Share on other sites
Ryan_001    3475

I wrote a C# visual studio extension that takes a source file with a custom extension and creates an .obj/COFF file out of it that is linked in directly.

 

I should clarify, wasn't trying to boast.  Rather, if that was an option you wished to pursue, I can provide links and/or advice on how I approached it.

Edited by Ryan_001

Share this post


Link to post
Share on other sites
Hodgman    51223

I wrote a C# visual studio extension that takes a source file with a custom extension and creates an .obj/COFF file out of it that is linked in directly.

 

I should clarify, wasn't trying to boast.  Rather, if that was an option you wished to pursue, I can provide links and/or advice on how I approached it.

Is it open-source or available for others to use? biggrin.png

Share this post


Link to post
Share on other sites
Husbj    658

Thanks for the suggestions. Didn't know you could just store binary data in a resource; thought you would have to pack it into a string table or a bitmap in that case. That's good to know :)

 

 

 

But we don't know those for your data. We don't know how much data you have, creating a 600MB DLL is going to put a serious strain on load time. We also don't know if the data has a separate lifetime from the DLL, if you want other people to be able to modify it without rebuilding a DLL, or if the resources come from only one file or many files.

You're right, I should have described it a bit better. Let's blame it on me making that post just before going to bed.

Anyway, I'm simply storing some default shaders and a bitmap font image that will be used unless the user specifies his own. So it's just a couple of small files.

 

 

 

Rather, if that was an option you wished to pursue, I can provide links and/or advice on how I approached it.

I think I'm fine for now but it would still be interesting to look at sometime if you don't mind posting it.

Share this post


Link to post
Share on other sites
Ryan_001    3475

I uploaded what I have to here: https://sourceforge.net/projects/rfx/.  Its my first C# program so it may be a bit rough around the edges, but I think you should get the idea.  The COFF.cs file handles COFF exporting.  PassData.cs stores data to be saved and handles C++ name mangling, though its easy to remove if you want to use extern "C" style declarations.

 

The 'addon' compiles .fx files and instead of saving them to a .cso file, wraps them up into a .obj/COFF object file to be directly linked into the executable.  The rfx.targets/.xlm/.props files are used to actually 'link into' Visual Studio.  You'd want to copy these 3 files and the compiled .dll into the appropriate MSBuild build customization folder.  Then add it via the Build Customization menu much like you would MASM or similar customizations.

 

Anyways, let me know if you have any questions, feel free to use it for whatever...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this