9 replies to this topic

[juddo]

hello,

i have spent an hour or so googleing how to do the following but didnt find anything that covered this specifically. (may have been how i phrased my words, not sure?)

i want to read (c++ - fread()) a binary file from a game that contains floating point vertex data stored in two byte signed integer format (x,y,z,w - w is always 0). then i want to convert that into floats that i can use with opengl to draw the vertex data. apparently i also need to divide this by the radix point (which is some multiple of two?)?


i would appreciate any help as i've hit a wall here...

thanks for your time,

best regards,


justin

Edited by juddo, 29 October 2012 - 11:02 PM.

[Hodgman]

data stored in two byte signed integer format (two byte float)

That's two different things -- is it two-byte signed integers, or two-byte floats?

[juddo]

hodgman, the data is stored as two byte signed integers. thanks for the clarification.

i have included some code i've been toying with all day. it's giving me normal looking floats, but im almost certain it's not correct (very similar numbers.)

[source lang="cpp"]signed int byte0;signed int byte1;float temp;fread(&byte0, 1, 1, f);fread(&byte1, 1, 1, f);temp = ((byte0 & 0xFF) / 256.f) + byte1;[/source]

i've got a script (quickbms script, but i want to do this in c++) that deals with this issue (converting two byte signed int to float) and by looking at it, suggests i am doing this wrong. this is the scripts pseudo code.

[source lang="plain"]byte0byte1float tempif byte1 > 127byte1 -= 256temp = byte1temp *= 256temp += byte0[/source]

i tried the above in a small console program i wrote just for this and get weird results.

[ultramailman]

fread(&byte0, 1, 1, f);
fread(&byte1, 1, 1, f);

It seems you want to read the two bytes seperately. Why not use signed char? chars are one byte.

If you want to store floats in binary, why not just do this?

float dat = 3.14;
float dat2;
FILE * fh = fopen("file", "wb");
fwrite(&dat, sizeof(float), 1, fh);
fclose(fh);
fh = fopen("file", "rb");
fread(&dat2, sizeof(float), 1, fh);
printf("%f == %f\n", dat, dat2);

This will write and read sizeof(float) bytes.

Edited by ultramailman, 30 October 2012 - 07:34 PM.

[juddo]

thanks for the reply ultramailman.

i am only trying to read the binary data, not store it. i am trying to port a game reading original game content (ps2 game files.)

i was reading the two bytes seperately because i believe they need to be the other way around for pc.

a script i have that converts the data into a 3ds file, looks like this;

[source lang="plain"]<loop one - read the data (x,y,z,w)>Get type1 Short 0 ;Get type2 Short 0 ;Get type3 Short 0 ;Get type4 Short 0 ;Put type1 Short MEMORY_FILE1 ;Put type2 Short MEMORY_FILE1 ;Put type3 Short MEMORY_FILE1 ;<loop two - convert data to floats>Get first Byte MEMORY_FILE1 ;Get second Byte MEMORY_FILE1 ;If second > 127 ;Math second -= 256 ;Endif ;Math nb = second ;Math nb *= 256 ;Math nb += first ;Put nb float MEMORY_FILE2 ;[/source]

Edited by juddo, 31 October 2012 - 05:54 PM.

[ultramailman]

i am only trying to read the binary data, not store it. i am trying to port a game reading original game content (ps2 game files.)

AH I see, thanks for clarifying. So you have to follow the format's specification then. What does each of those two bytes represent?

[juddo]

i hope this answers your question.

documentation on this file type says "models have vertexes in two byte signed integer format. to convert to float you convert the integer into a float, but you need to divide it by the radix point (some multiple of two - forgot which?)"

in my hex editor, i found the starting point of the vertex data which i have pasted below.

x, y, z, w (always 0?)
A9F3 1101 0EFD 0000

thanks for your time.

[Hodgman]

With your code, you're only reading a single byte into byte0/byte1, but they're uninitialized, so their upper 3 bytes contain random values -- you should initialize those ints to 0 before using fread.
Technically this isn't required for byte0 as you're masking it with 0xFF, but you're not masking byte1, so your results are undefined. Nonetheless, it's best practice to always ensure your variables are initialized properly.

i was reading the two bytes seperately because i believe they need to be the other way around for pc.

This is known as an endian conversion. Regular x86 PC CPUs are "little endian", and I'm not sure about the PS2's GPU, but from what you've said, I assume it must be "big endian".
When you're saving data for a "big endian" processor, it expects that the most-significant-byte appears first, and the lest-significant-byte appears last.
If you're reading a two-byte integer a byte at a time, the two methods would look like:

int readingBigEndianData = ((byte0 & 0xFF)<<8) + (byte1 & 0xFF);
int readingLittleEndianData = (byte0 & 0xFF) + ((byte1 & 0xFF)<<8);

Once you've decoded the integer, you can then convert it to a float by dividing by your radix point

float value = readingBigEndianData / 256.0f;

N.B. if your radix point is exactly 256.0f and your data file is big-endian, then this is the same as

float value = (byte1 & 0xFF)/256.0f + (byte0 & 0xFF);

because "<<8" is the opposite of ">>8", and ">>8" is coincidentally the same as "/256".

[juddo]

thanks for that fantastic reply hodgman. i really couldnt ask for more help!

[source lang="plain"]With your code, you're only reading a single byte into byte0/byte1, but they're uninitialized, so their upper 3 bytes contain random values -- you should initialize those ints to 0 before using fread.[/source]

as soon as i read this, the penny dropped. yesterday i spent almost all day on this and recalled getting completely random results every time i ran the program with a particular line of code i was using.

i went back to how i was trying to convert the integers and set byte0 and byte1 to 0 and wouldnt you know it! it worked!

thank you so much for all the help.

best regards,


justin

Edited by juddo, 01 November 2012 - 04:20 AM.

[juddo]

i am trying to read an unknown game file format (ps2 model file). all i know is it's a modified renderware dff file (which has elements of the gta sa dff format). this
particular game uses triangle strips to draw each model. there is duplicate/degenerate vertices present in every model file.

as i understand, the ps2 had serious memory limits. it appears the programmers decided there was significant gains to be had from changing the storage of floating
point data. for example, instead of storing a vertex float in four bytes, they stored it in two bytes. or normal floats in four bytes, they stored it in one byte! my
problem is converting this data to how it is meant to be read. originally i thought there was a fixed scale variable it had to be multiplied by... but now i am just
not sure at all. the total number of vertices for a model file i am looking at currently is 321. coincidently, theres also four different chunks of data that follows
this vertex data i believe is uv, colour vertices and normals. each chunk that follows the vertices chunk is 321 * 4. i believe uv's are stored as two byte floats,
colour vertices are stored as one byte unsigned chars and normals are stored as one byte floats?

eg;
- vertices; 321 * 8 (two byte float * 4) x, y, z, 00 - padding?
- uv's; 321 * 4 (two byte float * two?) u, v
- vertex colours; 321 * 4 (one byte unsigned char * four?) r, g, b, a
- normals; 321 * 4 (one byte * four?) x, y, z, 00 - padding?

i wrote a small console application to parse the manualy extracted binary data and dump it into floats but get undesirable values...

[source lang="cpp"]int n_vertices;signed short data;float (*normals)[3];float normal_scale = 1 / 126.076126092;fread(&n_vertices, 4, 1, f);normals = new float[n_vertices][3];printf("parsing normals...\n");for (int loop_loop = 0; loop_loop < n_vertices; loop_loop++) { for (int loop_iterator = 0; loop_iterator < 3; loop_iterator++) { fread(&data, 1, 1, f); normal = (float)data * normal_scale; normals[loop_loop][loop_iterator] = normal; } // skip the padding fseek(f, 1, SEEK_CUR);}[/source]
four bytes of binary data i presume is the normals looks like this (in hex);

7F000000

when i parse it and dump the floats in my program, i see;

-110.671234 -111.678558 -111.678558

i was originally dividing my vertex float values by 32768.0f. basically, i just dont know what i am doing. can someone shed light onto how a single or two byte float is
stored, then read correctly? im getting two very different values for one byte floats and two byte floats !

any help at all is appreciated.


thanks for your time.

best regards,

justin