Sign in to follow this  
Halsafar

[java] Write/Read Binary Data of/to a class

Recommended Posts

Halsafar    205
Okay I found two examples of using binary IO. One was reading and the other was writing -- both only used ints. Now lets say I create a class called A and I want to write it to file. This is an easy task in C++, but I'm not sure how I can tell Java to write the info at the memory location of class (A) to the file. Likewise, if I can pull the above off, I can imagine that reading it back won't be so easy. Unless telling the DataInputStream to take n (bytes, size of A) and cast it into class A. Edit: I'd like to point out that the file being read from will be created by C++ and Java, so they need to work together on this one. I know already from some reading that Java is BIG_ENDIAN and C++ is LITTLE_ENDIAN. But Java can read in bytes in the reverse order by changing a bytebuffer flag. I am not sure if it can write in the reverse order as well. Unless there is a way to write a C++ DLL to do both the read and write. I thought about it but then got lost in the cross-language concept. How would C++ read it and leave Java with all the data stored nicely. [Edited by - Halsafar on November 29, 2005 4:22:11 PM]

Share this post


Link to post
Share on other sites
aidan_walsh    739
Quote:
Original post by Halsafar
Now lets say I create a class called A and I want to write it to file.

ObjectOutputStream?

Quote:
I know already from some reading that Java is BIG_ENDIAN and C++ is LITTLE_ENDIAN.
But Java can read in bytes in the reverse order by changing a bytebuffer flag. I am not sure if it can write in the reverse order as well.


Read the Inputstream into a ByteBuffer, and change the endian encoding using ByteBuffer.order(ByteBuffer.BIG_ENDIAN || ByteBuffer.LITTLE_ENDIAN).

Quote:
How would C++ read it and leave Java with all the data stored nicely.

Your dll file could read the file, and pass it upwards through a JNI interface. But as you can see above, Java has a built in alternative.

Share this post


Link to post
Share on other sites
CaptainJester    523
You should use ByteBuffer and FileChannel from the java.nio.* and java.nio.channels.* packages. That way you can set the ByteOrder on the ByteBuffer then read directly into the ByteBuffer using the FileChannel. ByteBuffer also has primitive gettors attached which will make getting your info easier.

Share this post


Link to post
Share on other sites
Halsafar    205
If I use ObejctOutputStream to write data to file.
I have a good feeling I won't be able to read it back in with C++.
After reading the API doc you posted, it sounds like Java adds a lot of Java specific stuff to the file.

ImageInputStream, ImageOutput stream sound like they may work. But I'm not sure...

Java and C++ both need to be able to read/write.

Share this post


Link to post
Share on other sites
Son of Cain    480
Work at the binary level - ByteArrayOutputStream and ByteArrayInputStream, caring for the ByteOrder when reading/writting.

I don't know the data you're storing, but depending on what it is, XML would be a nice communication layer for this system.

Son Of Cain

[Edit: Use ByteBuffer and ByteOrder when controlling the endianness of the array, as described by aidan_walsh's post]

Share this post


Link to post
Share on other sites
Halsafar    205
Its a map file format for the game in design.
The map editor is being made in Java, since it also needs to be cross-platform and requires a heavy GUI, we decided Java would simply be the easiest choice for a cross-platform widget system.

Thanks for the help everyone.
I'll give the ByteArrayOutput/InputStream a shot. I'll post if I run into any more problems.

Share this post


Link to post
Share on other sites
Halsafar    205
Okay I have no idea how to convert an Object of any type to a byte array which I can use to store in a ByteArrayOutputStream, switch the endian, and save to file.

Honestly I cannot figure out to use these things properly. I'm reading the API's for them but they provide no sample.

Once I do get an Object into a ByteArrayOutputStream, how to I write the bytes to a file?

Share this post


Link to post
Share on other sites
tombr    148
You can not get a pointer to a Java object like you do in C++. Besides, is it fool proof even in C++ to read write classes/structs. Will there not be alginment and layout issues on different computers.

What you have to do is write all the member variables of your java object using a DataOutputStream. You can only write primitives so any objects in the data structure need to write it self to the stream. Then you've got to parse this file back in c++. Creating the c++ object and filling in its member variables.

Seems to me your a bit naive about how hease this would be :) (Or I'm clueless and don't know what I'm talking about, wich might be true)

Share this post


Link to post
Share on other sites
Son of Cain    480
The way I see it...


public interface BinaryPersistent {

byte[] toBytes();

void read(byte[] data);

}

public class MyBinaryClass implements BinaryPersistent {

public byte[] toBytes() {
ByteBuffer buf = ByteBuffer.allocate(1024);
buf.setShort(0);
buf.setXXX(x); // use setters to write data
// order buffer
return buf.array();
}

public void read(ByteBuffer buffer) {
// get data using getXXX() methods on the buffer;
}

}




The only way to send an object to a file is through Object Input/Output streams, but you won't be able to recover this information on the C++ side (obviously... but I'm not sure if JNI could handle it?)

Apart from that, I suggest you use a model like the one above... To work in both Java and C++ at binary level, in such a manner that both applications output the same binary content.

Son Of Cain

Share this post


Link to post
Share on other sites
Halsafar    205
So with your method above?

I only read/write primitives.

This model can work between C++ and Java?

Edit: Now that I can see how I can impliment each objects toByte. How should I go about writing the byte array to the file?

Okay I figured out how to write to file I believe using FileChannel.


[Edited by - Halsafar on December 1, 2005 9:50:23 PM]

Share this post


Link to post
Share on other sites
Son of Cain    480
Yes, a FileChannel is even better than ByteArray IO streams because you'll be able to work directly with ByteBuffers, changing that interface to something like:


public interface BinaryPersistent {

/**
* Writes the class members into a ByteBuffer instance
* and orders the buffer for proper endianness
*/

ByteBuffer toBytes();

/**
* Reads the class members from the given ByteBuffer instance
* Requires an ordering of endianness before attempting to read
*/

void read(ByteBuffer data);

}




Hope to have helped,
Son Of Cain

Share this post


Link to post
Share on other sites
Halsafar    205
Huge help yes. Rating++.
That is how I had to change the interface to get it to write.

Now, can I read in from file using FileChannel as well right?
If so, I'm sure I can figure it out.

Share this post


Link to post
Share on other sites
Son of Cain    480
Quote:
Original post by Halsafar
Now, can I read in from file using FileChannel as well right?
If so, I'm sure I can figure it out.


Yes, you can. And you'll get a ByteBuffer from it.

I suggest you read a bit of the docs for the API, because depending on the size of your files, you'll want to benefit from some features like MappedByteBuffers. This link has several code tips for NIO.

Son Of Cain

Share this post


Link to post
Share on other sites
Halsafar    205
Okay I believe you've left me with enough information to accomplish the task.
Particularly that site you gave in your last post.

However, all the examples I'm looking at, including the read/write to bytebuffer via FileChannel examples all allocate the ByteBuffer to be 1024 bytes, enough for 256x4 byte variables. Is there a reason for this? Or could I very well allocate however much space I know I'll need (powers of 2)?

Share this post


Link to post
Share on other sites
Halsafar    205
Okay I've almost got it.
Except the bytes are coming out proper.

I'll post the code, the expected output, the real output:
Nice to note, the code is a bit messy, cuz I'm messin around with trying to get this to work.



import java.nio.*;
import java.nio.channels.*;
import java.io.*;

public class test
{
public interface BinaryPersistent {

abstract public ByteBuffer toBytes();

abstract void read(ByteBuffer data);

}

private static class A implements BinaryPersistent
{
int i = 10;

public A()
{
System.out.println("Hello from A");
}

public ByteBuffer toBytes() {
//Fill some info
ByteBuffer buf = ByteBuffer.allocate(1024);
buf.putInt(50); // use setters to write data
buf.putInt(50); // use setters to write data
buf.putInt(50); // use setters to write data
buf.putInt(50); // use setters to write data
buf.putInt(50); // use setters to write data
buf.putInt(50); // use setters to write data
buf.putInt(50); // use setters to write data

buf.rewind();
// order buffer

System.out.println("Converted A to Bytes");
return buf;
}

public void read(ByteBuffer buffer) {
// get data using getXXX() methods on the buffer;
}
}

private static final class B extends A
{
int j = 20;
public B()
{
System.out.println("Hello from B");
}
}

public static void main(String[] Args)
{
B b1 = new B();
File file = new File("filename");
ByteBuffer bbuf = b1.toBytes();
boolean append = false;

try
{
// Create a writable file channel
FileChannel wChannel = new FileOutputStream(file, append).getChannel();

// Write the ByteBuffer contents; the bytes between the ByteBuffer's
// position and the limit is written to the file
wChannel.write(bbuf);

// Close the file
wChannel.close();
}
catch (IOException e)
{
}

try
{
// Obtain a channel
ReadableByteChannel channel = new FileInputStream("filename").getChannel();

// Create a direct ByteBuffer; see also e158 Creating a ByteBuffer
ByteBuffer buf = ByteBuffer.allocateDirect(10);

int numRead = 0;
while (numRead >= 0) {
// read() places read bytes at the buffer's position so the
// position should always be properly set before calling read()
// This method sets the position to 0
buf.rewind();

// Read bytes from the channel
numRead = channel.read(buf);

// The read() method also moves the position so in order to
// read the new bytes, the buffer's position must be set back to 0
buf.rewind();

// Read bytes from ByteBuffer; see also
// e159 Getting Bytes from a ByteBuffer
for (int i=0; i < numRead / 4; i++) {
System.out.println(buf.getInt());
//byte b = buf.get();
}
}


}
catch (Exception e)
{
}
}
}





Expected output:
50
50
50
50
50
50
50
0
0
0
0
... (0's for the rest of the btyes)

Actual Output:
50
50
3276800
3276800
50
50
0
0
0
0
.... (the rest is 0's)






Edit: Okay, I shoulda looked over it more. The bytebuffer was only being allocated to 10. So I'm sure I was just overlapping the bytes or something...

[Edited by - Halsafar on December 2, 2005 3:03:34 PM]

Share this post


Link to post
Share on other sites
Halsafar    205
Yes, that was my next step of business.

Okay so I able to write an int with java and read it with C++. That works fine.

Now my problem is, Java cannot read it back.
I am telling the ByteBuffer in the read part of my snippet to use BIG_ENDIAN. However, thats not working. I believe the ReadableByteChannel gets the bits as they are (LITTLE_ENDIAN) and then it doesn't matter what order the byte buffer is in...

I am always getting the hex opposite of 50, so obviously its coming out of the file in the wrong order.
0x00000032
vs
0x32000000


As soon as I allocate the bytebuffer I call the order() method.
Maybe you can point something out?

I figure a solution would be to try and read the bytes in backwards somehow...

Share this post


Link to post
Share on other sites
Son of Cain    480
Quote:
Original post by Halsafar
As soon as I allocate the bytebuffer I call the order() method.
Maybe you can point something out?


Are you ordering the ByteBuffer instance AFTER you read data into it? That's how it must be done. If that is not working, then I have no idea what it might be... weird.

If all else fails... is there a way to change the endianness of a binary file in C++? (I'm sure there must be something)

Son Of Cain

Share this post


Link to post
Share on other sites
Halsafar    205
Actually. LoL
I just figured it out.

See for Java to read/write a binary which C++ can read/write:

- write as little endian
- read as little endian


I was telling java to write as LITTLE read it as BIG. I have to tell Java to read/write as LITTLE, since the file is in LITTLE.

All fixed. All working.
Thanks a ton for your help.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this