Sign in to follow this  
Barguast

[.net] BinaryWriter / Streams Question

Recommended Posts

First of all, don't be frightened off by the length of this post, I think this is a relatively simple problem, but I'm just being as precise as possible :) OK, I've got a few questions regarding the BinaryWriter class and other stream-related things. There is a good chance I'm doing something really wrong, so feel free to point it out if I am! I'm writing a series of small projects from which I hope to learn more about C# and .NET in general. My current project centres around creating a Text Editor which let you save text in a specific format - optionally encrypted / compressed. The following (pseudo) code demonstrate the part that is causing me bother. The function basically writes the text in a specific format to the given stream.
public void WriteTo (Stream stream)
{
	BinaryWriter bw = new BinaryWriter (stream, Encoding.UTF8);

	// Write the length of the text	string
	bw.Write ((UInt32) this._plainText.Length);

	// If the text is to be encrypted then write to the stream
	// via a CryptoStream, otherwise just write straight to the
	// stream.
	Stream textStream;

	if (this.encrypted)
	{
		// Create & Setup a CryptoStream and make it piggyback
		// on the 'stream' argument
		textStream = new CryptoStream (stream, ...);
	}
	else
	{
		textStream = stream;
	}

	// ?
}


You'll notice that it ends with a question mark. At this point I want to write the text contained in the class (this._plainText) to the stream represented by textStream. The problem is that the BinaryWriter is 'attached' to the stream passed into the function and there seem to be no way to change it to textStream. I have tried simply creating another BinaryWriter and attaching 'textStream' to that, but I don't think the stream likes to have two BinaryWriters attached to it at once. Instead, I could close the old BinaryWriter and simply create a new one. However, doing so would also close the stream that was attached to it, and there would be no way to reopen it from this function. I hope that makes sense. As I said, if I'm doing something wrong here then please say so. Also, if you need to know more information then I'll be glad to post what you ask for. Thanks

Share this post


Link to post
Share on other sites
I probably could, yes! Rather than post my code (there is a lot of it), I'll try and go by example instead...

Lets say that if the first byte in a stream is equal to 0 then that indicates that the remainder of the stream is compressed. If the byte is non-zero then it is uncompressed.

private void ReadFrom (Stream input)
{
BinaryReader br = new BinaryReader (input);

Stream decompressor;
if (br.ReadByte () == 0)
decompressor = new HuffmanStream (input);
else
decompressor = input;

// How can I now tell the BinaryReader to read from the
// 'decompressor' stream rather than the 'input' stream
// that was originally assigned to it?
}


It's probably a crude example but I think it demonstrates my problem pretty well. (You don't need to worry about the compression stream or anything since it is only an example)

Anyway, the question is commented in the code. I now need to tell to the BinaryReader to read from the decompressor stream. I've tried to simply create a new BinaryReader but this won't work - presumable because 'stream' is already attached to 'br'. And I can't close the BinaryReader because that would close the stream.

One solution which did seem to work is to simply do:

private void ReadFrom (Stream input)
{
BinaryReader br = new BinaryReader (input);

Stream decompressor;
if (br.ReadByte () == 0)
decompressor = new HuffmanStream (input);
else
decompressor = input;

br = new BinaryReader (decompressor);
}


However, since I didn't close the original BinaryReader is doesn't look right to me...

I hope this made more sense!

Share this post


Link to post
Share on other sites
Quote:

// How can I now tell the BinaryReader to read from the
// 'decompressor' stream rather than the 'input' stream
// that was originally assigned to it?


Just change the order your do things in. You can read from the stream without wrapping it in a BinaryReader.


private void ReadFrom (Stream input){
Stream decompressor;
if (input.ReadByte () == 0)
decompressor = new HuffmanStream (input);
else
decompressor = input;
BinaryReader br = new BinaryReader (decompressor);
}



Or if your prefer it short:

BinaryReader br = new BinaryReader((input.ReadByte() != 0)? new HuffmanStream(input):input);


Share this post


Link to post
Share on other sites
Well, thanks for your reply but that was just an example - It's my fault, I tried to demonstrate my problem and ended up over-simplifying it.

My actual code is far more complex than reading a single byte from the stream, I have to read a few integers and strings before I get to the 'is it compressed or not?' part.

For this reason I still think the best approach is to alter the stream attached to the BinaryWriter - otherwise the code would become too cumbersome.

Can someone help me out with this? Again, if what I'm trying to do is stupid or just plain 'wrong' then please point it out!

Share this post


Link to post
Share on other sites
Ok, the point is that if you are doing something generic that choses between multiple methods of doing something you probably don't want to rely on any one of those methods until you know what method you are going to use. There are many, solutions to your problem as I see it. Here are a few:

1: use fixed size/dynamic header and read it off the stream before you asign it to a text/binary/whatever reader. (if the header contains binary data use a BinaryReader around a MemoryStream to read it)

2: don't use the readers at all and just use the stream.

3: get the stream out of your binary reader (see BaseStream property) and read from it before your use the reader.

4: assign multiple readers to the stream (probably won't work or will have unexpected results unless you are sure that no buffering is going on). (You mau be able to use seek methods to get around this)

5: read the entire stream into a buffer and than devide it up and parse it or convert it to streams.

6: Open the stream read the header close it and open it again.

7: Seperate the header info from the stream (on disk or where ever it is).

8: Use some sort of pre-existing format/parser (XML? Serialized Data?) and just forget about all of this stuff.

etc... etc... etc...

There it probably no reason to modify the BinaryReader (which you really cannot do -- though you could make a subclass if you really felt like wasting your time.)
Method 1 will work.
Method 2 could be best: if a tool does not work for your problem do not use it.

Share this post


Link to post
Share on other sites
Well, thanks for your reply. I'll look into using your 2nd method. However, I can't see how I could read integers without using a BinaryReader. A stream just allows you to read byte arrays, right? There is probably a way to convert these arrays to integers, but it seems a bit of a hack to me.

Of course, I'm by no means an expert. I'll look into it.

Thanks

Share this post


Link to post
Share on other sites
It is actually mostly trivial to encode/decode integers and numeric types because they are always the same size. An integer is 32 bits or 4 bytes long so to read an integer just read 4 bytes and then shift and add them like so:

//conceptual:
byte[] intData;//read it off the stream
int result = 0;
result += (intData[3] << 24);
result += (intData[2] << 16);
result += (intData[1] << 8);
result += intData[0];



Or the easy way in .NET:

int result = BitConverter.ToInt32(intData,0);




Share this post


Link to post
Share on other sites
Yeah, but doing the conversion myself (as I think turnpast recommended) seems a bit unnecessary when it is what the BinaryReader is for.

I guess, being a relative beginner, that I'm just a little confused as to what these classes actually 'are'.

As far as I am aware the BinaryWriter, BinaryReader (etc.) classes are just concerned with formatting. If you pass an integer in, they will convert it to a byte array and then pass that on to the BaseStream. Is that basically right?

In which case, why do these classes have Close methods? Since they are only data converters then why would you need a method to 'close' them? Or is the Close method just a convinient way of closing the underlying stream? In which case, is it really necessary to ever 'Close' a BinaryReader/BinaryWriter if I can just close the stream myself?

Quite a few questions there, but hopefully they will clear this up for me.

Thanks!

Share this post


Link to post
Share on other sites
Quote:
Original post by Barguast
Yeah, but doing the conversion myself (as I think turnpast recommended) seems a bit unnecessary when it is what the BinaryReader is for.

I guess, being a relative beginner, that I'm just a little confused as to what these classes actually 'are'.

As far as I am aware the BinaryWriter, BinaryReader (etc.) classes are just concerned with formatting. If you pass an integer in, they will convert it to a byte array and then pass that on to the BaseStream. Is that basically right?

In which case, why do these classes have Close methods? Since they are only data converters then why would you need a method to 'close' them? Or is the Close method just a convinient way of closing the underlying stream? In which case, is it really necessary to ever 'Close' a BinaryReader/BinaryWriter if I can just close the stream myself?

Quite a few questions there, but hopefully they will clear this up for me.

Thanks!


As you mentioned, as soon as you assign the BinaryReader to a stream, it's attached to that stream. As per your example, I don't understand why you're passing the initial stream to your WriteTo function if that's not always the stream that's going to be written to. Are you trying to write both unencrypted and encrypted data to the same file? You can't have more than one reader/writer attached to the same stream (ie file) at the same time if opened synchronously (which is the default). This is why the underlying stream is closed when you close the reader/writer. However, you can create an asynchronous filestream, pass it's handle to a second asynchronous filestream (your encrypted one) and have them both write to the same file. This could be dangerous though.

Filestream class

MSDN linky to filestream constructor

You'll need to set the useasync argument (boolean) to true to be able to have multiple streams writing to the same file. It will work on Win32, but not on older versions of Windows (Me, 98). You can only use synchronous streams on the older OS's.

Share this post


Link to post
Share on other sites
Guest Anonymous Poster
Quote:
Original post by Zul
As per your example, I don't understand why you're passing the initial stream to your WriteTo function if that's not always the stream that's going to be written to.

It is that stream that gets written to - but not always directly. As you rightly pointed out I am writing both unencrypted and encrypted (and, incidently, compressed / uncompressed) information to the stream, but this is done so by 'piggybacking' the streams on top of the stream passed into the function.

For example:

File Stream (the one passed into the function)
|_ Encryption Stream
|_ Compression Stream

or

CryptoStream cs = new CryptoStream (stream, ...);
AdaptiveHuffmanStream ahs = new AdaptiveHuffmanStream (cs, ...);


As I understand it, by writing to the compression stream this will cause the data to be compressed, then encrypted, and finally written to the file. This works in a few 'test' projects I have done.

One thing that is puzzling me here is the assumption that if I am finished with the BinaryWriter (or Reader) then I must also be finished with the stream.

This doesn't seem to apply to all situations - what if I wanted to write text to the same stream with different encodings? Given what you've said about being unable to have two Writers attached to the same stream, then it seems that closing and reopening the stream seems unavoidable.

Is there not some way of unattaching the BinaryWriter from a stream without this stream being closed?

PS. Also, thanks for the links, and thanks to everyone else for their help so far.

Share this post


Link to post
Share on other sites
Sorry, my last post was a little unclear. If you use the constructor I linked to for filestream, you can make two asynchronous streams pointing to the same file, meaning both are able to write to the stream and you don't have to close them. I don't know how easy it will be to keep track of the file position - you may need to keep track of that with a modular variable. Try doing that with one stream for unencrypted and one stream for encrypted data and see how it goes.

Share this post


Link to post
Share on other sites
Hmmm, I'm not sure if I've explained my problem well enough - or even if there is a problem to answer here... I really hope I'm not wasting everyone's time with a problem that isn't so problematic :)

I'm not entirely sure that an async stream is what is required here - for a start if can't be guaranteed that the stream passed into the function will be either a FileStream or created Asynchroniously. I could check of course, but I'm still not entirely convinced that this is the ideal solution...

The main reason is that I don't actually require two BinaryWriters (or Readers) attached to a stream at the same time - I just need to tell the BinaryWriter to write to a stream other than the one originally assigned to it. If the BinaryWriter is just a glorified data formatter (which, after looking through the source code, it seems to be) then I wouldn't have thought that this was much of a problem - but I don't know how to do it.

For the time being (before I fully understand the purpose, meaning, and workings of these classes) I've been using the following method, but I'm not sure if it is safe or 'proper'

public void WriteTo (Stream stream)
{
BinaryWriter bw = new BinaryWriter (stream);

// Write some (uncompressed) data from the stream using the
// BinaryWriter 'bw'

// Now comes the point where either compressed or
// uncompressed text is written, depending on the value
// of the 'compressed' flag
Stream textStream;

if (compressed)
textStream = new AdaptiveHuffmanStream (stream);
else
textStream = stream;

// Reassign the BinaryWriter
bw = new BinaryWriter (textStream);

// Write the text to the newly assigned BinaryWriter 'bw'
}



As you can see, textStream can either be the same stream as the original, or it can be a 'compression layer' which itself writes to the original stream.

However, it is the 'Reassign the BinaryWriter' part which I am unsure of - I am simply replacing one BinaryWriter with another. Is that safe?

I hope this has shed some more light on my problem. Thanks again.

Share this post


Link to post
Share on other sites

public void WriteTo (Stream stream)
{
// Now comes the point where either compressed or
// uncompressed text is written, depending on the value
// of the 'compressed' flag
Stream textStream;

if (compressed)
textStream = new AdaptiveHuffmanStream (stream);
else
textStream = stream;

BinaryWriter bw = new BinaryWriter (textStream);
// Write the text to the newly created BinaryWriter 'bw'
}






To be a bit more specific, why do you assign a binary reader to the initial stream? As far as I can tell, since you are writing, there is no reason to. If on the other hand you do have a reason (something funky or stupid), then you need to keep the original reference alive, since you don't know when the class will be disposed of, and hence Close called on the base stream.

Share this post


Link to post
Share on other sites
So, if I understand the problem, you want to read and write data to a file that may be compressed, encrypted or somehow otherwise encoded. You probably want to know by looking at an existing file if and how it is encoded.

One big question is: Does a file have one encoding from beginning to end or can there be multiple blocks with different encodings in a file?

Writing is not really a problem, just decide what you are going to write and then write it. If you need to use different Writers from the class library just open the in sequence on the same strream or write them individually to memory (see: MemoryStream) and then push the data into the file stream (someStream.Write(memoryStream.GeyBuffer(),memoryStream.Length);).

Reading is more tricky because you need to be able to tell from looking at the file (or chunk of the file) how it is encoded. To do this use a header at the beginning of the chunk that contains information about how the file is encoded, like a set of flags or a string whatever. Read this header first and use the information it contains to decide how to parse the rest of the file/chunk.

You could use a BinaryWriter/Reader to parse the header, but to do that you will probably have to buffer it in memory to not interfere with the orignal stream.


public struct HeaderInfo {
public int length;
public int encoding;
}
int headerLength = 8; //4 bytes per int, 2 ints.
HeaderInfo readHeader(Stream s, int headerLength){
byte[] buffer = new byte[headerLength];
s.Read(buffer,0,headerLength);
BinaryReader br = new BinaryReader(new MemoryStream(buffer));
HeaderInfo hi = new HeaderInfo();
hi.length = br.ReadInt32();
hi.encoding = br.ReadInt32();
br.close();// will not touch the orignal stream
return hi;
}
//do the reverse to write the header.



There are no doubt more elegant ways, but this is simple and addresses your problem as I understand it.

If your file contains multiple blocks with different encodings you will need a header for each block. (this is why length is part of the header).

hope this helps.

Share this post


Link to post
Share on other sites
Ahhh, I think I see now! :)

I think I'll be able to get this done now, thanks.

EDIT:
Quote:
One big question is: Does a file have one encoding from beginning to end or can there be multiple blocks with different encodings in a file?

Well, since you ask the file has three sections.

- The version information (contains data which identifies the format and which version of the format the rest of the file is in)

- The text format information (indicates whether the following part of the file is encrypted/compressed/both/neither)

- The text

As you can see, you only know the format of the third section after reading the second section - and this it the part that was causing problems since I would potentially have to divert the 'reader' stream through decryption and decompression streams.


[Edited by - Barguast on September 24, 2004 9:50:41 AM]

Share this post


Link to post
Share on other sites
Just one last question...

Why is it that the Dispose method of the BinaryWriter and BinaryReader closes the stream? It seems to me that these classes should only write to and read from the stream and not affectively 'own' the stream that was assigned to it. Basically, I think they should leave the stream open...

I'm sure there is a reason, but I would also have thought that there would be a way to specify whether or not the class behaves in this way...

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this