Jump to content

  • Log In with Google      Sign In   
  • Create Account

#ActualEctara

Posted 03 December 2012 - 10:36 AM

Well, then since you don't want to seek backward, I don't see a solution other than incrementing the file position by one byte repeatedly until you hit the newline.

That would be the way explicitly specified by the C standard; the other way would have the underlying implementation bend the rules a little bit behind the scenes, like a memory stream being able to read through the entire buffer without moving the file pointer, or a buffered stream being able to scan the buffer without performing any actions. Each stream knows the fastest way to scan its data without seeking backwards (or if the stream knows it can seek backwards, it can do so without the the caller knowing).

Is it really that bad to seek backwards? It does cut out a large class of possible optimizations and design methods.

I know that it won't always be possible to avoid seeking backwards; a lot of my streams require that the target stream beneath it be backward seekable for the functionality to work, but I'd like to have the parts of code that actually use the stream try to avoid seeking backward to allow things like pipes to work.

This is off-topic but just out of curiosity, how are you implementing encrypted streams? Are you using CTR mode for random-access encryption/decryption?

It's implemented in such a way that the encryption algorithm and mode don't make a difference. The stream makes use of my encryption module, and makes sure that block ciphers and stream ciphers both can work through the same code path. For instance, AES ECB and RC4 can both be used through the same stream implementation (not at the same time); all it requires is that you set up the encryption information correctly, and use it to attach an encrypted stream to the data. The encryption stream requires seeking backward, so if it cannot be seeked, it must be cached to a seekable stream type.

Granted, I don't have all of the block cipher modes implemented, but after doing so, it wouldn't be that difficult to implement it in a way that they function properly, and code for different ciphers don't affect each other in the general encryption stream; the stream only calls functions for optional bookkeeping, and general encrypt and decrypt functions based on the information it has.

Honestly I recommend a bit of both, leaning towards maintainability. Like having something completely self-documenting, but pathetically slow is just as bad as having a super-fast but unreadable implementation. Code naturally gets faster with time, but it doesn't get any cleaner.

Yeah. In the name of reasonable speed, it may be best to leave it until I can find a way to make all of the streams fast enough to make the generic version a reasonable option.

#1Ectara

Posted 03 December 2012 - 09:33 AM

Well, then since you don't want to seek backward, I don't see a solution other than incrementing the file position by one byte repeatedly until you hit the newline.

That would be the way explicitly specified by the C standard; the other way would have the underlying implementation bend the rules a little bit behind the scenes, like a memory stream being able to read through the entire buffer without moving the file pointer, or a buffered stream being able to scan the buffer without performing any actions. Each stream knows the fastest way to scan its data without seeking backwards (or if the stream knows it can seek backwards, it can do so without the the caller knowing).

Is it really that bad to seek backwards? It does cut out a large class of possible optimizations and design methods.

I know that it won't always be possible to avoid seeking backwards; a lot of my streams require that the target stream beneath it be backward seekable for the functionality to work, but I'd like to have the parts of code that actually use the stream try to avoid seeking backward to allow things like pipes to work.

This is off-topic but just out of curiosity, how are you implementing encrypted streams? Are you using CTR mode for random-access encryption/decryption?

It's implemented in such a way that the encryption algorithm and mode don't make a difference. The stream makes use of my encryption module, and makes sure that block ciphers and stream ciphers both can work through the same code path. For instance, AES ECB and RC4 can both be used in the same stream; all it requires is that you set up the encryption information correctly, and use it to attach an encrypted stream to the data. The encryption stream requires seeking backward, so if it cannot be seeked, it must be cached to a seekable stream type.

Granted, I don't have all of the block cipher modes implemented, but after doing so, it wouldn't be that difficult to implement it in a way that they function properly, and code for different ciphers don't affect each other in the general encryption stream; the stream only calls functions for optional bookkeeping, and general encrypt and decrypt functions based on the information it has.

Honestly I recommend a bit of both, leaning towards maintainability. Like having something completely self-documenting, but pathetically slow is just as bad as having a super-fast but unreadable implementation. Code naturally gets faster with time, but it doesn't get any cleaner.

Yeah. In the name of reasonable speed, it may be best to leave it until I can find a way to make all of the streams fast enough to make the generic version a reasonable option.

PARTNERS