Back to General and Gameplay Programming

Separate Functions to Compress From Memory and From a File

Ectara · 2012-11-28T21:57:23

In my compression module, I had reworked the whole system to accept a stream as input, and a stream as output, however I want to take advantage of the raw speed and simplicity of compressing from memory to memory, as well. I can make a stream from an array of bytes, and easily pass one array for input, and one for output, but it would be slow, while accessing it in the same way that I access a file. For instance, it would be fast and easy to read through an array with a simple for loop and incrementing an index, as opposed to getting data from a stream a byte (or a few) at a time through function calls, and making sure that we haven't prematurely hit the end of the stream, or any stream error codes. My point is, all of my algorithms have been rewritten to take stream sources and destinations, and I could make a stream from a buffer in memory. However, this would be much slower than accessing the memory directly in the special case of memory to memory compression/decompression. My problem is that to implement this specialization, I would need to write two versions of each function, one that takes streams, one that takes a pointer to a block of memory. As my compression module would easily double in size to accommodate this specialization, does anyone have a suggestion as to how I implement this duality?

General and Gameplay Programming Programming

Started by Ectara November 27, 2012 04:18 PM

20 comments, last by Ectara 11 years, 5 months ago

evanofsky

2,950

November 28, 2012 08:57 PM

You're right. I'm going to drop support for all of these features, remove several modules of my libraries, and use all features that aren't supported on all of the platforms I own. I was waiting for when the first "abandon the project" post would appear.

I apologize, my sarcasm was completely unnecessary. (Don't post at 12:36am, kids.) I don't think you should abandon the project, I guess I was just echoing mhagain's thought that it really becomes a management decision as to whether it's worthwhile to support an edge case like this. I just had a hard time imagining a use case for this. That's all.

etodd.io | @etodd_

Ectara

3,097

Author

November 28, 2012 09:57 PM

I guess I was just echoing mhagain's thought that it really becomes a management decision as to whether it's worthwhile to support an edge case like this. I just had a hard time imagining a use case for this. That's all.

I understand where you are coming from. I just have a platform, right here, in my hands, that does not support memory mapping through the OS, and there are many other reasons that I feel that simply using memory mapping is not the right solution _for me_. It may be the best practice for other situations, but I have long thought it over, and have recently moved _away_ from a solution where everything must reside in memory. I was just hoping to find out if anyone knows a magical 3rd solution, because I generally come here when I have examined all options that I know, and have decided that none are the best; this need not always mean that there is another solution.

Separate Functions to Compress From Memory and From a File

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Separate Functions to Compress From Memory and From a File

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines