Sign in to follow this  

lossless compression that utilizes SIMD architecture

Recommended Posts

Guys, can you think of an algorithm that utilizes SIMD architecture both in compression and decompression. I just can't think of one, actually I've came up with an algorithm that can utilize SIMD instructions at compression. But to utilize SIMD at decompression alone, I just couldn't think of how to do it. Anyway, if you wonder Why? Well.. this link keeps haunting me.

Share this post

Link to post
Share on other sites
Quoted from that page:
There may be no suitable solutions.

There are two problems that arise:
- Most compression algorithms are strictly sequential. Each output symbol depends on entire past state, as well as codec state at time of output
- General patterns encountered in such algorithms require very dynamic, branching code flow with random access to memory

One algorithm that might work would be something revolving around predictor/corrector schemes, perhaps something relating to wavelets. While not true compressor in itself, it does rearrange the data in a format that is considerably more suitable for entropy compression.

In practice, it is considerably more practical to instead perform block compression. Split input data into smaller chunks and hand them out to be individually compressed.

Compression is one of those areas where I'm somewhat afraid to even think about inventing something new. Perhaps there is some way to develop a novel, true distributed compression technique that would scale up to N nodes (where N is input size), perhaps there is even something related to quantum computing, but it's simply too in-depth topic that I am far from familiar with. But I have a hunch that SIMD in itself is dead end, or would provide marginal improvements only.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this