The main problem with just doing cells is that an always present (and temporally stable) specular term is part of the thing that really sells GI to begin with.As I understand it, the diffuse part is actually the costly one because of the large amount of cones you need to trace per pixel in the default solution. So for rather sharp glossy highlights you could keep tracing them per pixel without the intermediate accumulation step into the SH-volume. But that's of course just the theory.
Actually, it's the specular that's the most costly. This requires the most samples, especially for the high gloss, as you need to keep tracing through the octree/texture until you hit the right mip level or even the right voxel. Since it's all mipmapped diffuse may need several traces, but relatively few samples as you only need to trace a few steps into the tree.
But I hadn't thought of doing diffuse as a cell structure while keeping specular per pixel. It might not be a huge savings but it would be a savings.