I can't comment on your solution itself, but I'll address some of your reasons for the solution. If there are alternative solutions that are just as easy, that don't have the side-effect of huge compile times rippling through the codebase, you could decide for yourself if they are worth switching to.
I realize that it's standard to only declare what you need in a header, but it's far too easy to remove some code that you used to use and forget to remove the declaration that it required.
An understandable concern - it could be a good programmer-discipline standardized for your project to Ctrl+F the entire project for occurrences of that class name. (A find-and-remove of "class AudioEngine;"). If a programmer forgets to do this, then those declarations don't do any harm, since the code doesn't reference them, and they'll be discovered and removed by Ctrl+F-ing the entire project when they are later discovered.
Or when a class gets renamed, adding in its new declaration and forgetting to remove the old one.
I use QtCreator as my IDE. To rename a class, I right-click on the class name and do "Refactor -> Rename symbol under cursor", and it renames every usage, declaration, and definition of that class throughout the entire project effecting only the files and headers that use it. You can do the same with function or member-function names, and variable instances and member-variable names. It's very useful - are you sure your IDE doesn't have a similar feature?
Alternatively, the old "Find-and-replace" project-wide works just as well and has the same effect (with the added bonus that it even catches usage within comments).
The primary benefit is keeping code clean, as I said. I'm rather OCD about keeping my code organized and easy to read (both for myself and for the sake of others looking to understand it).
Which is a fantastic habit to have. 
Sure it's not a huge benefit and we could easily do without it, but keeping class declarations updated in one file is a heck of a lot easier than keeping them updated across numerous files.
An alternative, which partially addresses ChaosEngine's complaint, and partially-addresses the compile-time complaint is to have system-specific "Declaration" headers. "AudioDeclarations.h", "VideoDeclarations.h", "CoreDeclaration.h", etc... A change to the audio declaration only effects the audio-subsystem and the non-audio files that include audio-headers. It'd still be easy and maintainable, but also properly keeping the different sections of the project unaware of even each other's pre-declarations, except when explicitly included. Such declarations should also have any system typedefs, unless they are class-specific typedefs which are within the class' namespace.
The standard C++ library has a header file <iosfwd> that forward-declares alot of the iostream templates and classes.