It is actually pretty simple.
Try to remove the given keyframe, then sample the keytrack at the given time value of the key again.
Compare that to the version where the keyframe did not get removed.
Now you have an error value that gets introduced. You simply compare that error value in a way corresponding to the type of data it represents.
For a Vector3 you can just compare the distance, for a Quaternion you can do some special check etc.
If the error is below the threshold remove the keyframe permanently, otherwise keep it.
Iterate over all kefyrames this way. The first and last keyframe will most likely be special cases that you always need to keep.
Hope that helps
Edit: Ah I didn't see you need to add lot of keyframes in realtime. You could adapt this algorithm to that though, by checking if you can remove some of the last added keyframes. Not the last one, but say the one before, maybe batching it if you add multiple at the same time. Or already do this check while adding keyframes, it shouldn't be too hard, the principle idea stays the same