There are (at least) 3 undo principles:
a) Inverting the effect of the last action;
b) restoring the memorized state that was valid before the last action (as you do ATM);
c) replaying the history of actions exclusive the last one.
No one of them is per se suitable for pixel painting programs:
a) is not possible because information may be lost due to the former application of the action;
b) costs masses of memory (and bandwidth in your case);
c) costs much performance if the painting has progressed too far.
A way that is suggested now and then is to combine the above possibilities with the goal to lower the average costs. For example, a memento is made only after N actions, and for then at most N-2 remaining actions a replay is done. The drawing area can be tiled for the purpose of storing a memento, so that just the tiles effected during the last N-1 actions need to be memorized. Older mementos can be externalized by a background job.