Back to General and Gameplay Programming

Buffer Overflows and Stack Direction

General and Gameplay Programming Programming

Started by Nypyren October 03, 2006 12:40 PM

2 comments, last by TheAdmiral 17 years, 7 months ago

Nypyren

12,313

Author

October 03, 2006 12:40 PM

From my understanding, the main way that buffer overflows can lead to code execution is by overwriting the return pointer that's on the stack:


before:
[local variables (buffers go here)][return address][pushed arguments] (previous function's stuff here)

after:
[buffer OVERFLOWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWW][pushed arguments](previous function's stuff here)

This is all because "push" moves the stack pointer in the negative direction. What if push moved in the positive direction? buffer overflows would continue forward in the "unused" area and would hit the guard page.


before:
(previous function's stuff here)[pushed arguments][return address][local variables (including buffers)]

after:
(previous function's stuff here)[pushed arguments][return address][local variaBUFFER OVERFLOWWWWWWWWWWWWWWWWWW]

Unfortunately the way the "push" "pop" and "ret" instructions work on x86 machines won't let you do this. There's nothing stopping you from doing everything without using those particular instructions though... Is anyone else familiar with other types of problematic overflows? Does anyone know if there's any advantage to stack 'growing negative' like this (other than by silly convention)? Back in the day, the stack would grow in the opposite direction of the heap so that they could share one large region of memory. In modern applications, the stack is usually COMPLETELY isolated with nothing else around.

iMalc

2,466

October 03, 2006 01:20 PM

Some architechures/OSes do do this. In fact I had never considered that a stack might grow in the negative direction until I began programming for PCs.

I certainly agree that it was a stupid idea to grow in the negative direction. However, if the operating-system/programming-language did array indexing the other way around to match, then there wouldn't be a problem... i.e

val = array[idx];<->val = *(array - idx);

"In order to understand recursion, you must first understand recursion."
My website dedicated to sorting algorithms

Excors

715

October 03, 2006 01:36 PM

That wouldn't solve the problems with local variables being overwritten (where those local variables might contain flags to say whether an action is permitted, or function pointers, or objects with vtables, etc) - so compilers like VS2003/2005 (with /GS on) have to be careful in any case to arrange the stack so buffers can't overwrite local variables.

As on the diagram on that page, there's stuff on both sides of buffers during a function call - so changing the stack direction would add the problem that buffer overflows could overwrite any saved registers and temporary values which are stored on the stack (because those have to be pushed after the stack space is allocated for buffers). That could perhaps include the frame pointer or some similar critical value, so you could control where the function will return to and so you can control the return address (even if you can't modify the actual return address itself). You can add the security cookie just after the buffers (instead of before) - but then the compiler is doing the same amount of work as in the normal case, without a substantial security benefit, and with the problem of breaking compatibility with the processor architecture which has demonstrated that backward compatibility sells more than any other feature [smile]

TheAdmiral

1,122

October 03, 2006 01:52 PM

To be completely anal, RET can't be entirely emulated in x86, as it is impossible to pop a value into EIP. You could always

MOV EDX, [ESP]
ADD ESP, 4
JMP EDX

but that involves the use of a general-purpose register (in this case EDX). If you needed to preserve all the registers' contents, which is uncommon but feasible, you'd come unstuck. Anyway...

What you've described is a stack-overflow exploit, which was previously the hack-of-choice on Windows XP. Such exploits are much harder to pull off since SP2 though as the stack can no longer be executed under any circumstances. So you really have to get creative when coming up with some shellcode to 'execute arbitrary code', as the phrase goes.

There are many other types of exploit though. Heap overflows are similar to stack overflows, and have similar potential. Format-string attacks are also common, and have rendered millions of C programmers very dangerous, as they all seem to love printf() and friends. The web is rich with discussion on common exploits, so check it out some time.

I think the main reason the stack grows downwards is compatibility. Many programmers and compilers just got comfortable with the fact that local variables live at [EBP+], arguments at [EBP-] and that you could 'get away with' hacking the stack by manually pulling pieces of data from where they 'should' be. If, one day, the return value no longer lived at [EBP-4] then all hell would break loose.
It also makes things that little bit easier on the compiler, knowing that a page-fault at 0x12D000 is necessarily a stack-overflow. With compilers choosing different stack sizes for different applications, the guard-page would need to be put at a different place in lomem for each process.
Besides, when you list memory increasing downwards (as we all do) it's pleasing to see the stack growing 'up the page' [smile].

Regards
Admiral

Ring3 Circus - Diary of a programmer, journal of a hacker.

Buffer Overflows and Stack Direction

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Buffer Overflows and Stack Direction

This topic is closed to new replies.

Popular Topics

Recommended Tutorials

Reticulating splines