stack smashing detected

Started by
7 comments, last by mea36 17 years, 1 month ago
Hi, I wrote an opengl program on windows, and it works perfectly. However when I test it on linux (ubuntu) it works for a small input file, but not a large one. For example this works:
 $ ./a.out cube.stl 
When I use a larger input file I get a stack smashing error.
$ ./a.out Pump.stl
*** stack smashing detected ***: ./a.out terminated
Aborted (core dumped)
Does anyone know what this means? Thank you
Advertisement
google:
what is "stack smashing"

top link is a wikipedea article:
http://en.wikipedia.org/wiki/Stack-smashing_protection

Is the wikipedia article unclear?

My guess is that message means that the program has used up all of the available stack and therefore can no longer continue executing.

I would guess that the code for the program uses a local variable to hold the contents of the file. Local variables reside on the stack. So there's no problem reading in a small file but there is reading in a larger file. If that is the case, the solution is to allocate memory from the heap (ie using malloc or new) to hold the contents of the file.
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." - the Laughing Man
Quote:Original post by LessBread
My guess is that message means that the program has used up all of the available stack and therefore can no longer continue executing.

I would guess that the code for the program uses a local variable to hold the contents of the file. Local variables reside on the stack. So there's no problem reading in a small file but there is reading in a larger file. If that is the case, the solution is to allocate memory from the heap (ie using malloc or new) to hold the contents of the file.


But isn't that a stack overflow, not a stack smash?

A stack smash normally occurs when you overflow a buffer and overwrite the function calls return address, so when the stack tries to pop it goes into la-la land.
Quote:Original post by PlayfulPuppy

But isn't that a stack overflow, not a stack smash?

A stack smash normally occurs when you overflow a buffer and overwrite the function calls return address, so when the stack tries to pop it goes into la-la land.


Thanks
Quote:Original post by PlayfulPuppy
Quote:Original post by LessBread
My guess is that message means that the program has used up all of the available stack and therefore can no longer continue executing.

I would guess that the code for the program uses a local variable to hold the contents of the file. Local variables reside on the stack. So there's no problem reading in a small file but there is reading in a larger file. If that is the case, the solution is to allocate memory from the heap (ie using malloc or new) to hold the contents of the file.


But isn't that a stack overflow, not a stack smash?

A stack smash normally occurs when you overflow a buffer and overwrite the function calls return address, so when the stack tries to pop it goes into la-la land.


Ok.
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." - the Laughing Man
Quote:Original post by PlayfulPuppy
Quote:Original post by LessBread
My guess is that message means that the program has used up all of the available stack and therefore can no longer continue executing.

I would guess that the code for the program uses a local variable to hold the contents of the file. Local variables reside on the stack. So there's no problem reading in a small file but there is reading in a larger file. If that is the case, the solution is to allocate memory from the heap (ie using malloc or new) to hold the contents of the file.


But isn't that a stack overflow, not a stack smash?

A stack smash normally occurs when you overflow a buffer and overwrite the function calls return address, so when the stack tries to pop it goes into la-la land.


"Smashing the stack" colloquially refers to exploiting a buffer overflow (where do you think that buffer is? On the heap? No, it's on the stack, so the overflow is a stack overflow) deliberately in order to change the return address.

The OS can't distinguish malicious intent from an accident, however, so it assumes that anything that tries to overwrite the return address is an attempted stack smash.
Have you started up gdb to take a look at the spot where the problem his happening? With the amount of information available I can only speculate, but it would seem that either the difference in libraries, compilers, or compiler options (or clearly a combination of these) from windows to *nix is creating the problem (the stack overflow).

Starting up gdb and walking through should at least show you where this is happening and give you some clues as to why. That may help someone here diagnose the problem or at least what to research..


hth.. #dth-0
"C and C++ programmers seem to think that the shortest distance between two points is the great circle route on a spherical distortion of Euclidean space."Stephen Dewhurst
Quote:Original post by xiuhcoatl

Starting up gdb and walking through should at least show you where this is happening and give you some clues as to why. That may help someone here diagnose the problem or at least what to research..


I tried that already and suprisingly wasn't helpful. But thanks for the suggestion.

This topic is closed to new replies.

Advertisement