Jump to content
  • Advertisement
Sign in to follow this  
ga

gcc seems to ignore my inline assembly

This topic is 4519 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Hi, I've been playing with inline assembly in C, using the gcc compiler v3.3.5 under gentoo linux. Unfortunately, gcc seems to ignore my assembly code. I wrote a function which should crash the program by writing junk into the stack, but code execution continues normally after calling it. Here's the code: <code> asm ( "movl $1221,(%esp)\n", "movl $1221,4(%esp\n)", "movl $1221,8(%esp)\n", "movl $1221,12(%esp)\n" "movl $1221,16(%esp)\n" "movl $1221,20(%esp)\n" "movl $1221,24(%esp)\n" "movl $1221,28(%esp)\n" "movl $1221,32(%esp)\n" "movl $1221,36(%esp)\n" "movl $1221,40(%esp)\n" "movl $1221,44(%esp)\n" ); </code> Thanks in advance, ga

Share this post


Link to post
Share on other sites
Advertisement
In the meantime I've found that I should just omit the commas between the assembly command strings.. argh;)

ga

Share this post


Link to post
Share on other sites
There's a difference between the compiler ignoring your code, and the program not crashing.

If you want to know if the compiler ignores the asm, try getting it to print out the compiled asm file, and check for yourself. Or just run the program in a debugger.

But writing junk on the stack doesn't neccesarily mean your program will crash.

Share this post


Link to post
Share on other sites
Thank you for the replies.

Spoonbender: That's true, but I've now looked at the asm output of gcc and the code was not there when the commas where still there. The code is inside a function with only 1 pointer argument and no local variables, so it's quite safe to assume that the code overwrites the return address (and the program did indeed crash after i removed the commas).

Share this post


Link to post
Share on other sites
As a previous poster implied, you also need to tell gcc why the inline assembly needs to be there, or the optimizer will remove it when optimizations are enabled. One way to do this is to add the volatile keyword, but there are far more elegant solutions, especially ones that honour gcc's register allocation. Consider reading one of the relevant documents.

cu,
Prefect

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

Participate in the game development conversation and more when you create an account on GameDev.net!

Sign me up!