Calling convention clash....help

Started by
3 comments, last by personwholives 19 years, 6 months ago
I generally don't set my calling conventions, i let the compiler handle that, since it usually does a pretty good job. But i had a problem today, and, uh, it's really wierd. Heres the error message: Debug Error! Program: F:\theEngine\d3dtest\Debug\d3dtest.exe Module: File: i386\chkesp.c Line: 42 The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention. it is caused by this seemingly harmless line of code: if((cube = graph->CreateStaticVB()) == NULL) I have never seen an error like this before, and have no idea how to go about fixing it. Some details that may, or may not help diagnose the problem: graph is a pointer to an interface class, so the pointer actually points to a derived class. the object pointed to by graph was created in a dll, and returned to my program. when i did this previously, i never had this problem. neither the interface nor the implementing class has calling conventions declared on any of the functions. I hope someone knows what to do here, i'm stumped. Thanks personwholives

Stupid details.....always get in the way....it's time to write the dwim(x) function ("do what i mean to")
Advertisement
Well, the question is what you changed between when it worked, and when it didn't work. So, what did you change? [smile]
I've seen this before, it's nothing to do with calling conventions. Bad Things (tm) are happening with memory - bad pointer, overflows, some other error.

Make sure graph is a valid pointer, if not it could end up some weird location, which could cause this error.
Last time I saw that ESP not properly saved bit was in a fairly recent thread - basicly, it occured because the function in question took over 64kb of parameters (he was passing a game map by value, for 90KB in his instance <_<) - the bug was specific to debug mode with some compile flag, which I assume was not set by him.

I also assume similar could result if you're returning by value a class that's over 64KB in size (since CreateStaticVB obviously takes no parameters). This means static, C-style arrays. You're not using any of them are you?
i removed a few functions from the interface and combined two classes into this one. neither of these should have affected this. graph is a valid pointer, the code in the program did not change, just whats in the dll. Also, the return value is a pointer to a class, not an array or anything like that. Thats all the info i have.

personwholives

Stupid details.....always get in the way....it's time to write the dwim(x) function ("do what i mean to")

This topic is closed to new replies.

Advertisement