[.net] PInvoke/Unmanaged Callback Pain (SOLVED)

Started by
5 comments, last by turnpast 19 years, 1 month ago
I am haveing a really annoying problem with a unmanaged callback, let me see if I can describe it in some understandable way. The unmanaged library I am using relies on a number of callback to functoion properly. For the most part I have had no problem getting this to work, but one crutial callback fails in a very annoying way. The callback is crutial because the library is structured so that some very important state can only be set from the callback. What happens is this: I set everyting up and the unmanaged library starts calling my callback. It will work perfectly somewhere between 10 - 200 times (depending on the ammount of memory I use in the body of the callback) and then it fails. I either get a AccessViolationException from the callback code or I get a NullReferenceException from the library entry point. If a declare an array of 2000 floats in the callback it fails faster than if I declare an array of 500 which in turn fails faster than if I declare an array of 10. All this is happening in my managed code so it is probably not the the fault of the library. I am guessing that it is a memory leak (in the stack?) caused by calling back into the library from the unmanaged callback (as all the other callbacks that re-call into the library work dandy) What I don't know is how to fix it. I have already tried a great number of things (Calling conventions, unsafe pointers vs gc objects) unsafe code pointers and, but hopefully there is somehting simple that I missed. If anyone has any experience with this area or any suggestions I could really use some help. [Edited by - turnpast on March 3, 2005 3:29:51 PM]
Advertisement
Out of curiosity, how are you storing the delegate that pass to the p/invoke function in your C# program? I remember a long time ago an issue with the garbage collector cleaning up the delegate if it wasn't pinned in your program, and basically depending on the garbage collection running it might work or not.

I'm not sure if that was changed so that it automatically pins delegates passed to p/invoke calls, but you may want to try explicitly doing it. (Assign it to a static variable for an easy way to ensure the gc doesn't touch it, or play with the methods in system.runtime.interopservices)

Alex
The delegate is a variable in a an instance of a class that is maintained throughout the process, but I will try using GCHandle/Marshal to make sure that the delegate itself is not collected. The funny thing is that there are number of other similar delegates that work no problem and I am doing nothing to ensure their lifespan.

Thanks for the hint.
Sweet, it appears that the GC was in fact the culprit. Putting a GCHandle on the delegate fixed the problem. I guess it makes sense: the more memory I allocated in the callback the more likely the GC was to collect it sooner than later.

Thanks again.
Thanks for the rating++ :)
Yeah, as above, which you already know...

Generally, in my experience, the usual culprits of SEHExceptions are invariably either a calling convention mismatch or a delegate being collected in the manner you're seeing (there's also been a bug related to EnableVisualstyles that would show up as an SEHException). I'd also highly suggest you use CLRspy if you're not already and if you're doing any p/invoke of any size, it makes finding various p/invoke related problems a snap.
--Ridge
Thanks for the link Ridge, I will give CLRSpy a shot.

This topic is closed to new replies.

Advertisement