# Aligning ESP stack pointer

This topic is 3744 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

In a recent post I found a problem where a DLL compiled with GCC expects the stack to ALREADY be aligned to 16 byte boundaries, wherefore misalignment causes a crash in some SSE2 routines in the DLL. This happens when calling the GCC DLL function from a VC++ application, where the stack pointer is only guaranteed to be 4 byte aligned. I need to call this function in the DLL: int _cdecl ffmpeg_main(char* params); Can anyone here provide me with some Visual C++ (Express 2005) inline ASM (or otherwise), make sure ESP is 16 byte aligned once in the DLL function? It'd be greatly appreciated.

##### Share on other sites
__asm{	mov	eax, esp	and	eax, 15	add	eax, 4	sub	esp, eax	push	eax	push	params	call	ffmpeg_main	add	esp, 4	pop	eax	add	esp, eax}

ESP will be 16-byte aligned on entry to ffmpeg_main, params is your char* argument. Hope that works.

##### Share on other sites
Thanks alot! It worked after a slight modification ;) Rate up for you. Correction and capturing of return value:

	__asm	{		mov     eax, esp		and     eax, 15		add     eax, 8		sub     esp, eax		push    eax		push    cmdLine		call    ffmpeg_main		mov     retVal, eax		add     esp, 4		pop     eax		add     esp, eax	};

##### Share on other sites
You're welcome.

I don't understand your correction however, why are you pushing an extra 4 bytes? If you're stepping through it with a debugger note that the call instruction will also push the return address onto the stack, so your ESP should be 4 off just before and after the call, but on entry to the function it will be aligned.

##### Share on other sites
Ehm.. well, I have no idea to be honest. All I know is that the SSE2 stuff doesn't crash any more when the low ESP nibble is zero before the "call". When GCC expects the stack to be 16 byte aligned, perhaps that's what's meant?

##### Share on other sites
Just thinking about a DLL having that kindof requisites makes me sick. This is something that should be handled in the functions themselves, not be forced upon the consumer of the module.

##### Share on other sites
Hehe.. well fortunately my application will be the only consumer of this particular DLL. However it's likely it might always be an issue when using the original FFMPEG DLLs from VC++. It's the price we pay when there's no standard for these kind of things. Using a newer unofficial realease of GCC there is automatic stack alignment in critical functions. I just rather fix it in VC++ than in FFMPEG, as it's nice to just get the SVN without having to change the code. That and I've always thought GCC's inline ASM looked scarier than VC++'s ;)

##### Share on other sites
Yeah, the AT&T syntax scares me too.

##### Share on other sites
I am also having the same problem,
calling avcodec_decode_video() from microsoft visual studio 2005 crashes the application. I dont have knowledge in inline assemply so please provide me Inline assembly( with ESP 16 byte aligned ) to call this function .

int avcodec_decode_video(AVCodecContext *avctx, AVFrame *picture,
int *got_picture_ptr,
uint8_t *buf, int buf_size);

where AVCodecContext and AVFrame are structures.

• 9
• 23
• 10
• 19