I know that usually noobies who desparately need help overreact in subject topics but I assure you I don't have a nooby problem :).
So, I've a game I'm writing with lots of classes and so on. One day I decided to add additional variable to some class. I compiled it and... hmm... To put this straight I'll enumerate the "combinations":
- game run in Debug mode *through* Visual Studio (by pressing F5) - everything works
- game run in Release mode *through* Visual Studio (by pressing F5) - everything works
- game run in Debug mode by running the exe file by hand - everything works
- game run in Release mode by running the exe file by hand - *problem!*
The problem is that from time to time some specific function (the same all the time) is being called. I would check the call stack but I simply can't - the problem doesn't appear neither in Debug mode nor when game is ruh through VS.
So the problem appears only if I declare some variable in the class. That's the only difference between working and non-working version.
So, who to blame? Me because of some horrible bug in the code? The compiler becauses it messes something up when the class is bigger in few bytes? Or maybe some Windows's DLL that's get loaded when running the game outside VS? I simply have no idea.
I can only add that my friend compiled the game under Linux and didn't observe the problem.

You probably have an uninitialized data problem, or a buffer overrun.

Running from VS, or a file linked against the debug libs, will generally get you the debug heap, which has a number of quirks that hide those sort of things. Running release outside of visual studio doesn't.

If so, shouldn't VS in debug mode "scream" when my application uses uninitialized data or overruns a buffer?

Since you did not specify, I am assuming here you are working on a native Visual Studio C/C++ application.

Quote:Original post by maxest
One day I decided to add additional variable to some class. I compiled it and... hmm...

The first thing: do a full rebuild of the project to confirm that VS incremental rebuild was not messed up. If you have done any funny stuff to your solution/project build configurations, check that Clean all actually succeeds in cleaning up everything before proceeding. (nothing more fun than finding bad paths and images not getting updated with new version at all due to it)

Quote:Original post by maxest
- game run in Debug mode *through* Visual Studio (by pressing F5) - everything works
- game run in Release mode *through* Visual Studio (by pressing F5) - everything works
- game run in Debug mode by running the exe file by hand - everything works
- game run in Release mode by running the exe file by hand - *problem!*

The most commonly I've seen this is in the context of thread race conditions. Running in the debugger would not exhibit the same run patterns as running outside the debugger, and in debug mode the thread code execution patterns hide the issue present in release mode. To debug these:

• Write a crash handler that automatically produces a .dmp file. See for example msdn (a bit old, for VS 2005, but gives you more words and concepts to google for)
  
• For .dmp, compile the release mode with debug mode program info database enabled, and with /Oy- to make most use of .dmp files.
  msdn
  
• Add manual re-entrancy checks and debug mode lock guards to your code to find issues with lockless data sharing.
  

The second relates to bad memory use, as ReaperSMS already pointed out. To debug:

• Write code to generate .dmp files, as above.
  
• msdn . Particularly useful are the _CRTDBG_ALLOC_MEM_DF, _CRTDBG_DELAY_FREE_MEM_DF and CRTDBG_CHECK_ALWAYS_DF, but they make debugging your app run very slow. To find a middle ground, see msdn .
  
• Make sure your runtime libraries match in all objects you are linking together.
  
• Make sure all your code is compiled with the same #defines (minus those you know that can/should safely differ). Especially the CRT-altering ones, like _SECURE_SCL=x.
  
• Try Visual Leak Detector. While it's mainly for detecting memory leaks, it can also make hidden memory-related crashes manifest themselves. vld .
  
• Compile with /Oy-. Try disabling optimizations altogether in Release.
  
• Force assert()s in into the release build as well.
  

Quote:Original post by maxest
I would check the call stack but I simply can't - the problem doesn't appear neither in Debug mode nor when game is ruh through VS.

You can attach a debugger to a crashed process post-mortem, but you need to have the program database generated.

Quote:
# Try Visual Leak Detector. While it's mainly for detecting memory leaks, it can also make hidden memory-related crashes manifest themselves. vld .

I've tried. It showed nothing. The application doesn't have any kind of memory leaking

Quote:
# Compile with /Oy-. Try disabling optimizations altogether in Release.

I've used /Od and... it works. The problem doesn't occur. What can that mean in this context?

Quote:
I've used /Od and... it works. The problem doesn't occur. What can that mean in this context?

It means that when you use /Od it works and the problem does not occur. Don't use /Od then, but build with such optimization flags that will show the problem, and continue debugging the crash with other means.

I've tried attaching the debugger to my application and although the function is still being called (the one that shouldn't), the debugger doesn't expose any kind of crash.

Moreover, I simply changed linking from Multi-threaded to Multi-threaded DLL and the problem no more appears. However, since it causes the output exe file to be smaller, the data arrangement in it must differ so this "bad pointer" or something can probably be doing something else (calling some other function) that I can't see immediately.
Just to be sure.. after I had attached the process to the debugger and put some breakpoint to the function that is being called, the debugger indeed stopped in this place and gave me a call stack. But unfortunately I don't have an insight into the values of all the variables. And yes, I have a Program Debug Database generated for the Release. Yet they're smaller than those from the Debug version. Some magic switch should I make in the options?

This may also be something with the "thread race" you mentioned. The code is indeed multithreaded so I shall check these .dmp files.

[Edited by - maxest on August 29, 2010 10:45:51 AM]

Well, in fact yesterday I did a "binary search" in the project's repository and found the one from which adding this variable to the class causes problems. However, the change was very subtle so I'm not sure whether this could be the problem. But I will do fruther investigation of course and let you know.