Above was an example of how it usually looks (and how it supposed to look). I save ALOT of data so didnt want to post it all.

I accidently included a range with both pointers and lists. I'm not entirely sure why this did work in debug. I didn't use the new pointers/lists after load but in release mode  copying over trash data like this seems to have broken something (while in debug maybe it's handled differently).

I must check more regurarly that my games run correctly in release:) After moving the pointers/lists (and save/load them manually) load/save works fine both in release and debug.

16 hours ago, suliman said:

I found I had included something that wasnt strictly primitive data in my saved data-range...

This seem to work fine in debug mode but not in release (maybe debug allows some errors, or "fixes" the errors trying to be helpful but instead masking a problem i actually had all along).

Those save/load ranges that did only (truely) include primitive data worked well even in release build.

As matter of principal I still think in general it's not a good idea to rely on memory layout staying the same.  A different compiler could break it. A newer version of the same compiler could break it.  Compiling for a different architecture could break it. If you add a field above the data you are currently saving, that could break it too by causing the compiler to change alignment. By break it, I mean existing files would be invalid.   You don't really have a file format, that can be documented.  You have something closer to a memory dump.

As has been said, there are likely to be portability issues no matter what precautions you take, not only between platforms, but even between and within development environments (as Gnollrunner pointed out). You're probably already aware, but for what it's worth there are alternate approaches and third-party solutions that are insensitive to environmental factors like layout, endianness, and representation of primitive types, can handle complex types such as standard library containers, and can serialize to and from human-readable formats such as JSON.

If you intend to stick with your current approach though, there may be some things you could do to protect yourself a bit. One option would be to store serializable data in a single POD object, rather than trying to serialize portions of a more complex object. You could then use std::is_trivially_copyable to check for obvious oversights. This wouldn't protect against serializing raw pointers, but might catch other errors.