Skip to main content
GameDev.net gamedev.net
🔒 Locked

Handling "float" in in generic memory (blob)

Started by Juliean May 22, 2021 at 12:53 PM 20 replies 19.1k views
Original Post
Juliean
Juliean

Hello,

so for my bytecode/interpreted language, I have support for primitive types - mostly byte, int and float. For a while I've been treating int and float separately - there are multiple operations, like loading from and storing to a local variable:

case OpCodes::LoadInt:
{
	const auto offset = stream.ReadData<LocalOffset>();
	const auto value = m_stack.GetValue<int>(state.pFrame, offset);

	m_stack.Push(value);
	break;
}
case OpCodes::LoadFloat:
{
	const auto offset = stream.ReadData<LocalOffset>();
	const auto value = m_stack.GetValue<float>(state.pFrame, offset);

	m_stack.Push(value);
	break;
}
case OpCodes::StoreInt:
{
	const auto offset = stream.ReadData<LocalOffset>();
	auto& ref = m_stack.GetRef<int>(state.pFrame, offset);
	ref = m_stack.Pop<int>();

	break;
}
case OpCodes::StoreFloat:
{
	const auto offset = stream.ReadData<LocalOffset>();
	auto& ref = m_stack.GetRef<float>(state.pFrame, offset);
	ref = m_stack.Pop<float>();

	break;
}

The reason that I originally that it that way, is that I saw a lot of languages do it that way (Java for example), and I didn't think much of it.

Now that my language is pretty evolved, I'm trying to converse space in the “OpCodes", so that they can stay 8 bit (I'm currently using 222 out of 255). And that got me thinking - is there actually any benefit for treating “float” explicitely in a situation like the above? I'm thinking about changing the instructions above to “LoadWord" and “StoreWord”, which would handle word-sized variable, properly via ind or uint, instead of separate int/float.
I'm just not sure if its a good idea. I know from testing that in general reinterpret_casting a the content of a float to an int works and preserves certain aspects/operations (equality/ordering). But on the other hand, the c++-compiler always generates specific instructions/registers (XMM) for dealing with floating-point types. So is it actually advantageous to always treat float-data as “float”, or is the generated floating-point-assembly just better when dealing with large-scale floating point operations (like a full functions of float-operations following each other; which is not the case in my bytecode)?

Hope my question/concern makes any sense, perhaps from somebody who knows a bit more about the inner workins of CPUs and/or IEEE-standards and what not.

Oberon_Command
Oberon_Command

Juliean said:
I'm just not sure if its a good idea. I know from testing that in general reinterpret_casting a the content of a float to an int works and preserves certain aspects/operations (equality/ordering)

It may appear to work, and some shipping code has depended on it (the famous inverse square root from the Quake III codebase, for instance), but it's actually undefined behavior to reinterpret cast an int as a float and vice versa. I believe this is the case even in C, using C-style casts - it's just so common that most major compilers will “probably” let you get away with it in “most” cases (because of the sheer quantity of existing code that would break if this were enforced strictly), but it could cause the optimizer to produce some very strange results, if for example it decided that the UB can't happen and optimized out all of your actual code as a result.

If you're going to be moving memory around as opaque bytes, you'll want to treat it explicitly as raw bytes, and be sure you only cast it to the type you know it is back from raw bytes. That is what the std::byte type (introduced in C++'17) is for. In C and in earlier standards, chars serve the same function.

If you're in a situation where you need “type punning”, in general, I encourage you to watch this CppCon talk on this exact subject. The float/int punning case is mentioned explicitly at about 8 minutes in and the solution is to store the int/float by value and memcpy the raw bytes into the float/int whose lifetime has already started. This takes care of the lifetime and alignment problems that casting from raw bytes can cause.

Juliean
Juliean

Oberon_Command said:
It may appear to work, and some shipping code has depended on it (the famous inverse square root from the Quake III codebase, for instance), but it's actually undefined behavior to reinterpret cast an int as a float and vice versa. I believe this is the case even in C, using C-style casts - it's just so common that most major compilers will “probably” let you get away with it in “most” cases (because of the sheer quantity of existing code that would break if this were enforced strictly), but it could cause the optimizer to produce some very strange results, if for example it decided that the UB can't happen and optimized out all of your actual code as a result.

You're right about that. But i'm also pretty sure that is works on any major compiler - on MSVC by default, and as long as you don't enable “strict aliasing” on GCC or Clang. Which I'm pretty sure I wouldn't be able to do without some reworks if I ever went with those compilers, as I do depend on a UB-reinterprets at a few places.

Oberon_Command said:
If you're going to be moving memory around as opaque bytes, you'll want to treat it explicitly as raw bytes, and be sure you only cast it to the type you know it is back from raw bytes. That is what the std::byte type (introduced in C++'17) is for. In C and in earlier standards, chars serve the same function. If you're in a situation where you need “type punning”, in general, I encourage you to watch this CppCon talk on this exact subject. The float/int punning case is mentioned explicitly at about 8 minutes in and the solution is to store the int/float by value and memcpy the raw bytes into the float/int whose lifetime has already started. This takes care of the lifetime and alignment problems that casting from raw bytes can cause.

Ah, right, I forgot about the std::byte-type. But I already knew about the memcpy-trick, as well as that we now have “std::bit_cast” in c++-20. The only reason I don't use std::bit_cast or memcpy at that point in time is that it introduces a considerable overhead in debug-builds, which is not acceptable for my use-case. I was thinking about making a macro that does reinterpret_cast in debug and bit_cast otherwise though.

Shaarigan
Shaarigan

Languages like C/C++ are treating those instructions different for using the full power of the CPU they're running on. Thanks to the massive amount of impact from the games industry, modern CPUs today have optimized instructions for floating point arithmetics and so it is worth it for the compiler to handle them different. MSVC for example gives the option to also increase performance on the cost for precision in floating point arithmetic.

I guess the reason that languages like Java and C# of course, are handling integers and float different is simply type safety in the first and maybe performance improvements in the second. I know that the .NET JIT is compiling code into assembly when a C# application for example is launched, so the same rules take appearance as like for C/C++, increasing performance with CPU speciic floating point instructions.

I don't know much about your language but if I would implement my first naive thoughts of a compiled scripting language, I'd not make any difference between integers and floats as they're both nothing but data. One of the big benefits of C/C++ over C# in my opinion is that I can treat memory as whatever I want it to be, a byte array, an integer or a floating point number, for as long as I can pass the address as pointer

Juliean
Juliean

Shaarigan said:
Languages like C/C++ are treating those instructions different for using the full power of the CPU they're running on. Thanks to the massive amount of impact from the games industry, modern CPUs today have optimized instructions for floating point arithmetics and so it is worth it for the compiler to handle them different. MSVC for example gives the option to also increase performance on the cost for precision in floating point arithmetic.

Yeah, that makes sense - and it seems logical that the compiler would always opt to generate floating-point instructions even if a function does, say, nothing but take a floating-pointer parameter and return it (even if the result is the same if we were to perform the operations via generic mov-instructions). Seems just consequential to me, to always deal with “float” by using float-instructions, when available.

Shaarigan said:
I guess the reason that languages like Java and C# of course, are handling integers and float different is simply type safety in the first and maybe performance improvements in the second. I know that the .NET JIT is compiling code into assembly when a C# application for example is launched, so the same rules take appearance as like for C/C++, increasing performance with CPU speciic floating point instructions.

Ah yeah, it does make sense when we think about JIT. I'm personally not going to deal with JIT in the foreseable future. I'm already getting crazy good results in some syntetic benchmarks from my new versus old system (something like 32x (!) speedups - not that the new system is so good but that old was just really bad), so I'm more focused on getting stuff working again. So I think thats not a reason for me.
Type-safety is an argument. I currently don't have many type-checks in place. I was thinking about doing a debug-stack, but the need didn't really arise yet. Most problems with types just eigther appeared immediately (trying to treat an int as a string), or by getting a stack-underflow.

Shaarigan said:
I don't know much about your language but if I would implement my first naive thoughts of a compiled scripting language, I'd not make any difference between integers and floats as they're both nothing but data. One of the big benefits of C/C++ over C# in my opinion is that I can treat memory as whatever I want it to be, a byte array, an integer or a floating point number, for as long as I can pass the address as pointer

Yeah, I see it the same way. I mean, I was in a bit over my head when I started, so naturally I just made instructions for different data-types. I only now got the experience to go back and say “wait, those are actually functionally the same”. So I was trying to see if there are some obvious reasons for why you wouldn't want to do it, but I don't see anything tangible - I'll have to keep the issues with UB-reinterprets in mind, but other than that I think I'll just merge all the float/int-opcodes for now. Should probably even be a net gain in performance by increasing cache-hit rate and locality of reference/instructions.

SyncViews
SyncViews

Juliean said:
Yeah, that makes sense - and it seems logical that the compiler would always opt to generate floating-point instructions even if a function does, say, nothing but take a floating-pointer parameter and return it (even if the result is the same if we were to perform the operations via generic mov-instructions).

With say C/C++, the compiler can't know know that in the general case*. It is generally preferable to have a function calling convention that passes by register to some extent (e.g. the Microsoft x64 default uses RCX, RDX, R8, R9, and XMM0 to 3) , and will also generally be preferable to use the correct register type.
Pretty sure I have seen compilers use the basic mov instruction on floating point types when they are going from memory to memory.

The compiler on the calling side only sees the function signature from the header/function-pointer/etc. it doesn't know if float foo(float a); is going to do arithmetic, or just return. I assume they did the research before into more flexible calling conventions and decided it is not worth the pain for a little extra performance (also a lot of small functions where relative calling overhead is high will get inlined already), beyond the existing __stdcall, __cdecl, __fastcall, __vectorcall, etc.
And of course the compiler for the function itself has to accommodate what the caller will do, so will use the XMM registers even if just compiling a return a; (although I just realised a float foo(float a) { return a; } might actually be a no-op since is moving XMM0 to XMM0).

* I guess link time code gen changes this, as well as calling functions in the same translation unit, but it seems would be adding a whole mess of complexity that only applies to some cases so unless there was a compelling performance reason.

Juliean said:
I'll have to keep the issues with UB-reinterprets in mind, but other than that I think I'll just merge all the float/int-opcodes for now.

Isn't the memory to memory case the only one that is fully safe to merge though?

If I recall the comparison instructions are different because of NaN values, and I think some other considerations like signed zero. And of course all the arithmetic operations are different as well.

SSE and other vector extensions do actually combine some things (not sure on why exactly they kept scalar integers but transitioned scalar floats to SSE, maybe really wanted to avoid the 80bit stuff?), but there is still a lot of instructions that are integer or fp specific (for the same data size, e.g. packed 32bit float and int).

Juliean
Juliean

@sync views Thanks also for the insights!

SyncViews said:
* I guess link time code gen changes this, as well as calling functions in the same translation unit, but it seems would be adding a whole mess of complexity that only applies to some cases so unless there was a compelling performance reason.

I didn't really look at the link-time output, as I'm mostly using godbolt.org for this kind of stuff and I don't think they have a link-time optimizer. But if they did, I'm fairly certain it might end up producing the same functions - I've seen that kind of stuff where especially template-functions with different types all end up being merged back into one block of ASM.

SyncViews said:
Isn't the memory to memory case the only one that is fully safe to merge though?

Memory-to-memory is safe, but if you look at the current OpCodes I posted, then:

m_stack.Pop<int>();

results in an reinterpret_cast() on the stacks memory, which I'm pretty sure I agree is actually UB (you can cast anything to char* or void*, but not the other way around). In practice, as I said I'm only running compiler(s) that don't have a problem with this kind of stuff. But I know UB can be nasty. The most annoying issue that I ever had is with pretty much the following code:

void dontAskMeWhy(Class* pObject)
{
	bool isNull = !pObject;
	auto& local = *pObject;
	
	if (!isNull)
		local.Function();
}

Without talking about the details of the code in question, I was assuming that this was safe. But as dereferencing a nullptr is UB, Clang just decided that it doesn't have to do the if-check at all. So compilers can and will absoluetely take advantage of UB to pretty much decide your code is not valid. With reinterpret_casts, I think I've already read about cases where the compiler will discard an entire block of code because it knew that the initial cast is not valid (which I'm afraid would probably happen to my code here if I were on compiler that gave a fuck :D )

SyncViews
SyncViews

Juliean said:
I didn't really look at the link-time output, as I'm mostly using godbolt.org for this kind of stuff and I don't think they have a link-time optimizer. But if they did, I'm fairly certain it might end up producing the same functions - I've seen that kind of stuff where especially template-functions with different types all end up being merged back into one block of ASM.

Well godbolt uses GCC which does, but if you are only using 1 source file it doesn't matter. Link time code gen is just a way to optimise for multiple source files and even static libs since the linker sees all the files, but the compiler only sees the one source file when making the obj. It was just an aside that given a float foo(float a); a modern compiler actually might in some cases, know that it is OK to put a in an integer register or something, but in the general case, it should use the floating point specific conventions.

C++ templates of course complicated this a bit, but a linker merging identical functions is a lot simpler and doesn't use link time generation (it could just compare the final compiled functions in the object file).

Juliean said:
Memory-to-memory is safe, but if you look at the current OpCodes I posted, then: m_stack.Pop(); results in an reinterpret_cast() on the stacks memory, which I'm pretty sure I agree is actually UB (you can cast anything to char* or void*, but not the other way around). In practice, as I said I'm only running compiler(s) that don't have a problem with this kind of stuff. But I know UB can be nasty. The most annoying issue that I ever had is with pretty much the following code:

Yeah, but I meant only this memory to memory case is safe (and would be fully defined if you changed your implementation*), so you save only a few opcodes at most. You should still have type specific opcodes for all the comparisons, all the arithmetic, etc.

Since your store/load only copies values, you definitely should be able to make it safe. Since you are basically reimplementing memcpy(stack_base + stack_size, stack_base + offset, 4), and while memcpy is usually special in the compiler for optimisation reasons, I don't believe it is in the language spec and I believe a pure-C implementation is possible.

If there is a place you are completely breaking the rules, I'd guess it is in other ops, e.g. if you did say:

auto result = (*reinterpret_cast<float*>(stack_ptr + offset_a)) * (*reinterpret_cast<float*>(stack_ptr + offset_b));

And I think even then only if the compiler could prove that you previously accessed those as something other than a float.

But again copying to a local float first should be safe I believe, and might even get optimised out (into just a single load instruction for the register representing the local variable).


Juliean
Juliean

SyncViews said:
Well godbolt uses GCC which does, but if you are only using 1 source file it doesn't matter. Link time code gen is just a way to optimise for multiple source files and even static libs since the linker sees all the files, but the compiler only sees the one source file when making the obj. It was just an aside that given a float foo(float a); a modern compiler actually might in some cases, know that it is OK to put a in an integer register or something, but in the general case, it should use the floating point specific conventions.

It probably just needs to be switched on, I only really know the bare-bone switches for optimization-levels to get me by.

SyncViews said:
C++ templates of course complicated this a bit, but a linker merging identical functions is a lot simpler and doesn't use link time generation (it could just compare the final compiled functions in the object file).

I always just assumed they were they same. Its true that COMDAT-folding is an optimization with a separate setting even in MSVC.

SyncViews said:
Yeah, but I meant only this memory to memory case is safe (and would be fully defined if you changed your implementation*), so you save only a few opcodes at most. You should still have type specific opcodes for all the comparisons, all the arithmetic, etc.

Puh, I'm not an expert on the c++-standard, but from the wording I've read (can't find it quickly right now) I always assumed that even this was illegal:

char* pMemory;
const int value = *reinterpet_cast<int*>(pMemory);

No matter what I actually end up doing (which you are right, that actual operations on the data other than copying it would happen with the right type).

SyncViews said:
But again copying to a local float first should be safe I believe, and might even get optimised out (into just a single load instruction for the register representing the local variable).

Yes, it will definately be optimized out of release-builds. Unfortunately I'm in a situation where debug-build performance really matters. If it was only for release-performance, I wouldn't have needed the whole rewrite so badly if it wasn't for debug-performance (in release even the old system was fast enough for most intents and purposes). Now I know this is a delicate line. And I could also just always set the interpreter.cpp to compile as “release”. But for actual debug, I did measure a huge impact of at least std::bit_cast (2-3x as slow), so thats why I went back to reinterpet_casts. Thats BTW also why the code I posted is not a template-method but just same C&P code for int/float - I usually heavily use template-functions, and I usually don't have a problem with using small inline functions but I really don't want to impose the overhead of one additional function-call for all opcodes in debug.

Oberon_Command
Oberon_Command

SyncViews said:
Since your store/load only copies values, you definitely should be able to make it safe. Since you are basically reimplementing memcpy(stack_base + stack_size, stack_base + offset, 4), and while memcpy is usually special in the compiler for optimisation reasons, I don't believe it is in the language spec and I believe a pure-C implementation is possible.

memcpy is special for more reasons than that:

Objects of implicit-lifetime types can also be implicitly created by:

call to following object representation copying functions, in which case such objects are created in the destination region of storage or the result:

Oberon_Command
Oberon_Command

Juliean said:
And I could also just always set the interpreter.cpp to compile as “release”.

This is what I would suggest doing. My impression is that it is not unusual for large C++ codebases to have optimizations turned on across all files in all configurations devs use regularly, and optimizations are disabled at the per-file level when necessary.

SyncViews
SyncViews

Juliean said:
Puh, I'm not an expert on the c++-standard, but from the wording I've read (can't find it quickly right now) I always assumed that even this was illegal:

I'd have to look it up. I think the cast is legal as long as pMemory “is an int”, that is, in both C and C++ these are allowed

// 1. Round trip some type through signed/unsigned char* or void* and back again
int_ptr2 = (int*)(char*)int_ptr;
foo->user_ptr = (void*)&my_int; // or often a larger structure
int fd = *(int*)foo->user_ptr; // generally in a callback or such later

// 2. Take part of a char* buffer and treat it as some *single* type (basically a memory allocator)
char *memory = ...;
// can be any subset of memory, and might be via void*, but we must ensure correct alignment for the T* being made!
// malloc and friends I believe are speced to return a pointer aligned to all primitives types, but on some platforms this might still not be enough for some vector types outside the C/C++ standards
int *int_ptr = (int*)(memory + 20); 
*int_ptr = 20;
int y = *int_ptr + 5;
// On C++, you have placement new + delete, and is required for anything with a constructor or destructor!
// Still have to ensure alingment!
std::string *str = new(memory + 24) std::string("Hello world!");
str->~string(); // when done with it, does not deallocate "memory" but that block could now be used for another object safely

The bit I am not sure on in 2. is if this is legal, which would have to check

char *memory = ...;
int *int_ptr = (int*)(memory + 20);
*int_ptr = 5; // "allocated" memory
// later
int *int_ptr2 = (int*)(memory + 20); // Note I casted again, instead of using the same int_ptr as before
int x = *int_ptr2; // The compiler can see the cast and might consider this memory as not an integer and be UB

But this is also not exactly the same as what memcpy does, since it never casts the pointer to an incompatible type (you are only doing T* → char/void*, never char/void* → T*), it simply copies bytes.

void memcpy(void *dst, const void *src, size_t len) // I  beleive char* is equally valid
{
    char *dst2 = (char*)dst; // casting any pointer to char* is OK
    const char *src2 = (const char *)src;
    for (size_t i = 0; i < len; ++i)
        dst2[i] = src2[i]; // copying the char values is always legal
}

Also if I recall, copying the bytes from say an integer to a float or partial copies between different sizes is “implementation defined” rather than “undefined”. This is because the standard does not promise what the byte level format of these types is, and so it can't tell you what specific bit/byte values will be.

I think the newer standard promises twos-complement signed integers so copying between signed and unsigned integers of the same size is defined. But the floating point formats are still implementation specific, as is the sizes of char/short/int/long/float/double/etc. within certain constraints.

You will also encounter a similar thing with binary file or network IO.

SyncViews
SyncViews

@Oberon_Command Hmm, if I have time will have to see if can find that in the actual standards.

Of course as mentioned, in C++ you have placement new which should be used. Since C doesn't have this I believe it is OK to not use placement new specifically on primitive types which have no constructor or destructor and can be initialised by assignment.

And it does say “operations that begin lifetime of an array of type char, unsigned char, or std::byte, (since C++17) in which case such objects are created in the array,”. So my understanding here, is that m_stack is containing say a char stack[16*1024] or a vector of char, or such is meeting that?

And in practice, code uses many API's other than std::malloc or the other functions specifically listed, either their own allocators, or operating system provided ones, so not sure if cppreference is just providing a non-exhaustive list of examples. But at the very least it means every compiler I can think of is going to allow it, because Windows has VirtualAlloc etc. etc., Linux/posix has aligned_malloc etc.

Juliean
Juliean

SyncViews said:
I'd have to look it up. I think the cast is legal as long as pMemory “is an int”, that is, in both C and C++ these are allowed

Ah, that might be. But in my case, this would still be not allowed since I the data would actually be “float”. So if the compiler knew that I then tried to reinterpret the “float” as “int” to copy it, it shouldn't be allowed.
More so, the compiler simply has no way of knowing what the internal data actually is in my code (since I'm not passing it a data-source that it can trace through the last few lines of code or calls; but a pointer to a generic memory-pool that is written to and read from different sources). So the compiler can only eigther chose to ignore that fact that it knows nothing about what I'm trying to reinterpret_cast (which is what MSVC does) or treat it as UB and probably not read/write anything at all (which is what I belive other compilers may do).

Oberon_Command said:
This is what I would suggest doing. My impression is that it is not unusual for large C++ codebases to have optimizations turned on across all files in all configurations devs use regularly, and optimizations are disabled at the per-file level when necessary.

I just need to do some real-world testings before I can fully commit to that. I already tested that idea on its own and it worked. However, now I have implemented the ability to bind arbitrary c++-functions, this might still be an issue- since those functions essentially access the same “stack” operations Pop/Push, but compiled via template in the cpp-file where they are registered. I guess that the function-call itself is probably way more expensive then the added overhead of std::bit_cast or something for retrieving an int, but I still want to measure it in the actual game (which will still take me a while to get to run).

SyncViews
SyncViews

Juliean said:
Ah, that might be. But in my case, this would still be not allowed since I the data would actually be “float”. So if the compiler knew that I then tried to reinterpret the “float” as “int” to copy it, it shouldn't be allowed.

EDIT:

Well that depends on my question regarding case 2. It might be allowed, I am not sure. You initialise the memory, but it lets the pointer go out of scope, and later you cast again, which is the bit I think might break it.

So given the pointer goes out of scope, the compiler might say the assignment “has no effect” and optimise it out.
And in the read case the compiler might say “you are reading uninitialised memory” and so not do the read.

So the safe way I see is like memcpy which is operating on the chars, because that is defined as I recall, you could even just code it like this. I believe it is just the moment you try and go the extra step and cast to a larger type there is potential issues (even if the address is aligned, and if unaligned certainly could have problems).

static_assert(sizeof(int) == 4 && sizeof(int) == sizeof(float));
cosnt char *src = stack + offset;
char *dst = stack + stack_size;
stack_size += 4;
dst[0] = src[0];
dst[1] = src[1];
dst[2] = src[2];
dst[3] = src[3];

So in debug I guess this is stuck being a few times slower (4 separate moves vs one. Same overhead to compute the array offsets), but still faster than a memcpy call (which adds the function call overhead plus the generic memcpy implementation will have a loop of some form over size) in release it would probably be optimised to be the same as memcpy.

EDIT 2: Actually not sure if in release the optimisation would be as good. It probably has to assume that dst and src overlap which gives you memmove, while the internal implementations of memcpy can assume it does not. Not sure if you can do any of those internal tricks within standard C/C++.

Juliean
Juliean

SyncViews said:
So in debug I guess this is stuck being a few times slower (4 separate moves vs one. Same overhead to compute the array offsets), but still faster than a memcpy call (which adds the function call overhead plus the generic memcpy implementation will have a loop of some form over size) in release it would probably be optimised to be the same as memcpy. EDIT 2: Actually not sure if in release the optimisation would be as good. It probably has to assume that dst and src overlap which gives you memmove, while the internal implementations of memcpy can assume it does not. Not sure if you can do any of those internal tricks within standard C/C++.

Yeah, I just checked out of interest. The results are pretty interesting indeed:

https://godbolt.org/z/5cYoGjTeh

This is the code without any optimizations! Interestingly, the memcpy is pretty much being compiled to the same operation that would occur if you did x = y, while your version, well :D Thats why they say you should always measure, don't they.

I mean, thats still pretty insightful. I wasn't aware that memcpy is that good even in debug. I knew memcpy with a fixed size is heavily optimized (as its an intrinsic instead of a regular function), but that means I the whole discussion is solved anyway (I'll have to check MSVC as well, but yeah). Only thing thats still pretty shitty is that std::bit_cast has so much more overhead then (maybe its a MSVC-thing as well). Sure, memcpy is then the right choice but its still a lot more to type then std::bit_cast would be (and whats with the zero-overhead thing we got going in C++, huh?)

SyncViews
SyncViews

When I looked on MSVC debug memcpy was a call into the generic size version, but maybe there is a flag to get it to use intrinsic optimisations while still keeping most debug functionality. With optimisations definitely expect it to be equal or likely better.

Interesting that GCC does that without enabling optimisations.

I suppose you could macro it, since using a template/wrapper just gives you the debug overhead back.

EDIT: And just to confirm my original thought on memcpy, this looks about as good on GCC and MSVC release build as it is going to get.
The debug builds do mess around with the stack though. https://godbolt.org/z/Mve4x36x4

Juliean
Juliean

SyncViews said:
When I looked on MSVC debug memcpy was a call into the generic size version, but maybe there is a flag to get it to use intrinsic optimisations while still keeping most debug functionality. With optimisations definitely expect it to be better.

Perhaps I saw the same thing. I know that you can set whether or not to use “intrinsic” version of certain functions (https://docs.microsoft.com/en-us/cpp/intrinsics/compiler-intrinsics?view=msvc-160)​,​ but thought that it would only affect optimized builds.

SyncViews said:
Interesting that GCC does that without enabling optimisations.

I mean, it kind of also makes sense in hind-sight. At least when you consider that memcpy is treated as an intrinsic, and how intrinsics are handled. Or, at least how I personally handle intrinsics in my own compiler. Without going into much detail, but I also have the concept of functions which appear normally (as a node in my visual language), but are not actually functions but code-generators (not to be confused with macros). And there, I would do the same thing: Instead of emitting a “call” to memcpy and deferring it to the optimizer to remove, you evaluate the parameters, see that the size is fixed, and just produce a “copy 4 bytes” ASM right then and there.
At least thats my educated guess based on what I'm doing myself now (I of course don't have a memcpy specifically; but you do see this at other points like for-loops where I pretty much just copied the algorithm from what GCC seems to do under debug-builds).

SyncViews said:
I suppose you could macro it, since using a template/wrapper just gives you the debug overhead back.

Yeah, thats what I would probably do. I already started putting some things in the interpreter into macros, which I don't like from a point of view of code-cleanlyness. But my old system was way too much into keeping code clean VS performance, and I'm paying the price now. I mean, perhaps if the 32x speedup that I saw keeps up for the final game (not 100% sure if its going to be more or less), I might be able to refactor things back a bit.

SyncViews
SyncViews

“memcpy is treated as an intrinsic” but isn't that entirely a concept that GCC made up as an optimisation? And it only does it for known small sizes, it calls a memcpy function otherwise? I didn't think the C standard gave memcpy any particular special treatment compared to say memset.

void zero_int(char *data)
{
    memset(data, 0, sizeof(int)); // optimised in -O1 and above, function call otherwise
}

EDIT:

Juliean said:
But my old system was way too much into keeping code clean VS performance, and I'm paying the price now. I mean, perhaps if the 32x speedup that I saw keeps up for the final game (not 100% sure if its going to be more or less), I might be able to refactor things back a bit

Well all of this was talking about debug. In release if it doesn't inline smaller templates and other wrapper functions something is up, normally I managed to get it to do so and once inlined it ends up almost the same as if the logic was their directly.

So using macros should just be to potentially make a debug build more usable if don't want to mix flags on compilation units or need some header-only stuff to be fast.

Juliean
Juliean

SyncViews said:
“memcpy is treated as an intrinsic” but isn't that entirely a concept that GCC made up as an optimisation? And it only does it for known small sizes, it calls a memcpy function otherwise? I didn't think the C standard gave memcpy any particular special treatment compared to say memset.

Might very well be so. I might be totally wrong here. From own example, since I work in a visual language where everything is a node, I personally needed the concept of an “intrinsic” node for things like ifs and loops or int-add anyways. So it would feel natural in that system to treat the memcpy just like another node for genering code directly based on a condition. In that sense it is certainly a sort of “optimization”, but one that is applied during initial code-generation. Maybe it doesn't make fully sense in a text-based language, and its just what I came up with because it is more natural when you are dealing just with node and not grammar.
I did see that GCC also did the same thing for loops/ifs, where unreachable paths were discarded even without optimization, but perhaps thats also just local to that compiler. I did check and see that indeed, MSVC does call “memcpy” in debug-builds. But i didn't get the pragma intrinsic to run in godbolt, so I have to check that in my own Visual Studio.

EDIT: pragma intrinsic doesn't seem to change anything in visual studio. Seems that I'm going to stick with reinterpret_cast for that platform - if it insists. Doing a macro seems to be the most sane thing here after all - then I can choose the safe, defined behaviour on platforms where it matters (and where things like memcpy might be zero-overhead even in debug), and everywhere else I can stick to more dirty tricks. Would even allow me to benchmark different things to see if it matters later, without changing all the code.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.