Jump to content
  • Advertisement
Sign in to follow this  
Ryan_001

C++ constexpr challenge

This topic is 587 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

So been changing some of my older code to use constexpr where appropriate, to clean stuff up.  Came across something that I found to be rather difficult.  I needed to write a constexpr function that would take an object and swap 2 bytes in it.  So something like:

template<typename T> constexpr T SwapBytes(T x, size_t i, size_t j) {
 // swap the i'th byte in x with the j'th byte in x, return x
 }

For integer types or objects small enough to be converted into integers this is simple, but for any arbitrary type, this has proven to be a challenge.

 

Its not critical, just something I came across that proved to be more difficult than seemed at first, so I'm curious at how you guys would approach it.

Share this post


Link to post
Share on other sites
Advertisement

Not possible. In constant expressions, you don't have access to the bit representation of an arbitrary value. More specifically, you can't do anything that would have the effect of a reinterpret_cast.

 
Ahh, good to know.  I thought it rather funny that VS2015 compiles functions like the following just fine:

template<typename T> constexpr uint8_t GetByte(T x) {
	return *reinterpret_cast<uint8_t*>(&x);
	}

What I didn't realize is that it only works in 'run time', the results of the constexpr function above can't actually be used in constexpr situations (ie. to create an array).  Until you mentioned it I hadn't really given it much thought.  As to why I would want a function like that, I had no particular use for it.  I have a 'byte swap' template function, along with overloads with fast integer versions which I had converted over.  I was just changing some bit twiddling functions to constexpr (integer log base 2, one's count, base 2 rounding, etc...) so I could replace some of the template muckery with constexpr and came across the above problem.

Share this post


Link to post
Share on other sites

It's not very useful to need the first byte of an arbitrary object T, perhaps specialize to the object you need it from? For integers, a cast should work.

Share this post


Link to post
Share on other sites

It's not very useful to need the first byte of an arbitrary object T, perhaps specialize to the object you need it from? For integers, a cast should work.

 

Ya, sorry maybe I was unclear.  I just wrote that function to see if VS2015 allowed reinterpret_cast/pointer nonsense in constexpr.  It compiled and ran and so I didn't think twice until Oxyd mentioned it wasn't allowed.  I then realized the function would compile, but wouldn't execute in 'constexpr' mode, so I could get the 1st byte and print it to std::cout, but I couldn't assign it to a constexpr variable or use it to create a fixed size array.  The function of course has no real use.

Share this post


Link to post
Share on other sites

What I didn't realize is that it only works in 'run time', the results of the constexpr function above can't actually be used in constexpr situations (ie. to create an array).

Out of experience, I suggest being extra careful with that. You will notice that there are many more such little gotchas, and you might come to the conclusion that doing things the constexpr way "to clean stuff up" too hastily is not always a good idea. Maybe when C++20 will be out, constexpr will have sufficiently matured, who knows. As it stands, "cleaning stuff up" sometimes results in "fucking stuff up" in a non-obvious way. The most malicious thing about constexpr is that as long as you comply with the (sometimes arbitrary) restrictions, it "works fine", even when in reality it doesn't (that is, no compiler error, no warning, correct results, but surprise... also no compiletime evaluation).

Although I do understand why the initial wording for constexpr was just what it was -- some compiler writers were unwilling/unable to implement proper constexpr in a reasonable timeframe at that time, so "may" rather than "will be" was a necessity (for the same reason, the initial wording featured the single-return-statement) --- it's beyond my understanding why constexpr remains being needlessly restrictive on the one hand side, and maliciously deceptive on the other hand side almost a decade later, and there is no straightforward, usable way of communicating to the compiler your intent of what's supposed to happen, unless your intent behind calling the constexpr function is "yeah, whatever, I don't care, just gimme a result" (which, admittedly, is often the case... only just, not always!).
 
C++17 does, by the way, add if constexpr(cond), which is a massively awesome thing where applicable (that one really made my day when I read about it!), but in many cases, constexpr remains being a false friend.

In particular, be prepared that calling your nice little harmless constexpr functions will evaluate at compile time, every time, unless where you very painfully force the compiler on a 1:1 base to do it at compiletime. And even then the compiler will, again, evaluate your functions (with the same, identical, constant parameters) at runtime on the next call, even though it already knows the by all means fucking unchanging constant result of the function's evaluation. Thing is, you don't even notice that this is happening, unless either you look at the disassembly or it's happening a few hundred thousand times per frame, and you suddenly wonder why everything is so darn slow for no obvious reason. All you did is replace an ugly macro-template hack with a constexpr function, which is so much cleaner and nicer, and the exact same thing, a compiletime constant, right?
 
I'd really wish you could for example declare a function like so:
  [[strictly]] constexpr int foo(int bar){...}
  constexpr constexpr int foo(int bar){...}     // alternative
where evaluating the function at runtime is not allowed (and by consequence, calling the function with a non-constexpr argument is a hard error). Being able to call the same function with constant and not-constant values and "things just work" is seductive, and sometimes that's just what you want. But sometimes, when you use a word like "constant", then you actually have a good reason to say so, and "not constant" is not a valid option. But alas... we don't have that.
 
So... bottom line: Unless you don't care, be extra careful, and test, do not trust, that the compiler does exactly what you expect.
 

Memory addresses in constexpr context seems very weird to me at least, I would assume nothing has a memory address until runtime.

This is very true.

 

Which is why I don't understand the restriction on reinterpret_cast. It seems like either it should not be allowable at all to have a pointer type that is constexpr, or you should be allowed to write something like constexpr foo* invalid_object = reinterpret_cast<foo*>(-1); on a "trust me, I know this is not a valid pointer" base. But for some reason, one is allowed, and the other is not.

 

You see, if a constexpr function does something, anything, with a pointer, or returns a pointer type of sorts, then this is necessarily a "trust me, I know what I'm doing" kind of thing because there is no way you could know a pointer's address at compiletime (with very few exceptions, nullptr being one). However, it sometimes makes sense, such as for example returning (void*)0  or (void*)-1 as a kind of special value (which not few C APIs, including operating system APIs do, by the way). Or, returning the address of a sentinel object with static storage duration. Of which of course you have no way of knowing the address, but the compiler will choose one (so it does "know" it, and sure enough it is a compiletime constant!), and the only thing you want to do in your program is check that a particular result is not the sentinel object. Sentinels can be an effective strategy of eliminating bounds checks. Conceptually (although with an integer, not a pointer), the standard library does something very similar with e.g. returning  npos as "not found", too.

Edited by samoth

Share this post


Link to post
Share on other sites

Which is why I don't understand the restriction on reinterpret_cast. It seems like either it should not be allowable at all to have a pointer type that is constexpr, or you should be allowed to write something like constexpr foo* invalid_object = reinterpret_cast<foo*>(-1); on a "trust me, I know this is not a valid pointer" base. But for some reason, one is allowed, and the other is not.
 
You see, if a constexpr function does something, anything, with a pointer, or returns a pointer type of sorts, then this is necessarily a "trust me, I know what I'm doing" kind of thing because there is no way you could know a pointer's address at compiletime (with very few exceptions, nullptr being one). However, it sometimes makes sense, such as for example returning (void*)0  or (void*)-1 as a kind of special value (which not few C APIs, including operating system APIs do, by the way). Or, returning the address of a sentinel object with static storage duration. Of which of course you have no way of knowing the address, but the compiler will choose one (so it does "know" it, and sure enough it is a compiletime constant!), and the only thing you want to do in your program is check that a particular result is not the sentinel object. Sentinels can be an effective strategy of eliminating bounds checks. Conceptually (although with an integer, not a pointer), the standard library does something very similar with e.g. returning  npos as "not found", too.


I think the general point here is that the compiler has to essentially contain an interpreter of C++ in order to evaluate constant expressions at compile-time. And, in general, the compiler could be running on a different architecture than it's generating code for. So, if you were able to examine the bit representation of objects – such as pointers –, the compiler would not only have to be able to interpret stuff, it would also have to interpret stuff as though it ran on the target architecture, as opposed to interpreting it on the architecture it's actually being run on.

You can already see this in compile-time evaluation of floating-point expressions: It is permitted for a constexpr evaluation of a floating-point expression to give a different result than a run-time evaluation of the same expression would give. This is to ease the burden on compilers by not requiring them to exactly replicate the behaviour of the target CPU and instead allow them to use the host CPU for evaluating floating-point expressions.

So, it seems to me that the choice was between 1) we don't allow examining bit representations at compile-time at all; 2) we allow the results of examination of bit representations to differ between run-time and compile-time; or 3) we burden the compiler with exactly emulating the target architecture. You're free to argue that you would've preferred options number 2 or 3, but in the end option number 1 is what we have to deal with.

This is all a speculation on my part, though. If someone is willing to dig up the exact rationale behind this, that would be nice.

Share this post


Link to post
Share on other sites
So, it seems to me that the choice was between 1) we don't allow examining bit representations at compile-time at all; 2) we allow the results of examination of bit representations to differ between run-time and compile-time; or 3) we burden the compiler with exactly emulating the target architecture. You're free to argue that you would've preferred options number 2 or 3, but in the end option number 1 is what we have to deal with.

 

I don't think thats the case at all, since the exact value for objects to be used with constexpr-function has to be known at compile-time anyways. Ok, I might misunderstand something about the details of the implementation, but have a look at how it works:

struct Foo
{
    constexpr Foo() :// required, otherwise you cannot create a constexpr with Foo
		a(5), b(0.75f)
    {
    }
	
	int a;
	float b;
}

constexpr Foo test; // mouse-over gives: = {5, (0.75f)}

So it appears that the objects representation is already set when you create the constexpr-variable, which would make sense to me (unless I misunderstand something about compile-time optimizations). So there shouldn't be a specific issue with that which would disallow bitwise manipulation of complex objects. (Note that you absolutely do have to use a constexpr-variable with a constructor-constructor if you want to have a struct/class work within a constexpr-function).

 

I think its more of a limitation in the constexpr-syntax - taking an adress of a constexpr-variable at compile-time makes no sense, since every single value inside the constexpr-function has to be a compile-time constant:

template<typename T>
const char* test(T t)
{
    const char* p = (const char*)&t; // what numeric value would "p" have? whats the address of the compile-time constant t?

    return p;
}

So in order to allow something like that, they would need to introduce special syntax/functions for bitwise-access of compile-time constant-complex objects, which I don't think there is a good need for (OP even said his example was purely artifical).

 

 

 

Out of experience, I suggest being extra careful with that. You will notice that there are many more such little gotchas, and you might come to the conclusion that doing things the constexpr way "to clean stuff up" too hastily is not always a good idea. Maybe when C++20 will be out, constexpr will have sufficiently matured, who knows. As it stands, "cleaning stuff up" sometimes results in "fucking stuff up" in a non-obvious way. The most malicious thing about constexpr is that as long as you comply with the (sometimes arbitrary) restrictions, it "works fine", even when in reality it doesn't (that is, no compiler error, no warning, correct results, but surprise... also no compiletime evaluation).

 

That really is a major annoyance. It took me some time to figure out why I could not get constexpr ctors to work, turns out you have to have them defined inline. Oh, thanks compiler for letting me know... oh wait you didn't, you just pretended everything was right except you didn't produce any compile-time constants. sigh.

 

I think thats a good reason for declaring all your constants "constexpr" instead of "const" even though both produce identical results in most simple cases. Just that constexpr will actually complain when the expression cannot be evaluated to a compile-time constant (at least when declaring variables).

Edited by Juliean

Share this post


Link to post
Share on other sites

To add a recent experience on what a piece of shit constexpr is and how reluctant you should really be:

 

Been using the std::array with std::integer_sequence approach to do constexpr compiletime hashes for a long whole while... until today. Which, once you grok this perverse approach necessary to work around the deliberate quirks built into the language, indeed turns out being quite straightforward. If you can agree with the paradigm, the code is even readable, and of course it's so much nicer blah blah, and works fine, and is indeed constexpr by all means.

 

Until you discover by accident that, again, the compiler is being deliberately fraudulent. Grrrrr. Everything is fine for string literals up to, and including 19 characters. Which is great because as everybody knows, all names are 10-12, at most 15 characters long. Until, by accident, you have a name with 20 characters. Bummer, those exist too?

Instead of an integer, the compiler stores the character string in the executable and hashes it at compiletime. Now, if you happen to have a character literal with 21 characters as well (oh come on, those really don't exist, do they!), the compiler creates two separate functions, since they're instantiations of a variadic template function, and the parameter pack expansion gives a different number of parameters! Go figure what happens for 22 or 23 characters.

 

What's really the worst thing about it is the outright malicious way in which you are being cheated. It looks like it works fine and as intended, and it does, too... until it doesn't. And unless you either open the binary in a hex editor or step through the program in the debugger for an unrelated reason and stumble upon something like  callq 0x401cb0 <hash<20ull, 0ull, 1ull, 2ull, 3ull, 4ull, 5ull, 6ull, 7ull, 8ull, 9ull, 10ull, 11ull, 12ull, 13ull, 14ull, 15ull, 16ull, 17ull, 18ull, 19ull>(char const (&) [20ull], std::integer_sequence<unsigned long long, 0ull, 1ull, 2ull, 3ull, 4ull, 5ull, 6ull, 7ull, 8ull, 9ull, 10ull, 11ull, 12ull, 13ull, 14ull, 15ull, 16ull, 17ull, 18ull, 19ull>)> you never find out.

 

(Note that the code is being compiled with -O4, and clang is being invoked with -fconsexpr-depth=2500 -fconstexpr-steps=1000000 so it's not like doing 20 iterations should run against a configured limit, nor is there an excuse like "oh but it's debug build, not optimized".)

 

That does it. Back to using macros... at least they do what they promise.

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!