Dawoodoz, those are implementation defined, not undefined or unspecified. Those words have meaning You are partly right that it is for performance, but the performance is so that you can choose something that fits the hardware rather than something that fits the language.
Early versions of Java suffered badly from that, with Java implementations being mixed between doing what the Java language specified for math operations (especially floating point), or doing whatever the hardware did even if it didn't quite match what the language specified. The Java language mandated that floating point needed exact reproducibility, which didn't exist on most FPUs; Java language handling of infinity and NaN results differed from IEEE hardware standards, and differences with IEEE floating point traps (invalid floating point operation, overflow, underflow, division by zero, and inexact results). Because Java over-specified the implementation of floating point, organizations in the late 1990s and early 2000s were forced to either use hardware-supported operations and fail Sun's Java certifications, or use slow software-based floating point computation and pass Sun's certifications. People doing advanced numerical processing were mixed on it, games developers hated the slow versions and wanted what the hardware already did.
In the case of size_t and similar types the underlying hardware and the implementation details define the number of bits, the sizes of data types, and more. C and C++ both specify minimum sizes, but leave the exact sizes up to the hardware for implementation details.
When I first learned to program, we had an old PDP machine that operated on 18-bit devices that used 9 bits, just like 16-bit devices often used 8 bits. C worked just fine on them, because C doesn't mandate specific sizes. If a language mandated that a byte held 256 values then a device that held 512 values couldn't be used. Similar, an int must be at least 65536 values, but holding 262114 values is also legal. The implementation defines the range.
Over the years I've worked with a bunch of hardware, including DSPs and FPGAs that did away with the old 8-bit concepts entirely. All the datatypes for char, short int, int, and long int were all 32 bit values. They are also covered by the language definitions, and 32-bit char is perfectly acceptable in the language standard. This means that sizeof(char), sizeof(bool), sizeof(char16_t), sizeof(char32_t) were all equal to 1.
Just like other sizes, size_t is an implementation defined type. It is defined everywhere, but the exact value is up to the compiler.
In the latest C it is defined as: "the unsigned integer type of the result of the sizeof operator".
In the latest C++ it is defined as: "an implementation-defined unsigned integer type that is large enough to contain the size in bytes of any object", with a recommendation that it be no greater conversion than a long int unless required to hold all values.
Any other use is outside the standard. It may be valid in one compiler implementation but not valid in another. That's part of the joy of using implementation defined behavior. It is up to the individual compiler implementation what it means, and it may or may not be the same on a different compiler.