The technical term for this is "endianness".
Ok, seems I made an embarassing mistake in understanding what little endian means
So in a little endian system, val = 0xAABBCCDD is stored in memory as:
0 1 2 4
DD CC BB AA
That would explain why vec[0] == 0xDD, etc. Thanks!