Actually the topic name should be "How to profile code".

Well I am writing SIMD math library. I got 2 implementations SSE and scalar.

I'm not shure how measure the code speed. Currently Im not using optimization, and no debug symbols are generated for profiling.

I'm creating a loop that repeats the operation...

The compiler is cl

I'm expecting SSE dot product to be slower than scalar version?

But the cross product is also slower!?!@

SGE_FORCE_INLINE SGVector vec3_cross(const SGVector& a, const SGVector& b)
{
#if defined(SGE_MATH_USE_SSE)
__m128 T = _mm_shuffle_ps(a.m_M128, a.m_M128, SGE_SIMD_SHUFFLE(1, 2, 0, 3)); //(Y Z X 0)
__m128 V = _mm_shuffle_ps(b.m_M128, b.m_M128, SGE_SIMD_SHUFFLE(1, 2, 0, 3)); //(Y Z X 0)


//i(ay*bz - by*az)  + j(bx*az - ax*bz)  + k(ax*by - bx*ay)
T = _mm_mul_ps(T, b.m_M128);//bx * ay, by * az, bz * ax
V = _mm_mul_ps(V, a.m_M128);//ax * by, ay * bz, az * bx
V = _mm_sub_ps(V, T);


V = _mm_shuffle_ps(V, V, SGE_SIMD_SHUFFLE(1, 2, 0, 3));
return SGVector(V);
#else
const float x = (a.y*b.z) - (b.y*a.z);
const float y = (b.x*a.z) - (a.x*b.z);
const float z = (a.x*b.y) - (b.x,a.y);


return SGVector(x, y, z, 0.f);
#endif
}

Currently Im not using optimization

Ah? If you want to measure performance you need to compile using optimizations. at least /O2, otherwise every instruction will go through memory.

The best tool for profiling is Intel's VTune, they have 30 days free trial. Visual studio comes with a profiler you can use as well. If you want to profile yourself, you can use the timestamp counter (rdtcs instruction, visual studio has an intinsic).

__forceinline is very good for performance, I use it almost everywhere in performance critical code.

The problem is probably the shuffles. Not sure what CPU you are using, but shuffle performance is limited. You have 3 shuffles and 3 math instructions, which is not a good ratio. The SSE math instructions have data dependency on the shuffles, which will stall the CPU pipeline.

Also, your inputs and outputs are structs. Even though you pass them by reference, both shuffles and return values will got through memory (probably, need to look at the generated assembly).

To maximize SSE performance you need to design your whole code around it. Have long SSE math sequences, and reduce the amount of memory acceses and shuffles.

BTW, you can tell CL to use scalar SSE for floating-point instructions by using /arch:SSE. Check this.

Don't profile code, profile apps. Pass vectors by value, not reference (although that only works for the first 3 or so args, the rest will need to be by reference). Forceinline is just stupid. Stop using it. Stop constantly assigning things to V & T - It just introduces dependency chains for no good reason. Don't fear the use of multiple variables - they'll just end up as registers instead...

static const int	swizzle1	=	_MM_SHUFFLE( 3, 0, 2, 1 );
static const int	swizzle2	=	_MM_SHUFFLE( 3, 1, 0, 2 );

__m128	v1	=	_mm_shuffle_ps( lhs, lhs, swizzle1 );
__m128	v2	=	_mm_shuffle_ps( rhs, rhs, swizzle2 );
__m128	v3	=	_mm_shuffle_ps( lhs, lhs, swizzle2 );
__m128	v4	=	_mm_shuffle_ps( rhs, rhs, swizzle1 );

__m128	p1	=	_mm_mul_ps( v1, v2 );
__m128	p2	=	_mm_mul_ps( v3, v4 );

__m128	result	=	_mm_sub_ps( p1, p2 );

Well, there are many good, sensible points here. And I've heard all of them before. And they are not true...

__foceinline ...can blow out your code locality, instruction-cache, and the very small cache of decoded instructions too

That's true. But so does loop unrolling. That can't be the basis for dismissing __forceinline, it's just another factor to take into consideration. I've never encountered any I$ problems, and I put __forceinline on a 40-lines-of-code-functions more than once.

the compiler is in a much better position than you to determine whether to inline a particular call-site or not

Not always true, not even with the new, much improved VS12 and Intel Compiler 13. Compilers are not, and probably will never be perfect, especially when they need to support a wide variety of CPUs. Sometime you do know better, so why not help the compiler?

Forceinline is just stupid

Hmmm... That stupid thing gave me very nice performance improvments.

static const int swizzle1 = _MM_SHUFFLE( 3, 0, 2, 1 ); 
static const int swizzle2 = _MM_SHUFFLE( 3, 1, 0, 2 ); 
__m128 v1 = _mm_shuffle_ps( lhs, lhs, swizzle1 ); 
__m128 v2 = _mm_shuffle_ps( rhs, rhs, swizzle2 ); 
__m128 v3 = _mm_shuffle_ps( lhs, lhs, swizzle2 ); 
__m128 v4 = _mm_shuffle_ps( rhs, rhs, swizzle1 ); 
__m128 p1 = _mm_mul_ps( v1, v2 ); 
__m128 p2 = _mm_mul_ps( v3, v4 ); 
__m128 result = _mm_sub_ps( p1, p2 );

That's even worse that the original post. You are assuming that this code will be used sparsely, and that the output of this sequence will not be needed immediatly. Relying too much on the compiler and CPU OOO is one of the main reasons for not-optimal-SSE-performance. If you are sure your assumptions are true, than that code is fine. If you are implementing a library to be used by others - not so good.

Stop constantly assigning things to V & T - It just introduces dependency chains for no good reason

As V&T are local variables, the compiler will alias them using other registers. The dependency chain comes from the fact that output of an instruction is input to another instruction, has nothing to do with the amount of local variables.

I define a new __m128 at each step, the compiler optimizes better due to that, usually

Again, variable aliasing is from Compiler101, it's really one of those places that compiler does very good on its own.

It does make the code much more readable, though.

VC 2012 and Clang though... it is REALLY impressive to see the compilers doing such things

So true...

Other things:

1. If you are using a CPU with AVX, use the '/arch:AVX' compiler flag. It will magically convert your SSE code to use AVX128 instructions. AVX instructions have non-destructive-destination-register, which translates to better register utilization and better performance.

2. 64-bit application have access to more registers(16 instead of 8). The drawback is larger code size, but the extra registers more than make up for that. Compile to 64-bit if you can.

Now, before everybody starts thumbing me down - my job for the past 9 years was to the squeeze the hell out of Intel's CPUs, especially using vectorization, so I have some experience with the subject.

As a final note, there are a lot of fine details when doing performance optimizations. There's a lot of trial-and-error involved, a lot of profiling, manually inspecting the assembly, scratching your head not understading why things don't work as expected...

But most importantly, it's not black-and-white, there are no absolute truths. Getting to 70% of the optimal performance is easy. The performance cookbook will help you get to 90%. It's the last 10% that makes perf-opt so challenging (and so rewarding...).

Pass vectors by value, not reference

Hi Rob,

Why do you recommend passing vectors by value rather than reference?

I've made it a habit to use const ref parameters wherever possible, and wherever ref makes sense
compared to the size of the type (I wouldn't pass a char by reference)