So, after about twenty years of hobby game development experience and fourteen years as a professional developer, I've finally decided it was time really understand the math. I don't consider myself good at math, and I haven't studied math in school since I graduated in 1999. So I'm rusty.

I've been working through the exercises at the end of the chapter of this book and I'm struggling with cross product. I understand how to calculate it, and I understand what it's used for, but not necessarily the math behind it. I understand it like a tool, but not in theory. I'm trying to use the cross product to determine the angle between two vectors.

The first part of the book's exercise:

Given the two vectors:

a = (0, 1, 1) b = (0, -1, 0)

Compute the angle between them using the scalar product. That's easy:

2.356

The second part is what I'm struggling with:

Using those same two vectors, calculate the angle between them using the cross product. The book seems to indicate that the following formula is used:

|| a x b || = ||a|| ||b|| sin(theta)

It goes on to explain that's the same as this:

|| a x b || = ||a|| ||b|| sqrt(1 - (a dot b)**2)

But this isn't what I'm coming up with. I'm calculating the magnitudes of the two vectors as:

||a|| = sqrt(0*0 + 1*1 + 1*1) = 1.414 ||b|| = sqrt(0*0 + -1*-1 + 0*0) = 1

Cross product:

a x b = (a lot of typing) = (1, 0, 0) || a x b || = 1

Dot product:

a dot b = 0*0 + 1*-1 + 1*0 = 1

Then to plug the values into the formula:

1 = 1.414 * 1 * sqrt(1 - 1) = 0

Obviously this cannot be correct. I know there's some piece I'm missing, but I can't quite figure it out from the text. Could someone help me out here?

Wikipedia stated that the magnitude of the cross product of the unit vectors yields the sine, but even this isn't giving me the correct angle:

â = (0, 1, 1) / sqrt(0*0 + 1*1 + 1*1) = (0, 0.707, 0.707) b̂ = (0, -1, 0) / sqrt(0*0 + -1*-1 + 0*0) = (0, -1, 0) â x b̂ = (.707, 0, 0) sin(theta) = ||â x b̂|| = .707

So I thought inverse sine of .707 yield the actual angle, but it doesn't reproduce the same value I calculated with the scalar product (2.356):

arcsin(.707) = 0.785

But I have noticed that

arccos(-0.707) = 2.356

So what am I missing here? Please help!