Sounds like you expect the look vector to be the direction you want to view in but it is in fact the point you want to look at. I don't know if that's the case or not since I don't know the particular function you use, but that is how gluLookAt, and most other look at-functions i have seen, works.

Oh snap, you are right, I am passing in my direction vector into the values of lookX, lookY, and lookZ. And I'm passing in my position vector to the values eyeX, eyeY and eyeZ.

So I just changed it to
[source lang="java"]Matrix.setLookAtM(mViewMatrix, 0, eyeX, eyeY, eyeZ, eyeX + lookX, eyeY + lookY, eyeZ + lookZ, upX, upY, upZ);[/source]

And it works now

Sounds like you expect the look vector to be the direction you want to view in but it is in fact the point you want to look at. I don't know if that's the case or not since I don't know the particular function you use, but that is how gluLookAt, and most other look at-functions i have seen, works.

Oh snap, you are right, I am passing in my direction vector into the values of lookX, lookY, and lookZ. And I'm passing in my position vector to the values eyeX, eyeY and eyeZ.

What must I do to correctly set my view matrix given a direction vector, up vector, and position vector?