To further elaborate:
max() returns whichever of the two parameters are larger.
min() returns whichever of the two parameters are smaller.
p.x and p.y are the location, in pixels, of the mouse on the screen.
The screen's width is 800, and the height is 600.
p.x = max(min(p.x, 800), 0); //If 'p.x' is greater than the screen's width, it returns the screen's width (just to keep it in bounds).
p.x = max(min(p.x, 800), 0); //If 'p.x' is less than 0, it returns 0 (just to keep it in bounds).
And the same with 'p.y' and the screen's height.
float x = (p.x - 400) / 800.0f; //Subtracts half the screen's width from 'p.x', to put 'p.x' in a range from -400 to 400 with 0 being the center of the screen.
float x = (p.x - 400) / 800.0f; //Divides it by 800, to convert the range from "-400 to 400" to "-0.5 to 0.5".
And the same with 'p.y'...
D3DXVECTOR3 lookAt(x, 1.5f - y, -1.0f); //Creates a 3D math vector for Direct3D. Based off the name of the vector, it'll probably be used to reorient the camera to look in that direction.