Inverse trigonometric functions are applied to a length ratio which is a scalar value; there is no definition for vectors / points! With your formula you would need to calculate a division of two vectors what also is not defined that way.

Looking at the illustration in the OP, you want to compute the angle between the principal **x** vector

**x** = [ 1 0 ]^{T}

and the vector **v** between point **p**_{1} and point **p**_{2}

**v** = **p**_{2} - **p**_{1}

The arccos computes the angle between the adjacent and the hypothenuse, where both of these legs are what you have given as **x** and **v**. Taking advantage of the dot-product

**v** . **x** = |**v**| * |**x**| * cos( <**v**,**x**> )

where the angle <**v,x**> is your theta, you get

<**v**,**x**> = arccos( ( **v** . **x **) / ( |**v**| * |**x**| ) )

Now one of the vectors, namely **x**, is a unit vector, i.e. has length |**x**| == 1, so that the formula is simplified to

<**v**,**x**> = arccos( ( **v** . **x **) / |**v**| )

Furthermore the y component of **x** is always 0 and its x component is always 1, so the dot-product is simplified to

**v** . **x** = v_{x} * 1 + v_{y} * 0 = v_{x}

and hence the formula becomes

<**v**,**x**> = arccos( v_{x} / |**v**| )

EDIT: The OP mentioned to compute "the smaller angle between the line and the x axis", which, following the more detailed requirements in one of the following posts, is too broadly interpreted by the above solution.

**Edited by haegarr, 26 February 2014 - 02:11 AM.**