I agree with you about references. You can define arguments as references and then mutate the parameters in ways that are unexpected. References is usually only a syntactic sugaring.
That's completely incorrect. The use of references, and even more importantly 'const', states a contract between the function and the caller that defines if and how the function may modify those parameters. Yes, it's often possible in C++ to wiggle out of those contracts by abusing casts, but just because no contract can force a party to act in good faith does not mean that the contract itself is not important. In C++ the contract is important because the compiler will help you to inadvertently violate it (violating the contract usually requires tricky syntactic gymnastics), and also helps the compiler to understand the code more-fully by precluding certain possibilities, the result of which is that the compiler can often make more-aggressive optimizations.
There is one way I use references now and then: When I find optimization is needed and arguments are objects of a certain size. For example, sending a 4x4 transformation matrix as argument. In this case, declaring the argument as "const matrix4x4 &transform" will give the speed of a pointer with the syntax of a normal object. This can be done with pointers also (const matrix4x4 *transform), but then you have to change all calls to the function. And it gets messy if you want to use overloaded functions defined for the type (like projectionMatrix * viewMatrix * modelMatrix * vertex).
References should always be preferred over pointers, except where pointers are necessary (dynamically-allocated memory or other late-binding, null values, pointer arithmetic, interacting with legacy interfaces that use pointers). When you do need a pointer, the standard library provides a number of smart-pointers that should be preferred over home-grown pointer classes (I refuse to call them "smart" pointers, as home-grown pointer classes are almost universally broken in ways both obvious and not) or raw pointers. With C++ 11, just about every need you'd have for a pointer, aside from ones to deal with legacy interfaces, is met by one of the pointers in the standard library.
'const' should be applied appropriately regardless of whether the parameter is passed by value, pointer, or reference.