Templates are the types you see with dimension modifiers. For example, float4 (a vector) and float3x4 (a matrix) are template types. Basically, they just allow you to specify a base type and the dimensions of it.

Components are the unmodifiable base types. For example, int, float, and bool. Components are used in templates as the base type. For example, float4x4 uses a component type of float with dimensions 4x4.

The docs do a good job at illustrating this.