Drag acts in the direction of the green arrow but drag isnt what makes the ship turn. The red arrow decompose into a drag force (green arrow) and a steering force (not in the picture, but should point to the right)

The rudder works like a wing, I guess you could call the red arrow the "lift" of the rudder

The rudder will work like a wing - which means:

1. Sometimes there will be laminar flow over it, and sometimes it will be stalled.

2. The forces on the rudder can be calculated as the sum of the lift force (which always acts in a direction perpendicular to the fluid flow relative to the rudder - so it's not the red arrow, but will act directly to the right in your diagram, given the blue arrow is pointing "up"), and the drag force (which always acts in a direction opposite to the fluid flow). These forces will be proportional to the square of the (local) fluid speed, the rudder area, and the coefficients of lift and drag.

You need to remember that you need to calculate the local fluid flow velocity at the rudder. When the boat is turning, this will not just be -v.

You can actually estimate these coefficient of lift (CL) and drag (CD) values pretty accurately - but you might need to do so over the whole 360 degrees - i.e. you need to know CL and CD as a function of angle of attack. I'd expect them to look rather similar to curves in air - so if you have something that looks like this it should work OK: http://www.aerospaceweb.org/question/airfoils/q0150b.shtml

Alternatively, if your boat has got a keel, when the fluid flow is significantly from the side, the effect of the rudder is probably minimal - in which case you only need to calculate the rudder force when the flow is laminar. For this CL is going to be directly proportional to alpha (the angle of attack) up to around +/- 15 degrees or so - then the rudder will stall and you can set it to zero. CD will be something like constant + alpha^2 up to the same angle (and beyond that you could clamp it).

This might all be overkill if you're not trying to do anything like a proper boat simulation :) If you want something simpler, just making the torque proportional to the rudder deflection and forward speed will probably work (assuming you have another rotational damping term that represents the reluctance of the boat (due to the keel) to turn when moving fwd/back).

The rudder will work like a wing - which means:

1. Sometimes there will be laminar flow over it, and sometimes it will be stalled.

2. The forces on the rudder can be calculated as the sum of the lift force (which always acts in a direction perpendicular to the fluid flow relative to the rudder - so it's not the red arrow, but will act directly to the right in your diagram, given the blue arrow is pointing "up"), and the drag force (which always acts in a direction opposite to the fluid flow). These forces will be proportional to the square of the (local) fluid speed, the rudder area, and the coefficients of lift and drag.

You need to remember that you need to calculate the local fluid flow velocity at the rudder. When the boat is turning, this will not just be -v.

You can actually estimate these coefficient of lift (CL) and drag (CD) values pretty accurately - but you might need to do so over the whole 360 degrees - i.e. you need to know CL and CD as a function of angle of attack. I'd expect them to look rather similar to curves in air - so if you have something that looks like this it should work OK: http://www.aerospaceweb.org/question/airfoils/q0150b.shtml

Alternatively, if your boat has got a keel, when the fluid flow is significantly from the side, the effect of the rudder is probably minimal - in which case you only need to calculate the rudder force when the flow is laminar. For this CL is going to be directly proportional to alpha (the angle of attack) up to around +/- 15 degrees or so - then the rudder will stall and you can set it to zero. CD will be something like constant + alpha^2 up to the same angle (and beyond that you could clamp it).

This might all be overkill if you're not trying to do anything like a proper boat simulation :) If you want something simpler, just making the torque proportional to the rudder deflection and forward speed will probably work (assuming you have another rotational damping term that represents the reluctance of the boat (due to the keel) to turn when moving fwd/back).

^this. you are correct in understanding the drag opposite velocity, but you need to account for the effect that force has on the rudder. the rudder directs the flow to the left, which creates an opposite force on the rudder, pushing the rear of the boat to the right. There is a ton of good info on the web about this stuff. I know that "https://www.amazon.com/Physics-Game-Developers-David-Bourg/dp/0596000065" does a decent job of describing the background math behind this problem, check your local library