You seem to be using the coordinates of the screen directly, with (0,0) being the top-left corner. Notice that in those coordinates, an increase in y goes down, while the convention used in Math is that an increase in y goes up. This explains why you need to negate the sin(...) part. The sign of the cos(...) part should be positive, but your particular test is symmetric, so you haven't noticed the problem yet.

I will use traditional Math convention for everything. I will also assume the player is at (0,0) to simplify things. If your player is facing straight up, I guess player.rot = 90 degrees. You can draw an arrow of length 1 from the origin pointing in that direction, it will have coordinates (0,1). In general the coordinates are (cos(player.rot), sin(player.rot)). The length of 1 is arbitrary. If it helps you visualize it better, you can think of it as an arrow of length 100, which would have coordinates (0,100). In the end that factor of 100 won't matter. From now on, let this vector be (x,y).

Now I am going to draw a segment that represents the screen (I assume we are viewing the whole thing from the top of the scene). It will be a segment that is perpendicular to (0,1). You can find a direction that is perpendicular to (a,b) by computing (-b,a) (that's a rotation of 90 degrees counterclockwise). In our case, we'll get (-1,0). The segment I will draw will have (0,1) as the center and the left end will be at a point (0,1)+K*(-1,0). Reminder: That notation means (0+K*(-1),1+K*0) = (-K,1). The other end will be (0,1)-K*(-1,0), which is (K,1). I don't know what K is yet, but it will depend on FOV somehow.

So let's compute K. If you have been making a drawing of this (which you should), you can look at the triangle formed by the origin (0,0), the center of my "screen segment" (0,1) and the far right end of my screen segment (K,1). This triangle is rectangle at (0,1), and I am interested in the angle at the origin, which should be FOV/2 = 30 degrees. The tangent of that angle can be computed as "opposite side / adjacent side", which is K/1. That tells me that K/1 = tan(30 degrees) = 0.57735, and that's how I compute K.

Now my code would look something like this:

  double const distance_to_screen = 300.0;
  double look_angle = Math::to_radians(player.rot);
  // The next two definitions have sings different than the text, to																				
  // try to adapt to the convention that y increases going down.																					
  sf::Vector2f look_vector(distance_to_screen*std::cos(look_angle),
						   -distance_to_screen*std::sin(look_angle));
  sf::Vector2f left_vector(look_vector.y, -look_vector.x);

  double K = std::tan(0.5 * player.fov);
  sf::Vector2f far_left_end = look_vector + K * left_vector;
  sf::Vector2f one_column_increment = -2.0 * K * left_vector / w;

  sf::VertexArray ray(sf::Lines, 2);
  ray[0] = player;
  ray[0].color = sf::Color(255,255,255,30);
  ray[1].color = sf::Color(255,255,255,30);
  for (int i = 0; i < w; i++) {
	// Instead of adding +5 to x and y here, change your code so the																				
	// meaning of `player' is the center of the player																							  
	ray[1] = player + far_left_end + i * one_column_increment;
	rwind.draw(ray);
  }

I don't have a complete program to play with, so there might be mistakes in that code. If it doesn't work and it's not trivial to fix, I'll try to post a complete program, but this will probably need to wait until tonight.

You seem to be using the coordinates of the screen directly, with (0,0) being the top-left corner. Notice that in those coordinates, an increase in y goes down, while the convention used in Math is that an increase in y goes up. This explains why you need to negate the sin(...) part. The sign of the cos(...) part should be positive, but your particular test is symmetric, so you haven't noticed the problem yet.

I will use traditional Math convention for everything. I will also assume the player is at (0,0) to simplify things. If your player is facing straight up, I guess player.rot = 90 degrees. You can draw an arrow of length 1 from the origin pointing in that direction, it will have coordinates (0,1). In general the coordinates are (cos(player.rot), sin(player.rot)). The length of 1 is arbitrary. If it helps you visualize it better, you can think of it as an arrow of length 100, which would have coordinates (0,100). In the end that factor of 100 won't matter. From now on, let this vector be (x,y).

Now I am going to draw a segment that represents the screen (I assume we are viewing the whole thing from the top of the scene). It will be a segment that is perpendicular to (0,1). You can find a direction that is perpendicular to (a,b) by computing (-b,a) (that's a rotation of 90 degrees counterclockwise). In our case, we'll get (-1,0). The segment I will draw will have (0,1) as the center and the left end will be at a point (0,1)+K*(-1,0). Reminder: That notation means (0+K*(-1),1+K*0) = (-K,1). The other end will be (0,1)-K*(-1,0), which is (K,1). I don't know what K is yet, but it will depend on FOV somehow.

So let's compute K. If you have been making a drawing of this (which you should), you can look at the triangle formed by the origin (0,0), the center of my "screen segment" (0,1) and the far right end of my screen segment (K,1). This triangle is rectangle at (0,1), and I am interested in the angle at the origin, which should be FOV/2 = 30 degrees. The tangent of that angle can be computed as "opposite side / adjacent side", which is K/1. That tells me that tan(30 degrees)=0.57735, and that's how I compute K.

Now my code would look something like this:

  double const distance_to_screen = 300.0;
  double look_angle = Math::to_radians(player.rot);
  // The next two definitions have sings different than the text, to																				 
  // try to adapt to the convention that y increases going down.																					 
  sf::Vector2f look_vector(distance_to_screen*std::cos(look_angle),
						   -distance_to_screen*std::sin(look_angle));
  sf::Vector2f left_vector(look_vector.y, -look_vector.x);

  double K = std::tan(0.5 * player.fov);
  sf::Vector2f far_left_end = look_vector + K * left_vector;
  sf::Vector2f one_column_increment = -2.0 * K * left_vector / w;

  sf::VertexArray ray(sf::Lines, 2);
  ray[0] = player;
  ray[0].color = sf::Color(255,255,255,30);
  ray[1].color = sf::Color(255,255,255,30);
  for (int i = 0; i < w; i++) {
    // Instead of adding +5 to x and y here, change your code so the																				 
    // meaning of `player' is the center of the player																							   
    ray[1] = player + far_left_end + i * one_column_increment;
    rwind.draw(ray);
  }

I don't have a complete program to play with, so there might be mistakes in that code. If it doesn't work and it's not trivial to fix, I'll try to post a complete program, but this will probably need to wait until tonight.