No matter what I do, I can't figure out a way to convert the mouse position to a world coordinate. SDL_GetMouseState gives the window position, but doesn't take into account the logical size. It's got me completely lost and frustrated, I just keep thinking of a way to fix it, and completely mind blanking. I need to click a place on the window, convert it into a place within the 640x480 size I've set, then convert it into a world position, but I just can't convert the position down to the 640x480 size 

Not sure I understand your problem correctly, especially because I've not used SDL before. (I just built an example to see how it would work for this post) But... 

Based on your title: "SDL logicalrenderscale and the mouse" and searching the SDL API I'm going to assume you mean calling SDL_RenderSetLogicalSize.

If this is the case then it seems that the mouse position is disconnected from the logical scale. If we were to create a window of say 512x512. And we set the logical size to 200x200.

window = SDL_CreateWindow("title", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, 512, 512, 0);
SDL_Renderer* renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
SDL_RenderSetLogicalSize(renderer, 200, 200);

Then if we render a line like so:

while (!quitting) 
{
	SDL_Event event;
	while (SDL_PollEvent(&event)) 
	{
		if (event.type == SDL_QUIT) 
		{
			quitting = true;
		}
	}

	int x, y;
	SDL_GetMouseState(&x, &y);

	SDL_SetRenderDrawColor(renderer, 128, 128, 128, 0);
	SDL_RenderClear(renderer);

	SDL_SetRenderDrawColor(renderer, 255, 255, 255, 0);
	SDL_RenderDrawLine(renderer, 0, 0, x, y);
	SDL_RenderPresent(renderer);
}

Then I think I see the problem that you are describing.

If this is the case then it is important to understand that the rendering size is not connected in any way to the actual size of the window. This is usually done to make it easier to layout UI elements on a virtual desktop.

However if you want a mapping from one coordinate system to another then you need to do that your self. In the example above what this means is that you can remap the mouse window coordinate to say [0, 1) by doing float fx = float(x) / 512.0f; then you can remap it after that to the logical render size simply by multiplying that by the logical render size (in our example 200). So given that the new logic becomes:

int x, y;
SDL_GetMouseState(&x, &y);

int windowWidth, windowHeight;
SDL_GetWindowSize(window, &windowWidth, &windowHeight);
int renderWidth, renderHeight;
SDL_RenderGetLogicalSize(renderer, &renderWidth, &renderHeight);

float fx = float(x) / float(windowWidth) * float(renderWidth);
float fy = float(y) / float(windowHeight) * float(renderHeight);

SDL_SetRenderDrawColor(renderer, 128, 128, 128, 0);
SDL_RenderClear(renderer);

SDL_SetRenderDrawColor(renderer, 255, 255, 255, 0);
SDL_RenderDrawLine(renderer, 0, 0, int(fx), int(fy));
SDL_RenderPresent(renderer);

With this change the point (fx, fy) maps the window mouse coordinate into the logical render window.

I hope this helps.