Sign in to follow this  

OpenGL GL_INVALID_OP in fullscreen mode only

Recommended Posts

karwosts    840
I've had a simple game working fine, and have had no problems until I decided to switch it over to fullscreen mode. Unfortunately once I enable fullscreen mode I am starting to receive GL_INVALID_OPERATION errors every time I check glGetError. This doesn't happen in windowed mode, and I'm trying to understand what might be causing this to happen. The window goes into fullscreen mode just fine, and actually partially works, all of my fullscreen background quads are rendered and some of my background shaders are executing properly, but none of my meshes are appearing. So I know that OpenGL is functioning to some extent. Anyway that is all anecdotal, just really trying to figure out why I'm getting these errors. I'll paste my startup code and see if anyone notices anything.
int WINAPI WinMain(HINSTANCE hInstance, HINSTANCE hPrevInstance, LPSTR lpCmdLine, int nShowCmd)
	WNDCLASSEX windowClass;		// window class
	HWND	   hwnd;			// window handle
	MSG		   msg;				// message
	RECT	   windowRect;

	windowRect.left=(long)0;						// Set Left Value To 0
	windowRect.right=(long)WINDOWWIDTH;	// Set Right Value To Requested Width;							// Set Top Value To 0
	windowRect.bottom=(long)WINDOWHEIGHT;	// Set Bottom Value To Requested Height

	// fill out the window class structure
	windowClass.cbSize			= sizeof(WNDCLASSEX);			= CS_HREDRAW | CS_VREDRAW | CS_OWNDC;
	windowClass.lpfnWndProc		= MainWindowProc;
	windowClass.cbClsExtra		= 0;
	windowClass.cbWndExtra		= 0;
	windowClass.hInstance		= hInstance;
	windowClass.hIcon			= LoadIcon(NULL, IDI_APPLICATION);	// default icon
	windowClass.hCursor			= LoadCursor(NULL, IDC_ARROW);		// default arrow
	windowClass.hbrBackground	= NULL;								// don't need background
	windowClass.lpszMenuName	= NULL;								// no menu
	windowClass.lpszClassName	= "GLClass";
	windowClass.hIconSm			= LoadIcon(NULL, IDI_WINLOGO);		// windows logo small icon

	if (!RegisterClassEx(&windowClass)){
		DWORD err = GetLastError();
		return 0;


Looking good so far...
bool CreateGLWindow(WNDCLASSEX windowClass, RECT windowRect, HINSTANCE hInstance, HWND *hwnd){

	DWORD	   dwExStyle;		// Window Extended Style
	DWORD	   dwStyle;			// Window Style


	if (fullscreen)								// fullscreen?

		int monitors = GetSystemMetrics(SM_CMONITORS);
		dd.cb= sizeof(dd);

		std::vector<DISPLAY_DEVICE> devices;
		std::vector<DEVMODE> devmodes;
		int i = 0;

		DEVMODE dmScreenSettings;					// device mode
		dmScreenSettings.dmSize = sizeof(dmScreenSettings);	
		dmScreenSettings.dmPelsWidth = devmodes[1].dmPelsWidth;			// screen width
		dmScreenSettings.dmPelsHeight = devmodes[1].dmPelsHeight;			// screen height
		dmScreenSettings.dmBitsPerPel = devmodes[1].dmBitsPerPel;				// bits per pixel

		if (ChangeDisplaySettingsEx(devices[1].DeviceName,&dmScreenSettings,0, CDS_FULLSCREEN,0) != DISP_CHANGE_SUCCESSFUL)
			MessageBox(NULL, "Display mode failed", NULL, MB_OK);
			fullscreen = FALSE;	

	if (fullscreen)								// Are We Still In Fullscreen Mode?
		dwExStyle=WS_EX_APPWINDOW;					// Window Extended Style
		dwStyle=WS_POPUP;						// Windows Style
		ShowCursor(FALSE);						// Hide Mouse Pointer
		dwExStyle=WS_EX_APPWINDOW | WS_EX_WINDOWEDGE;			// Window Extended Style
		dwStyle=WS_OVERLAPPEDWINDOW;					// Windows Style

	AdjustWindowRectEx(&windowRect, dwStyle, FALSE, dwExStyle);		// Adjust Window To True Requested Size

	*hwnd = CreateWindowEx(NULL,									// extended style
		"GLClass",							// class name16829936
		"Isle",	// app name
		0, 0,								// x,y coordinate
		windowRect.right - windowRect.left,
		windowRect.bottom -, // width, height
		NULL,								// handle to parent
		NULL,								// handle to menu
		hInstance,							// application instance
		NULL);								// no extra params

This section is the only difference I have between fullscreen and windowed mode, just changing some Display Settings. I'm not real experienced here so its possible I'm doing something stupid. Calling CreateWindowEx fires off a CALLBACK MainWindowProc with message WM_SIZE, where I'm checking for opengl error. In windowed mode this passes just fine, while failing with GL_INVALID_OPERATION in fullscreen. Any ideas?

Share this post

Link to post
Share on other sites
szecs    2990
I don't understand this:
Anyway, could you post the actual change code? If it's just a ChangeDisplaySettingsEx, then I think you need a AdjustWindowRectEx too.

What happens if you start with fullscreen initially? (not just toggling to it).

Share this post

Link to post
Share on other sites
karwosts    840

Guess I should have explained that better. I'm using a two-monitor environment, and that is the device for my second monitor. Code could be incorrect, but I'm just cobbling together bits and pieces I've found from various sources. I get the same behavior if I use devices[0] too (main monitor).


Anyway, could you post the actual change code? If it's just a ChangeDisplaySettingsEx, then I think you need a AdjustWindowRectEx too.

I'm not sure I understand what 'actual change code' you're referring to. That is every line of code I use up through when I get the glError. I do have AdjustWindowRectEx too, its the last line before CreateWindowEx.


What happens if you start with fullscreen initially? (not just toggling to it).
I'm not sure how you would do this.

Thanks for looking.

Share this post

Link to post
Share on other sites
szecs    2990
Sorry, I guess it's morning.

I was referring to your change-to-fullscreen code. When you push the "change-to-fullscreen-key".

Or maybe I misunderstood something.

"This section is the only difference I have between fullscreen and windowed mode, just changing some Display Settings"

I don't understand this sentence. Clarifying "some Display Settings" would be useful.

Share this post

Link to post
Share on other sites
karwosts    840
Oh sorry. This is the initialization code for my program, I'm not changing it once its running.

Program opens with WinMain, WinMain calls CreateGLWindow, CreateGLWindow calls CreateWindowEx which causes WM_SIZE message where the opengl error is caught.

When I said the only difference between fullscreen and windowed mode, I meant that I'm selecting fullscreen or windowed mode by hardcoding variable 'fullscreen' to true or false. When I start defined to windowed mode initialization works fine, if I start in fullscreen mode the initialization is causing errors.

I said I was 'changing Display Settings' because I was calling the winapi function "ChangeDisplaySettings". Not actually changing anything once its running.

Share this post

Link to post
Share on other sites
szecs    2990
What happens if you remove the device-lister stuff, and just use hard-coded devmode settings for the fullscreen?

My window create for fullscreen:

You doesn't seem to use the dwExstyle (you pass NULL instead)

Share this post

Link to post
Share on other sites
szecs    2990
Maybe you miss this from the device lister stuff:
memset(&dm,0,sizeof(DEVMODE));//the same for other DEVMODE variables
dm.dmSize = sizeof(DEVMODE);//the same for other DEVMODE variables
There are some flags that need to be set to zero (or something else). I use zero.

Share this post

Link to post
Share on other sites
karwosts    840
Found the problem, I was calling setup projection on WM_SIZE event. For some reason when using WS_POPUP style on the first WM_SIZE event the OpenGL context is not ready, although when using the WS_OVERLAPPEDWINDOW style the context is ready on the first WM_SIZE event.

Anyway just threw an if(isInitialized) check around my projection setup and its got no problem now.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

Sign in to follow this  

  • Similar Content

    • By povilaslt2
      Hello. I'm Programmer who is in search of 2D game project who preferably uses OpenGL and C++. You can see my projects in GitHub. Project genre doesn't matter (except MMO's :D).
    • By ZeldaFan555
      Hello, My name is Matt. I am a programmer. I mostly use Java, but can use C++ and various other languages. I'm looking for someone to partner up with for random projects, preferably using OpenGL, though I'd be open to just about anything. If you're interested you can contact me on Skype or on here, thank you!
      Skype: Mangodoor408
    • By tyhender
      Hello, my name is Mark. I'm hobby programmer. 
      So recently,I thought that it's good idea to find people to create a full 3D engine. I'm looking for people experienced in scripting 3D shaders and implementing physics into engine(game)(we are going to use the React physics engine). 
      And,ye,no money =D I'm just looking for hobbyists that will be proud of their work. If engine(or game) will have financial succes,well,then maybe =D
      Sorry for late replies.
      I mostly give more information when people PM me,but this post is REALLY short,even for me =D
      So here's few more points:
      Engine will use openGL and SDL for graphics. It will use React3D physics library for physics simulation. Engine(most probably,atleast for the first part) won't have graphical fron-end,it will be a framework . I think final engine should be enough to set up an FPS in a couple of minutes. A bit about my self:
      I've been programming for 7 years total. I learned very slowly it as "secondary interesting thing" for like 3 years, but then began to script more seriously.  My primary language is C++,which we are going to use for the engine. Yes,I did 3D graphics with physics simulation before. No, my portfolio isn't very impressive. I'm working on that No,I wasn't employed officially. If anybody need to know more PM me. 
    • By Zaphyk
      I am developing my engine using the OpenGL 3.3 compatibility profile. It runs as expected on my NVIDIA card and on my Intel Card however when I tried it on an AMD setup it ran 3 times worse than on the other setups. Could this be a AMD driver thing or is this probably a problem with my OGL code? Could a different code standard create such bad performance?
    • By Kjell Andersson
      I'm trying to get some legacy OpenGL code to run with a shader pipeline,
      The legacy code uses glVertexPointer(), glColorPointer(), glNormalPointer() and glTexCoordPointer() to supply the vertex information.
      I know that it should be using setVertexAttribPointer() etc to clearly define the layout but that is not an option right now since the legacy code can't be modified to that extent.
      I've got a version 330 vertex shader to somewhat work:
      #version 330 uniform mat4 osg_ModelViewProjectionMatrix; uniform mat4 osg_ModelViewMatrix; layout(location = 0) in vec4 Vertex; layout(location = 2) in vec4 Normal; // Velocity layout(location = 3) in vec3 TexCoord; // TODO: is this the right layout location? out VertexData { vec4 color; vec3 velocity; float size; } VertexOut; void main(void) { vec4 p0 = Vertex; vec4 p1 = Vertex + vec4(Normal.x, Normal.y, Normal.z, 0.0f); vec3 velocity = (osg_ModelViewProjectionMatrix * p1 - osg_ModelViewProjectionMatrix * p0).xyz; VertexOut.velocity = velocity; VertexOut.size = TexCoord.y; gl_Position = osg_ModelViewMatrix * Vertex; } What works is the Vertex and Normal information that the legacy C++ OpenGL code seem to provide in layout location 0 and 2. This is fine.
      What I'm not getting to work is the TexCoord information that is supplied by a glTexCoordPointer() call in C++.
      What layout location is the old standard pipeline using for glTexCoordPointer()? Or is this undefined?
      Side note: I'm trying to get an OpenSceneGraph 3.4.0 particle system to use custom vertex, geometry and fragment shaders for rendering the particles.
  • Popular Now