I would suggest reading up on raster display devices. A lot of the terminology comes from the days of CRT displays, but most of it still applies to LCD displays.

The display adapter (on your video card) is configured to send frame information at a set refresh rate. For normal windows desktop usage this refresh rate is configured as part of your display settings, for fullscreen D3D applications the refresh rate is specified when you create a swap chain for an adapter. Once the refresh rate is configured, the adapter will send frames to the display at the agreed upon rate. The data is sent one scan line at a time, until the the bottom line is sent and the image is complete. Once complete, the display is now in its VBLANK period. The scanline data that's sent to the display comes from an area of memory on the video card called the frame buffer. Typically you have 2 frame buffers for a double buffering setup, where one buffer (the back buffer) is written to by the GPU while the other (the front buffer) is being read and sent to the display. At the end of each frame once you've finished rendering the back buffer, you'll "flip" the buffers and the back buffer will become the front buffer (and vice versa). If this flip happens during the VBLANK, everything is fine: you'll see one image for an entire refresh period of the display and then you'll see the next image for the next refresh period. However if you flip outside of the VBLANK while the scanlines are being sent to the display, you'll get a "tear". This is because some of the lines from the previous frame will have been sent to the display, and then suddenly the scanlines switch to the next frame.

Now what VSYNC does is it causes the GPU to wait until the VBLANK period before flipping. This ensures you have no tearing, since you always flip in the VBLANK. The downside is that you might have to wait a while for the next VBLANK, especially if you just missed it. So for instance if your refresh rate is 100Hz your VBLANK will be every 10ms. So if it takes you 10.1ms to finish rendering a frame, you'll have to wait until the next VBLANK which means it will have effectively taken you 20ms to display that frame. If VSYNC had been disabled, you would just get a tear at the top of the screen instead. This is why frame dips are more jarring when VSYNC is enabled.

When you're not in fullscreen mode, things are different because the your D3D/OpenGL adapter won't have exclusive access to the display. Instead it might go through a desktop manager, depending on the OS you're using. On Windows Vista/7/8 with desktop composition enabled, the GPU will present the back buffer to the DWM instead of the display, and then the DWM will composite the image with the rest of the desktop. This is why you don't get tearing in windowed mode, even if VSYNC is disabled. Enabling VSYNC will still limit your app to the refresh rate, however the behavior will be different.