Definitive perfect resolution-independent camera (using 2D raster graphics)

Started by
9 comments, last by Finalspace 6 years, 8 months ago

Hello forum people. So after years of being an independent developer with a degree in computer programming, I can safely say that I can't find a single resource that thoroughly scours and glosses over every single aspect of game resolution. I have loads of books on the subject, coding knowledge, and a repertoire of games I've made in the past - but these projects have always used fixed resolutions or workarounds to support other resolutions.

After posting on various forums and asking around, people seem to just redirect me to resources about resolution which are still incredibly lacking and bare-bones, not teaching me more than I already know.

Since resolution is such a huge and inherent part of game design, and since there is not a single game **EVER** developed that doesn't use a resolution, I am so surprised that there seems to be such a lack of material on the subject! It is a fundamental part of every single game in history.

So this post will be asking a load of questions all about resolution. What I am also proposing is that if I can gain enough ground on the subject, I'd like to create my own video/page/tutorial which brushes over every single aspect of game resolution, aspect ratios, etc.

**So with the introduction out of the way, here is my current project and the tasks I could use help with:**


----------


So I'm currently working solo on a 2D platformer, called DEA, which uses pixel art for the main game stage, and a mixture of raster-based (but not "pixel looking") assets for the far background. The genre/style of the game isn't important, but if you're interested, the blog link is here:

DEA Game Blog


Anyway, I wanted to just go through how the camera works and then what doesn't work and should work.

Firstly, my camera does not scale the main stage based on resolution, so increasing or decreasing the resolution simply increases the camera size and gives a bigger or smaller view of the game world, centering on the player. Below is a hypothetical screenshot at a lower resolution:

smallres.png.e3c093d4dbdfcd85d783ed076c2db9b1.png

and shown here the same still of the game at a higher resolution:
medres.png.3c2546e5f4c01dfdaf6c721df33fbad2.png

 

Now the forum formatting actually scaled the images and made them look out of proportion, but they remain the same level of zoom, and no images are scaled. I believe that the Terraria developers take the exact same approach to their game resolution, giving players with higher resolution monitors an advantage. This essentially means that at higher resolutions, a bigger slice of the game world is basically displayed on-screen, like so:
EXAMPLE1.thumb.png.e27d29b818a2f51fcd555ff9bc06c535.png

However, because of the nature of Terraria's genre, it doesn't seem to be a huge problem. My game has a competitive multiplayer element, but it's couch co-op without online play, so people with better monitors won't be given a competitive edge.

From my understanding, games that use vector-based graphics (or even rasterised graphics with high resolutions) don't give a larger screen view. Instead, they simply scale all of the graphics to always fit within the same proportions regardless of resolution.

My understanding of this is like so:
EXAMPLE2.thumb.png.d75967da77a24e4a9dbdba392ac536e6.png

So with this, the game is displayed in the same proportion on every resolution, but a higher resolution simply gives a much crisper image, meaning more "image pixels" are displayed per "screen pixels". This is nice, because regardless of screen monitor size, everyone will get the same experience.


----------


So from my understanding, is it easy to change the zoom levels to allow for this proportioning when using vector-based graphics, because the graphics will scale accordingly, and maintain a "clean" look when stretched and transformed. Pixellated graphics or "8-bit" style graphics can't be stretched using float values because of their strict 1x1 pixel ratio, so when applying a zoom effect to games using pixel art, the zoom has to be in uniform whole numbers (i.e. 1x, 2x, 3x, 4x, etc). I've detailed this out below with examples from my game:
EXAMPLE3.thumb.png.fc97c832fc9e66de0fd6655f5037bb06.png

This means that we can't recalculate sizes of pixellated sprites to accommodate every resolution, surely? It seems that the actual viewport size of the camera can be changed, and zoom can be manually changed (maybe in the options menu) in whole number intervals. However, is there no truly "smart" way of handling resolution in games that use pixel art?


----------


This brings me to my next problem. I've seen that a lot of modern "retro-style" games use other tricks to handle varying resolutions. The most common I see is the toss-up between "true" resolution and letterboxing, giving the player the option to either play the game in a distorted-looking aspect ratio, or to play at the intended aspect ratio, but to have the game "letterboxed" to look like a widescreen film. As an example, I tried loading up Owlboy (amazing game btw) and went straight to the options:

EXAMPLE4.thumb.png.f20d7c04573eb45bed2de8855567f342.png

The first thing that I noticed is that if I enable letterboxing or disable it, nothing actually happens. My monitor resolution is 1920x1080, and the game seems to run in that resolution, as the Steam overlay fits in the clearest and densest resolution. The graphics seem to be scaled absolutely perfectly with no distortion. My only guess as to why this is happening is that the game was developed with resources for a 16:9 aspect ratio, meaning that any resolution of the same aspect ratio will not distort the image, and any resolution of any other aspect ratio will adopt letterboxing to maintain the same ratio.

My understanding of this is as follows, but I'm not sure if this is correct:
EXAMPLE5.thumb.png.a978c3316742c51026c9f69b17b13892.png


Judging by all of the complications and horrors that come with managing resolution-independence, the Terraria-esque approach seems to be the easiest by far, giving players the option to choose a "zoom" level, and allowing the resolution to represent the amount of screen coverage. However, this seems to leave on huge, gaping problem in the game's design - A good rule of game design is to direct the player's attention using correct framing. You want to draw the player's attention in a certain direction using lighting, colours, movement, etc, but if the level is all displayed, this really draws this power away from the creator. In addition, if the resolution is too low, this will lead to the player maybe not having enough time to react to an upcoming trap or enemy, as their screen view doesn't encapsulate the thing ahead. Finally, on the converse, players playing on high resolutions will simply be able to see everything ahead of them, removing any element of surprise from your level design.

Should resolution really be this complicated? I feel like it's melting my brain into mush. Also, I haven't even begun to start with the problems that arise with parallax scrolling...

With my huge, confusing rant in mind, my questions are:

  1. Is there a smart way to handle aspect ratios with pixel art? It seems like art can only be stretched in a 1x1 ratio and zoomed in intervals of whole numbers, meaning that every resolution cannot be "truly" supported.
  2. Is it better to create art for the most common aspect ratio (such as 4:3 or 16:9)? Would this be neglecting other audiences with different aspect ratios, or would letterboxing fix this issue for different aspect ratios?
  3. Is the "Terraria" way of doing things much simpler and more accessible to all developers? If so, how would one go about "framing" a scene, so that people with lower resolutions won't miss part of the scene? Would the developer be limited to creating smaller levels that always fit within the smallest possible resolution?
  4. Is it simply better to say "f*** it" and develop the game in one resolution, and let people with large, high-resolution televisions suffer and have to deal with a pixellated mess to save buckets of time? Does this not exclude the couch-coop audience who prefer to play in a living room?


----------


Any and all advice and correction would be greatly welcomed, and hopefully this can all eventually be collaborated into the "Ultimate noob's guide to resolution and aspect ratios".

Advertisement
  1. As you already figured out, the only smart thing you can do with pixel art is sticking to integer scales and nearest neighbour interpolation. Of course, if scaling down is an option it isn't pixel art.
  2. Simplified art for low resolutions is a good idea (in the extreme cases in which it's needed), but deformed art to match a different aspect ratio is not: one variant would be canonical and the others would be wrong and ugly.
  3. Showing a larger or smaller view of the playfield (e.g. seeing or not the top of the 3x3 block cluster in the top right first picture) depending on screen size is totally unacceptable for gameplay reasons; stretching graphics is fair but it looks bad beyond very mild aspect ratio changes; only letterboxing, or more generally surrounding a standard size view with inert decorative elements is a good option to adapt to different screen sizes.
  4. A standard 1920x1200 view on a 3840x2400 TV can be scaled up to 2x2 blocks of pixels without quality issues: it's going to look less pixelated than displayed without scaling on a 1920x1200 monitor that is half the size but three times closer.

Omae Wa Mou Shindeiru

I believe that pixel art is designed to be shown at exactly one resolution, namely the one it is created for. I don't know about you, but zoomed pixel art (and I don't even consider non-integer ratios, because ... just yuck!) just looks so "zoomed", it lacks the detail you'd expect at that zoom level. As such, I consider zooming of pixel art a dirty cheap programmers hack rather than a graphics artist preferred way of doing things.

Ideally, you'd have art at every resolution you want to support, and you adapt levels to the display. If that's not feasible, you'll end up in the mess of resolutions and size of displayed level, like you describe.

I would like to suggest that you consider zoom-level of art as a separate issue from size of the displayed level. The former exists also if I buy a screen that has a double (quadruple?) higher resolution than my current screen, but is physically the same size. The latter is also somewhat related to the game itself, for some games it's no problem, while for others it is a huge problem.

For art, one option you may not have mentioned is downscaling rather than upscaling. Make the art in the highest resolution, then reduce its size as needed. It won't be as good as when you hand-design it, but it should be better than enlarging pixel art. This way, it might be more feasible to create art in various sizes. Make the large one, automagically reduce the size, then do some small touch up to correct the most obvious flaws. (I am not an artist, so no idea how feasible that is.)

For level design, it may be possible to design levels such that it doesn't matter (much) how far you can look ahead. Hide eg coloured keys in a box, move surprises to behind a corner (ie when you enter a new screen). For focus, make the light and moving area near the player, and keep him so busy he has no time to look at the other parts of the level.

Other options is to move the "point of no return" to much earlier. When I select one corridor, I have to walk further before I see the surprise, but make the corridor long enough such that the player with wide screen also has to walk into that corridor.

You can also adapt the level, space out the obstacles depending on the displayed screen size. You can possibly add some background art here and there as well to fill the area. Player with the wide screen has to walk/fly/jump/crawl further, to reach the next stage.

On 09/08/2017 at 10:57 PM, zuhane said:

From my understanding, games that use vector-based graphics (or even rasterised graphics with high resolutions) don't give a larger screen view. Instead, they simply scale all of the graphics to always fit within the same proportions regardless of resolution.

...yes, but bear in mind that for 3D games this is not generally thought of as "scaling the graphics" - it's about picking a field-of-view for the camera - independent of the screen resolution - and the projection mathematics puts the pixels wherever they need to be. Similarly, they don't think of zoom levels because they're not working relative to the pixel size of the assets - they're working with the field-of-view value, and zooming is about narrowing the field of view, which naturally stretches out a smaller physical area across the same visual area, creating the zoom effect.

So, if you've been looking to find out how modern games are "scaling the graphics" for different resolutions, you won't find it under such a name - that's handled by the projection matrix/transform part of the graphics pipeline.

This is why there's a "lack of material" - todays games rarely use pixel art, older games which did use pixel art had little choice over the resolution, and the way game cameras work in 3D games don't have to concern themselves with issues of resolution.

However, it is totally possible - and actually relatively trivial - to change the projection matrix to maintain a 1:1 pixel ratio onscreen, for whatever resolution your monitor has. Obviously you have to make compromises: you can fill the whole screen in any resolution and cover arbitrary numbers of pixels, you can maintain a 1:1 art-pixel:screen-pixel aspect ratio, and you can show exactly the same amount of the game world to every player, but you cannot do all three at once! This means you have at least three potential options:

  1. fill any resolution, maintain aspect ratio: you will show more of the world on larger monitors
  2. fill any resolution, show same amount of the world to everyone: the field of view must be scaled to fit, so you lose the 1:1 guarantee
  3. maintain aspect ratio, show same world to everyone: you must letterbox/pillarbox/windowbox because the usable display area is larger than the area you wish to show

You should be able to answer your 4 related questions yourself, given the above information, and your knowledge about what is important to your game, and knowledge about what the most common fullscreen resolution is (1920x1080, by far).

Thank you all for the comprehensive and detailed replies.

 

@LorenzoGatti I'm surprised that this approach (mentioned in question 3) has been adopted by Terraria. I think it surprised me because it's such an incredible and ambitious game with such talented developers that you'd except camera work to seem trivial compared to the scope of that they've created, but I suppose given the genre of game it can work. As for my game, I feel that framing is very important and this cannot be the case.

@Alberth As for the art assets, I'll be creating all of the art assets once, with the exception of a few GUI assets that may need scaling differently. However, I won't be zooming the same assets at different levels at once, as I feel that different levels of pixellated zoom combined together looks horrific. As for downscaling, I don't think I could apply it in this scenario. I'm hoping to appeal to a more retro audience who might appreciate the pixellated look! I also like the ideas for obscuring areas of the level until you enter them, and I think I'll be taking that approach in level design.

@Kylotan It seems silly, but being told that you can only choose one of the three options seems to have just given me absolute clarity about how I should approach this. It seemed confusing trying to fight over which elements to meld together, but now I think I'll go for the letterboxing approach.

On the topic of letterboxing, I take it that developing for the most common aspect ratio will thus reduce the size of the letterboxing margins for most aspect ratios, since there's less difference between some of the more common aspect ratios, rather than hypothetically designing a game in, say, 10:1 (stupid looking) aspect ratio, for example?

As for the actual view of the game, if I wanted to always show the same slice of the game on every resolution, would I be picking a uniform size to display and then working my way down by letterboxing appropriately? It's hard to explain what I mean using text, but as an example:

 

Say I develop the game to have this arbitrary field of view. The game's tiles are 30x30, so I'd want the FoV  width and height to be divisble by 30. So let's say I arbitrarily create a view of 600 x 300. If I had a monitor with a display of 600 x 300, the game would essentially display like this:

smallres.png.ea1d43557acb4808000562b3c0afbb21.png

this is the slice of game world I always want to display for the rest of the game. Say now I change the resolution to something bigger, like 1200 x 600, I can just double the zoom since it's exactly double the size and the aspect ratio will cause it to fit perfectly with 2x zoom.

However, if I increase the screen resolution to something of a different aspect ratio, such as 1100 x 800, I get a different effect. Using the Terraria approach, I'd just end up with a larger camera boundary displaying more of the level, like so:

medres.png.353c91708dede34e2d274f43a3150867.png

 

This isn't what I want. So essentially, I'd rather have it so it maintains the same framing regardless of monitor size, like so:

fdfdgd.png.08ddb4fb9bc8439365ac726d4a7c62fe.png

 

Given that the native FoV does not go into this new resolution perfectly, would it mean that I could not 2x the display and would essentially have to create absolutely huge letterboxing as shown above? (Given that 600 x 300 does not go into 1100 x 800 to produce a whole number)

Alternatively, if had a much bigger resolution that went over 2x, could I increase the zoom and letterbox less? This is hard to explain, but say I have the native display of 600 x 300, and a monitor is 1300 x 700, I could essentially double the size (1200 x 600) and then simply letterbox a margin of the remainder, i.e. (1300 - 1200 & 700 = 600) = (100 x 100), meaning that I could essentially display the original display at 2x zoom with a margin of 50 pixels each side of the display? Producing a result more like this:

fdfdgd.thumb.png.b388157700d96e90825f9e900c853016.png

As I say, it's hard to vocalise such an abstract concept without actually showing you in person what I'm talking about, but if it essentially goes into a higher resolution with leaving a remainder then you get a perfect scaling, and if not, the remainder is used as the margin?

If I were to do this, I would be doing "2d in 3d", orthogonal. Have relatively high-res graphic assets, and then have them auto-scale depending on the properties of the billboard which they display on.

7 hours ago, zuhane said:

 

Alternatively, if had a much bigger resolution that went over 2x, could I increase the zoom and letterbox less? This is hard to explain, but say I have the native display of 600 x 300, and a monitor is 1300 x 700, I could essentially double the size (1200 x 600) and then simply letterbox a margin of the remainder, i.e. (1300 - 1200 & 700 = 600) = (100 x 100), meaning that I could essentially display the original display at 2x zoom with a margin of 50 pixels each side of the display? Producing a result more like this:

fdfdgd.thumb.png.b388157700d96e90825f9e900c853016.png

I was also thinking about this problem when I started making my game (also pixel art). I decided to go this route and just scale 2x,3x... and then letterbox if their resolution is not the same aspect ratio and my game.  I'm doing it in 16:9 aspect ratio and 640/360 as my native resolution. That seems to be the best solution. 

Shameless bump.

Thanks for the replies, guys. Does Lamentation's idea seem to be the most solid? Still not exactly sure which route to take.

I would say that it is, given the assumptions that I think you're making.

Let's assume you want to have the same amount of the world showing to everybody, you want to preserve aspect ratio, and you don't want the pixels to be fuzzy. Okay, reasonable.

The Steam Hardware Survey suggests that about 50% of people are on 1920x1080. 20% more are using 1366 x 768. Next most popular on 5% is 1600 x 900. These all have the same 16:9 aspect ratio but obviously the 'no-fuzzy' requirement rules out non-integer scaling factors. To accommodate the smallest of those (1366x768) means that is the upper bound, so you could find dimensions smaller than that which can be scaled by whole numbers into something close to the other resolutions.

One example might be 960x540. That maintains your 16:9 aspect ratio, it fits into 1366x768 slightly windowboxed, it fits into 1600x900 very windowboxed, and you can scale it by 2 to fit 1920x1080 exactly. If a native resolution of 960x540 works for you, given your game, this is an option.

You could consider going smaller with native resolution, e.g. 640x360? This scales up 3x into 1920x1080 exactly, it windowboxes in 1600x900 quite closely if you scale 2x.... but it's a pretty poor match for a 1366x768 screen, and you can't fit it in with a 2x scale.

And the elephant in the room is that we've only really covered the top 75% of resolutions - and leaving out 25% of the audience is a bad idea. The next most popular resolution is 1440 x 900, which has a different aspect ratio, so you can forget about ever scaling into it perfectly. 960x540 fits into it at 1x, with large windowboxing. 640x360 fits into it 2x, with reasonable windowboxing.

What can we learn from the above?

  • No matter what you do, probably half of all gamers will need to windowbox the game if you don't want to scale it by non-integer amounts. So...
  • ...maybe you should offer scaling as an option, so the player can decide whether they prefer the windowbox or the blurriness.

 

In development terms, it's relatively easy to do the mathematics:

  • the maximum 'crisp' multiplier to offer players is the current screen resolution divided by your chosen 'native' resolution, minus the remainder. If this happens to be zero, your native resolution is too big for their screen!
  • the effective resolution being used is the native resolution multiplied by that multiplier
  • the size of the windowboxing/pillarboxing/etc is the screen resolution minus the effective resolution (divide by 2 to decide the offset from the edge of the screen)
  • if the player wants to use the full screen regardless, you can implement that arbitrary scaling quite easily either via a second rendering step or by changing the camera (although it depends on the tech you're using)

I totally agree with Kylotan, you cannot make your game fit every screen resolution / monitor - either you show more/less of the world or you letterbox it. But locking down your maximum viewable game size is much better and more controllable.

 

Also i think its a good idea to fully drop the idea about "pixels" altogether and use game units as metrics only, for example meters.

But its fine to stick with pixels if you want to...

 

All my games uses a fixed game width in meters and a target aspect ratio. Game height is calculated based on game width and aspect ratio. Only when you render your game, you calculate a scale factor based on your monitor screen width in pixels and the fixed game width in meters. This scale factor is used in the projection matrix and to calculate the letterbox offset and size.

 

It looks something like this (Area means game world size visible on the monitor - Unproject means converting screen coordinates into world coordinates):

 


external Vec2f RenderUnproject(RenderState *renderState, s32 x, s32 y) {
	Vec2f result = {};
	if (renderState->areaScale > 0) {
		result.x = (r32)((x - renderState->viewportOffset.x) / renderState->areaScale) - renderState->areaSize.w * 0.5f;
		result.y = (r32)((y - renderState->viewportOffset.y) / renderState->areaScale) - renderState->areaSize.h * 0.5f;
	}
	return (result);
}

external void RenderViewportUpdate(RenderState *renderState)
{
	//
	// Calculate target viewport based on area size, screen dimension and aspect ratio
	//
	Vec2i *viewportSize = &renderState->viewportSize;
	Vec2i *viewportOffset = &renderState->viewportOffset;
	renderState->areaScale = (r32)renderState->windowSize.w / renderState->areaSize.w;
	renderState->viewportOffset = V2i();
	viewportSize->w = renderState->windowSize.w;
	viewportSize->h = (u32)(renderState->windowSize.w / renderState->aspectRatio);
	if (viewportSize->h > renderState->windowSize.h) {
		viewportSize->h = renderState->windowSize.h;
		viewportSize->w = (u32)(viewportSize->h * renderState->aspectRatio);
		renderState->areaScale = (r32)viewportSize->w / renderState->areaSize.w;
	}
	renderState->viewportOffset.x = (renderState->windowSize.w - viewportSize->w) / 2;
	renderState->viewportOffset.y = (renderState->windowSize.h - viewportSize->h) / 2;
}

 

This topic is closed to new replies.

Advertisement