Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 15 Dec 2001
Offline Last Active Today, 06:20 PM

#5179609 Code organization without the limitations of files

Posted by phantom on 11 September 2014 - 09:17 AM

However if you're asking "I want to see all the code that executes when I cast a fireball" then... well... that's what a debugger is for, surely? You're asking a run-time question.

Not really; you come to a new system, you are trying to add 'something' to it but in order to do so you need to follow the flow of the code and this can get complicated FAST when moving across multiple dependent and independent files.

I've had to deal with this a lot recently and my current method of dealing with it is to use the 'pin tab' functionality of VS to track files I've been in (and if jumping among those files even that isn't great) as well as noting down the flow on paper... all of which is a bit of a faff.

Even a small example, adding ADPCM support for Android in UE4, resulted in my having four or five files pin'd while trying to track all the logic going on.

What would have made this easier would have been anything which could have integrated the code fragments into a single view (or a sub-set of them into a couple of views if that made sense), bonus points for being able to display call linkage either in the same view or in a 'map' view so I can jump around and see where things go to/come from.

So, in the sound case, I wanted to see how 'createbuffer' was called; my current solution was to use VAX to 'find references' and open the correct file from there. What would have been 'nicer' would be same initial flow ('find references') but instead of files the output would be a list of tagged versions so that I could say 'ahah! AndroidAudioDevice!', double click on it and get that function inserted below the current code (which I could then optionally drag/drop above so that the flow made more sense) so now I can see call site and destination in the same view.

By the end of this session I might have 7 or 8 functions in the chain on screen which would have meant having 4 or 5 files open but instead all viewable on screen in a single session.


This does feel like something which needs to be also tackled at the language level however; many existing languages rely on the concept of a file and file system structure to control compile order or dependency information which could cause problems and, ultimately, not make it useful for things like C or C++.

(Although, for languages like that you could maintain the existing file setup and have the IDE work magic to give you the new views and handle changing the original data when you update the functions).

Version control would either have to be based on 'fragments' too or you'd have to provide a compatibility layer so that while the fragment 'void foo()' doesn't sit in a file normally the comp layer shoves it into one for version tracking - however being able to track on the fragment level would probably be better, bonus points for allowing a checkin to reference multiple fragments; "changed Audio to support ADPCM" for example would reference the 7 or 8 fragments changed to support it which makes it clear what has changed in that check in.

I think the idea has merits, but you will need to build it in from Day 1 and it would require retooling for a few things as well as a fundamental shift in how people think about code and structure.

I could see it making life easier however when you are tasked with doing Things in a large code base.

#5179111 Sorting/structuring renderables and cache locality

Posted by phantom on 09 September 2014 - 10:46 AM

That assumes one draw call per object, plus by the time you've got to the 'sort draw calls' you'll have already done a lot of dead object removal so you should never see a 'dead' object in your draw call lists to sort by.

At the highest 'game' level you'd be tracking the game entity which any attached renderables (1 or more draw calls) are associated; when these die the renderer never sees them.

Vis-culling per "camera", again above renderer submission, takes care of visible objects for a given scene.

Only once you get beyond vis-culling do you start breaking renderables down into their draw-call components and start sorting them and routing them to the correct passes for a scene.

#5179075 Sorting/structuring renderables and cache locality

Posted by phantom on 09 September 2014 - 06:59 AM

Dead flags, while they might seem like a good idea, are something that requires some thought however and it is important to always make sure the work you are skipping is worth the cost of the compare and potentially mispredicted jump as well as the extra cache space you are taking up. This one flag has bulked your data up by 4bytes.

In some cases you might be better off keeping the 'dead' flag separate from the objects themselves; that way a single cache line can contain information about the alive state of 16 objects and keep the size of your larger structure down meaning more cache wins and less overall trips to memory.

In other situations killing and sorting might be a better win; if you can execute your main loop with the assumption that everything is alive it makes the logic easier there and a second sweep can be performed to kill any dead elements and compact your data structures.

I built an experimental particle system on this principle with the 'simulate and draw' phases assuming all particles are alive and then a 'kill' phase which would walk the (separate) life time information and do a 'mark and compact' scheme on the particle data. So if you had 1000 particles you would walk up until the first 'dead' particle and remember that point and continue walking until you hit the next 'alive' particle and note that, then you'd walk again until the next 'dead' particle (or end) at which point you would copy the data (which was in separate memory locations per component) for the 'alive' block down so all particles were alive up until that point. Repeat from step two until end of particles.

Particle data was just data so a nice fast update/draw phase, the 'are you dead?' phase touched the minimal amount as life time information was in it's own place in memory. The copy down might have had some cost but it wouldn't have been paid often and was a trade off vs fast update and it was a linear access so would have been prefetcher friendly.

(This was a few years ago but recently an article surfaced which backed up my idea where someone tested the cost of sorting a huge array + access vs jumping about in memory a bit (including pools and the like); short version - sort + linear access won.)

But, as with all things, it requires some thought; don't go blindly throwing in 'dead' flags but at the same time don't blindly throw the idea out either.

Know your data.
Know your processing.

#5178848 Sorting/structuring renderables and cache locality

Posted by phantom on 08 September 2014 - 07:27 AM

Btw are you implying that I should sort by both transform and key? Say, I first sort by keys, and then again within all renderables that have the identical key(unlikely) sort again by transforms?


You'll want your draw calls ordered by material (shader+textures+constants for example) and then the instances inside that in a rough front to back order (z sort based on the centre point of a sphere would probably do it) which will get you good z coverage (for non-translucent objects; stuff which blends generally need to be drawn back to front).

However, you can build all this into a single sort key where the top 32bit hold details about pass, material id and other stuff and the bottom 32bits contain the depth information allowing you to sort on a single thing in one pass.

#5178845 Unreal engine 4 free for students !

Posted by phantom on 08 September 2014 - 07:21 AM

One of the nice details about this is that if you use the engine on your course even once you've finished the course you can keep using the version of the code you had when you left without having to pay to get access (you won't get updated code without paying of course) which means you can effectively make and release a game at zero cost.

(On release you are subject to the normal royalty fees of course but that is on money made so your outlay for the engine could still be zero.)

#5178605 Texture Array Memory Allocation

Posted by phantom on 06 September 2014 - 04:16 PM

I can't recall the precise semantics for when OpenGL textures get their backing store but all of the memory is reserved at once.

So if you created a 1024*1024 64 layer texture then as soon as memory is allocated ALL the memory will be allocated at once.

This is the reason for the sparse texture and buffer extensions; they allow you to create the texture but only commit a certain amount of backing memory to it as required on demand; so you'd create your texture as before but before writing data you'd tell OGL 'give me memory for level 0, mip 0' and at that point it would allocate that memory, and only that memory.

#5177901 my SIMD implementation is very slow :(

Posted by phantom on 03 September 2014 - 12:48 PM

but I don't know anything about assembly...

For this kind of low level fiddling this WILL be a problem; in order to get the best from SIMD you are going to need to have some understanding of the underlying assembly code, cache line and memory fetches.

Just throwing intrinsics at the problem won't help without a consideration of how it fits in with the data flow, as we've already seen with regard to your huge data structure.

#5177607 Did I begin learning late?

Posted by phantom on 02 September 2014 - 01:43 AM

Seriously, where did this weird ageist meme come from?

I think people see the rare exceptions who have done super things by the time they were 18 and think 'oh, it is too late for me' because of that - its a fun human thing to compare ourselves to others and then get depressed/unhappy when we don't feel like we measure up.

#5176914 Is Unreal Engine 4 easier than Unity for mobile game development by an artist...

Posted by phantom on 29 August 2014 - 08:14 AM

When it comes to the 'as little code as possible' requirement then UE4 has the blue print system which will let you hook up game logic without having to write a line of code; just drop nodes into the blue print, hook 'em up with lines and away you go.

There will be a learning curve on them of course but that might do just what you need.

#5176701 Abstract away the API?

Posted by phantom on 28 August 2014 - 10:19 AM

The only problem here is how to abstract away the ID3DxxDeviceContext, which would be needed for drawing the mesh, without inventing a new API for it.

Do not have the mesh draw itself; instead have the renderer subsystem ask the meshes for their information data (buffer IDs, material IDs, draw counts etc) and have the renderer draw the object.

A centralised list also brings with it the benefit of being able to centralise your state sorting (and using smaller objects for it) AND making collapsing for instancing much easier.

#5174826 So... C++14 is done :O

Posted by phantom on 19 August 2014 - 02:37 PM

A small number of developers push back when they see new things in code, but most will eventually give in. The language has changed, and they need to either adapt or die (figuratively).  I have not seen any serious resistance to adopting the features when they are available on all the target platforms.

Annoyingly at the last place I worked I did see this, with arguments (without seeing code I might add) that C++11 features are 'too easy to misuse' or 'ugly' or 'hard for new people to understand' (although my reaction to that last one was 'they need to keep up with the language then'). I was pushing for adoption (mostly in C++ tools, with some features as the compilers allowed, MS being the limiting factor) but it felt like hard going.

New company is actively picking up C++11 features as they make sense/are supported; the main barrier for non-platform specific code using C++11 features was Android, but with moving across to Clang the limit is pretty much, again, the MS compilers which dictate feature level.

The difference seems to be 'older, battle scared, comfortable' developers vs 'keen and punchy' developers (not young, by any means, but just more energy for things) and that itself might come down to the company in question and the mindset of those working there.

#5174652 So... C++14 is done :O

Posted by phantom on 19 August 2014 - 03:24 AM

C++14 would frame that more like;
std::for_each(std::begin(Values), std::end(Values), [](auto value)
and once you get use to reading that then there is always this fun setup...

std::erase(std::remove_if(std::begin(values), std::end(values), [](auto value) { return !value->alive();}), std::end(values));

#5174647 So... C++14 is done :O

Posted by phantom on 19 August 2014 - 03:09 AM

Why would you write the loop the 2nd way when you have std::for_each and lambda functions?

#5173053 Can't solve problems without googling?

Posted by phantom on 12 August 2014 - 04:34 AM


When at work, when confronted by a problem my first thought it ALWAYS "I wonder if anyone has had this issue and solved it already..." and away to google I'll go to see if that's the case and to learn from something someone else has already solved.

The 'learn from' part is pretty key; if you find a solution, copy and paste it and don't take in why it worked then you've learnt nothing. If you take a moment or 7 to think about the solution, how it works and how to refine it to your situation then yay! experience!

And that's where the ability to solve 'new' problems comes from; experience.

Most seemingly new solutions will always have a seed in an existing solution to an existing problem which someone then thinks about some more, applies knowledge and experience gained over the years to refine the solution and create something new.

Nothing is created in a vacuum and all new ideas are feed from old ideas and as you gain more knowledge you'll find yourself doing this naturally.

Problem solving itself is nothing more than looking at something and then taking it apart so that one big problem because two smaller ones, which in turn become two smaller ones and so on. This gets easier with experience and practice as you get use to it and solve problems.

The key point is 'research' isn't a dirty word.

#5171319 Where to store container of all objects/actors/collision models/etc

Posted by phantom on 03 August 2014 - 04:55 PM

Why is having Tree and Missile be completely separate classes (that both have the common functionality of being in a location and needing to be drawn to the screen) a better solution?

Because you start coming up with crazy inheritance structures just to add some extra functionality.

Lets say you start with something which needs to be drawn; Actor { Vector2 position; Sprite* graphic; }
But now you need one to be able to have collisions in some cases; CollidableActor : public Actor { BoundingBox box; }
But not you need one to also be moveable in some cases; MoveableCollidableActor : public CollidableActor { void Move(); }
But now you need one to also be able to fire; ShootableCollidableActor : public MoveableCollidableActor { void Fire(); }
Oh, but now you need a tank; Tank : public ShootableCollidableActor { void Fire(); void Move(); /* because tanks move differently */}
Oh, but now you need a tank which also has rockets; RocketTank : public Tank { void FireRocket(); }
And now I want a Jeep with no gun; Jeep : public MoveableCollidableActor { void Move(); /* because Jeeps move differently */}
And now I want a Jeep with a gun; GunJeep : ShootableCollidableActor : public MoveableCollidableActor { void Fire(); }
And now I want a turret which doesn't move and ooops.. all my shootables are moveables...

At which point you have a ridge hierarchy and things in the wrong place.
If you try to resolve it by moving 'shoot' down and 'move' up then you end up with anything which can move must shoot, but our Jeep doesn't so that's wrong.

This is why the advice is that you should prefer composition over inheritance by default.

In this case;
Actor { Vector2 position; Sprite * graphic }
Tank { Weapon * primaryWeapon; Actor *; CollisionObject * collision; MoveLogic * movelogic; }
RocketTank : Tank { RocketWeapon * secondaryWeapon; }
Jeep { Actor *; CollisionObject * collision; MoveLogic * movelogic; }
GunJeep : Jeep { Weapon * primaryWeapon }
Turret { Actor *; Weapon *; CollisionObject *}

There is still inheritance there but they are short chains (Jeep -> GunJeep, Tank -> RocketTank, MoveLogic -> {TrackedMove, WheelMove}) and are not bound so if you wanted to make a new object like a helicopter it is easy to plug it in and give it more weapons.

Chopper { Weapon* weaponArray[4]; Actor *; CollisionObject * collision; MoveLogic * movelogic; }

So we could build a Chopper with 2 tank guns and 2 rocket guns without having to worry about their firing logic.

(There is also an added bonus that these entities don't have to do their own processing; a container somewhere could hold all active CollisionObjects and process them at once, this means nice cache bonuses for code and data).