How do I measure risk?

Started by
15 comments, last by Gian-Reto 7 years, 11 months ago

>> Like Briandigitalis points out, a small or single person team can't afford to hire all specialists (business, Artists, Extra programmers) required. I wish I can hire all the business-risk analyst, but for obvious reasons I can't afford to do that. So the next thing to do is to improvise which is do most aspect - meant for specialist - on my own

same here. such is the life of the lone wolf developer. but you don't have to be a big or rich studio to be good at estimating - you just have to be good at estimating. although a rich studio might be able to contract such skills.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

Advertisement

>> it still might not give a true picture since corners are being cut here and there

then you are not prototyping everything you should or to the depth/degree you should. prototypes should move you from a state of uncertainty to a state of absolute certainty going forward (at least with respect to that which was tested).

>> So rapid prototyping might as well be developing a full draft prototype, which one can then refactor, refine and rewrite algorithms

you want to test only what you must to determine feasibility - no more - no less. however its not unheard of for a prototype to become the core code of the final game.

ideally, pen and paper analysis and prototyping when necessary will allow you to choose the correct non-optimized algos for the use case in question so refinement or rewriting is unnecessary - unless you must later optimize - as indicated by profiling. note that this applies to selection of existing algos, not inventing new ones.

>> and this is close to what I'm currently doing... except its taking too much time than estimated

the longest i've ever allotted to a prototype was a week on three occasions: a realistic airship flight model derived from scratch using the laws of chemistry and physics,

hierarchical A* pathfinding for entities of arbitrary size, and skinned meshes. nothing else has ever been more than 2-3 days max. the whole idea is to get exactly the answers you need as quick and dirty as you can - and nothing more. its a prototype - a testbed - it doesn't have to be release grade code. its just has to work well enough to prove or disprove what you're trying to figure out. never spend more time on a prototype than you must. if it pans out, you can then refactor, or rewrite, or cleanup the code. sounds to me like your prototype is too big. if you figured it might take more than a week, odds are you're testing too much at once, or perhaps some things you don't have to. if you figured it should take a week or less, and that time has passed, you should have a reasonable idea of how long til you can get it going. if you find yourself unable to say how long to test something - you've moved past mere prototyping and into the area of research, where there are no clocks. invention knows no schedule but its own. if this is the case, you're probably misleading yourself if you think you can invent according to some schedule or timetable. its been tried in the past on projects both large and small, and seldom works. the Apollo moonshot being the last case that comes to mind where they actually managed to invent everything new in the time required. a couple military projects by various nations over the years may qualify as well.

>> I don't have that luxury of time

to do the estimate or to make the game once you adjust you estimate and see how long it may actually take in reality? multiplying your best guess by two or five or ten takes almost zero time, so you must mean that the result is a game which may in reality take longer than you can afford to spend. so be it. in that case, the scope of the project is simply too big for you to undertake by yourself with the time you have available. you'll need help to proceed (IE a team), or will need to select a smaller scope project to pursue. such is life in the lone wolf lane. i could build incredible games with unlimited time, resources, and manpower at my disposal, but i don't have those those, so i build what i can than can be competitive in its category.

>> But I prefer long stretches of day job working and long stretches of project development. That way my focus is full and unbroken during project development phase,

understandable. long uninterrupted stretches of time are the most important thing to a gamedev, especially the lone wolf. until you can quit you day job for real forever, you'll have to suffer life interfering with your development time. choose your priorities wisely. having a life and being a successful game entrepreneur are often mutually exclusive activities.

>> Still I do run out of savings (and my wife does her bit to help out anyway). Because the stretches project development tends to go longer I do run out, but its a catch 22 situation

like i said before you need to secure a source of income to support yourself during development. IE you have to work a job (or whatever) AND build games AT THE SAME TIME. otherwise, you'll just keep running out of money. the one thing you can be sure of on any engineering project which is not a repeat (and even sometimes when it is) is that it will take longer than planned. that's life and murphy's law. so you can't work for X amount of time and save Y dollars to fund Z hours of development to accomplish goal A, cause Z is always longer than expected. however, you might save up enough for 2Z, 5Z. or 10Z hours of development, while still estimating only Z hours required, in which case you might not run out of money. if no matter how you juggle the numbers its just takes too long to save up the money to get more than enough time to do the dev work, then your project is simply too big for you to take on alone. i could build a better skyrim than skyrim, but i couldn't do it alone, unless i had about 30 years

>> maybe, just maybe if they are way more advanced than me and know the maths and the right algorithm straight out (so don't need to experiment - make mistakes through dead ends like I did several times) then yes, they can do a draft prototype in 1 months )

ok, this explains a bit. you're not inventing new algos. you're trying different ones. never sit down to the keyboard until you know what to write. research your algos first. choose the best, implement it, move on. if unsure, prototype. this should almost never be necessary for selecting between known algos for a specific use case. prototyping is most properly used to test that which is untested by all - not just by you personally. IE to boldly go where no one has gone before, not to reinvent the wheel. and learn your math and physics before trying to implement. when you're at the keyboard, its time to type - not time to figure out what to type. you should figure out all that before you decide whether you'll even build the game or not. you can't judge if its worth your time until you know how long it will take - which means you need to know the algos and math to be used. if you try to estimate with no idea of the math or algos to use, i'd say off by a factor of 10 is a damn lucky guess - i'm surprised you weren't off by more.

obviously you must know the math and algos to be used BEFORE you can get any kind of idea about how long anything is going to take. and LONG before you sit down to code. you have to do that before you can even vet a potential title for feasibility.

in brief the process should be as follows:

1. RESEARCH!!!!!

2. prototype only as absolutely required.

3. vet for feasibility

4. code.

you seem to have skipped step 1 altogether, are going way overboard on step 2 seeking the answer to step 1, are having difficulty with step 3 due to skipping step 1 and wasting too much time on step 2 trying to get the answers you should have gotten in step 1, and thus you never reach that happy place known as step 4 - where you can code with confidence.

its now obvious your fundamental problem is lack of up front research.

fix that, and everything will become MUCH MUCH easier.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

I had to take a project management course in graduate school and one of the things that was emphasized is that while most tools project managers use are not too complicated (including risk analysis), project management itself is currently heavily based around experience-- using knowledge gained in working one project to do a better job working the next. That may change as the field is still professionalizing and developing core standards and techniques but for now there is no real recipe approach.

I used this book, which was OK but not stellar (and the Kindle formatting was awful). It would be enough to get you started, and with a tool like Project Libre you should be able to be start working as your own project manager. There aren't any substitutes for experience though. At my last job my department was put under a new project manager fresh out of graduate school (for project management). Her lack of experience showed, both as the two projects she worked with us on unfolded and when they finished (unsuccessfully by any measure: budget, schedule, results). But the second project went better than the first, and I've no doubt that she's far better today.

-------R.I.P.-------

Selective Quote

~Too Late - Too Soon~

Recently i read a thread (damn! am not able to locate it anymore), where the OP asked why a particular game should take up to 4 years to make by the solo developer.
Overwhelmingly the replies where that 4 years (some even suggested 6 years) is just about right and not too long.


Here's the thread.

I was the one who said I've been working six years on my project, but it was due to learning as I go, and wasted effort by focusing on the wrong things at the wrong time. I wasn't trying to say that game development should take six years. Personally, I think 18 months is a much better time-frame to focus on, but that requires some existing experience under your belt, methinks.

From what you write, I think your problem is less with not accounting for risks, but more bad project planning.

There is only one way to learn to plan and scope correctly: trying, failing, adjusting, and trying again (of course you could read up on common wisdom on how long projects in a certain area tend to take, but we all know how accurate that will be for projects with a lot of research and experimentation involved).

You seem to be doing at least the trying part already. I would guess its the adjusting part that is giving you problems, right? How are you analyzing your projects outcome? How do you plan your next project? Do you apply the lessons learned, do you adjust the scope and/or the projected time needed to not repeat the same thing again?

Do you work with enough reserve time? Or if you think something should take a week, do you write down "1 week" in your plan (should be 4 weeks,really. Might be two weeks, if you are not working for someone else and are not in danger of getting whatever you write down halfed by some stupid higher-up, but given you know you are bad with planning time, keep it at 4 weeks)?

Did you analyze what did take so long in your last few projects? Are you sure this time was "wasted" on an important part of your endproduct? Or was it rather a nice to have, while neglecting the important features of your project?

Keep this in mind: if someone is extremly good at predicting project timelines, and always delivering on time, he/she is either crunching himself, his coworkers and subordinates to death to achieve that.

Or this person just has a lot of expierience in project planning and EXECUTION. He/she knows from personal expierience (most probably hard gained in spectacular failures as well as crunching to success) how long certain things take, and he/she knows how much time to factor in for the things that you don't know how long they will take.

As much as project leads are often seen as "not doing anything worthwhile and just pushing other people around" by some devs, a good project lead is worth his weight in gold, because he can steer the project in a different direction when it would else spiral into eternal crunch, or even worse, failure.

the longest i've ever allotted to a prototype was a week on three occasions: a realistic airship flight model derived from scratch using the laws of chemistry and physics,

hierarchical A* pathfinding for entities of arbitrary size, and skinned meshes. nothing else has ever been more than 2-3 days max. the whole idea is to get exactly the answers you need as quick and dirty as you can - and nothing more. its a prototype - a testbed - it doesn't have to be release grade code. its just has to work well enough to prove or disprove what you're trying to figure out. never spend more time on a prototype than you must. if it pans out, you can then refactor, or rewrite, or cleanup the code. sounds to me like your prototype is too big. if you figured it might take more than a week, odds are you're testing too much at once, or perhaps some things you don't have to. if you figured it should take a week or less, and that time has passed, you should have a reasonable idea of how long til you can get it going. if you find yourself unable to say how long to test something - you've moved past mere prototyping and into the area of research, where there are no clocks. invention knows no schedule but its own. if this is the case, you're probably misleading yourself if you think you can invent according to some schedule or timetable. its been tried in the past on projects both large and small, and seldom works. the Apollo moonshot being the last case that comes to mind where they actually managed to invent everything new in the time required. a couple military projects by various nations over the years may qualify as well.

I don't fully grasp how you allocate just one week to prototyping, knowing that this is a proof of concept test? If we are talking about the same thing here, then....

...I suppose the difference may depend on the complexities of how other parts of the game interlink with the prototype being tested.

If the interlink of the rapid prototype code is intended to be complex and multifaceted in the finished game, then the question is- would the the tests be valid, since other parts of the game meant to work with it in final working mode - are not part of the tests?.

I am not trying to disprove you, in fact I may be wrong and willing to learn how your test on an isolated code (as described) is valid, when you are not able to test how it works with other parts. Are you sure when it comes to interlocks with other moving parts of the games as in the real case scenario, it would't break?

But this is about wanting to learn... not justify my own methods which hasn't worked so far (or more accurately- takes too much time), so feel free to let me know how

The reason I develop a full draft prototype rather a rapid prototype the way you described, is to ensure my tests are valid under the real working game scenario...

Remember the many kickstarter projects, with their proof of concept prototypes, who eventually failed to deliver - not saying my game is as complex as their projects though, but just saying it could be because they tested some prototypes in isolation from other parts and thought that was valid. Note: not being an insider to any of these failed projects - this is only a speculative guess from me

Sometimes some maths would work correct theoretically, but in the real software context with its interlocking parts, there may be lots of special cases, ... or the dynamism may transform the problem, which means you have to re-analyze, modify or re-develop algorithms. Complex projects are very dynamic. It may have to do with the fact that (unless you are taking well tested concepts from a text book) there is a limit to how many levels deep you can theoretically analyze and research ideas

But i am open to learn and I may still be wrong. If this is so and there are ways of validly testing isolated rapid-prototypes under real working game scenario please let me know

ok, this explains a bit. you're not inventing new algos. you're trying different ones. never sit down to the keyboard until you know what to write. research your algos first. choose the best, implement it, move on. if unsure, prototype. this should almost never be necessary for selecting between known algos for a specific use case. prototyping is most properly used to test that which is untested by all - not just by you personally. IE to boldly go where no one has gone before, not to reinvent the wheel. and learn your math and physics before trying to implement. when you're at the keyboard, its time to type - not time to figure out what to type. you should figure out all that before you decide whether you'll even build the game or not. you can't judge if its worth your time until you know how long it will take - which means you need to know the algos and math to be used. if you try to estimate with no idea of the math or algos to use, i'd say off by a factor of 10 is a damn lucky guess - i'm surprised you weren't off by more.

obviously you must know the math and algos to be used BEFORE you can get any kind of idea about how long anything is going to take. and LONG before you sit down to code. you have to do that before you can even vet a potential title for feasibility.

in brief the process should be as follows:

1. RESEARCH!!!!!

2. prototype only as absolutely required.

3. vet for feasibility

4. code.

you seem to have skipped step 1 altogether, are going way overboard on step 2 seeking the answer to step 1, are having difficulty with step 3 due to skipping step 1 and wasting too much time on step 2 trying to get the answers you should have gotten in step 1, and thus you never reach that happy place known as step 4 - where you can code with confidence.

its now obvious your fundamental problem is lack of up front research.

Text can mis-construe real intended meanings sometimes. I can see why you interpreted what I wrote as - I'm only using trial and error methodology. If I set out like to work on complex projects with trial and error in mind, even if i'm given all the funds and time in the world, I will still make a mess of it and get no where. On the other hand you are correct in your above outline of what is required to start and complete a project successfully.

Having said all of the above, It is true that I really do need to improve my research and project planning phase and devote more time to these at the initial stage (as Gian-reto also writes). It would save me a lot of headaches and allow me to evaluate the risks more accurately

From what you write, I think your problem is less with not accounting for risks, but more bad project planning.

There is only one way to learn to plan and scope correctly: trying, failing, adjusting, and trying again (of course you could read up on common wisdom on how long projects in a certain area tend to take, but we all know how accurate that will be for projects with a lot of research and experimentation involved).

You seem to be doing at least the trying part already. I would guess its the adjusting part that is giving you problems, right? How are you analyzing your projects outcome? How do you plan your next project? Do you apply the lessons learned, do you adjust the scope and/or the projected time needed to not repeat the same thing again?

Do you work with enough reserve time? Or if you think something should take a week, do you write down "1 week" in your plan (should be 4 weeks,really. Might be two weeks, if you are not working for someone else and are not in danger of getting whatever you write down halfed by some stupid higher-up, but given you know you are bad with planning time, keep it at 4 weeks)?

Did you analyze what did take so long in your last few projects? Are you sure this time was "wasted" on an important part of your endproduct? Or was it rather a nice to have, while neglecting the important features of your project?

Keep this in mind: if someone is extremly good at predicting project timelines, and always delivering on time, he/she is either crunching himself, his coworkers and subordinates to death to achieve that.

Or this person just has a lot of expierience in project planning and EXECUTION. He/she knows from personal expierience (most probably hard gained in spectacular failures as well as crunching to success) how long certain things take, and he/she knows how much time to factor in for the things that you don't know how long they will take.

As much as project leads are often seen as "not doing anything worthwhile and just pushing other people around" by some devs, a good project lead is worth his weight in gold, because he can steer the project in a different direction when it would else spiral into eternal crunch, or even worse, failure.

The problem seems to be that the implementation of new ideas don't reveal all their gremlins at once, but they unfold gradually as the project builds up.

My image processing nightmare is an example of this

But like i mentioned above, I really do need to improve my research and project planning phase

Overall talking about it here is good because I'm learning a lot and getting ideas from replies. And also someone else could learn from my mistakes of not being able to judge to complexity, time and risks of a project adequately

can't help being grumpy...

Just need to let some steam out, so my head doesn't explode...

The problem seems to be that the implementation of new ideas don't reveal all their gremlins at once, but they unfold gradually as the project builds up.

And this is exactly where expierience kicks in. At some point you KNOW that just because things run smooth during the first half of the implementation doesn't mean it will not slow down in the second half.

And you will start to move the unexplored to the front, working on the proof of concepts for new conepts first, and implementing the known things second (which now goes in the direction of risk planning, true). If you work on your biggest unknown first, and then move to the second biggest, and so on, you can either fail fast, or nail down milestones much quicker as the uncertainity is moved out of the way sooner.

Of course that cannot prevent you from misjudgement, and some things that just not work in your current project even though they worked just fine in your last few.

This topic is closed to new replies.

Advertisement