How to time bomb a beta?

Started by
79 comments, last by Mouser9169 10 years, 1 month ago

finding bugs is the coders job. they should not be allowed to sign off on a section of code until it has been thoroughly tested and debugged BY THEM! Then others can have at it.

"Testing can reveal the presence but never the absence of bugs." Even with 100% unit test coverage and round-the-clock QA with a full staff (2 testers to each 1 developer) you still want to plan on your beta testers finding bugs. <3 Because odds are very, very high that they'll hit something weird that you all haven't found yet. That's much of the value of beta testing: a larger volume of monkeys hittin' the keyboard. smile.png

Gotta agree with this. All the things you listed (compiler showing all warnings, etc...) are good best practices, and will show any errors in syntax or scope, but they can't show logic flaws. Code that 'works' perfectly fine, except it doesn't do what you wanted it to, for whatever odd reason. Unless you're saying logic flaws aren't bugs.

"The multitudes see death as tragic. If this were true, so then would be birth"

- Pisha, Vampire the Maquerade: Bloodlines

Advertisement

So if you were the programmer of Ocarina of Time you would have fixed all these bugs before the game shipped?

I will echo the comment that if the only feedback you get from the game is "it's awesome", you need to find better testers then friends, family, and former players. There are things wrong with you game and there are design decisions you made that people won't like. If people are already biased towards your game then they won't mention the little things they think are wrong. You need some assholes that have never heard of your game to try and rip it apart. Hopefully this is what you are trying to do with the beta.

It may not be a fair comparison, but you game does sound a lot like "Rust" that is in beta on Steam. Visually, Rust is light years beyond your game (maybe your screenshots are super old so my comparison is off) and is cheaper.

Hidden RDTSC instruction - Awaits if a debugger is present. Basically is just a hidden windows timer, akin to GetTickCount, but screwed up in the byte code.

Runtime functions - Use VirtualProtect (or similar for your OS) and implement it. Is a lot of fun and you'll learn a lot.

Implement a random obfuscation virtualization environment. Hard, but not as hard as one would perceive. You'll just need to dive into the PE format (agian, for your OS, mine is Windows..), it's honestly quite intriguing.


So when you say your game is "as big as Skyrim", do you mean the game world is the same size, or does Caveman have as much content in it for the player to do as Skyrim does?

the game world is 2500x2500 miles in size, and you can walk all the way across it (barring any oceans in the way). you can build rafts and sail across the oceans. and NO LOADSCREENS! all assets are loaded at program start (about 10-20 seconds load time). walk speed is about 4mph. the game runs in realtime (but has accelerated time too!). you have to sleep in caveman, like in the sims, but we'll ignore that for now. so if you walked across the caveman world in reatime without sleeping it would take you 625 hours of real time (26+ days).


I believe Daggerfall is the largest commercial RPG ever produced (It would take real life weeks for a character to run from one side of the world to the other - they tested it).
Good point. running speed in Caveman is twice walking speed - so, again ignoring the need to sleep, we're at 13+ days (2+ real life weeks @ 24 hours a day) to RUN across the world without using accelerated time, cross country travel, or the teleport playtesting cheat. if you used sprinting in Caveman when not fatigued, it might speed that up a little.
the game world contains 60,000 caves, which can be unoccupied, or occupied by animals or cavemen.
it contains 5000 rockshelters - which are similar to caves, but only found along cliffs.
it contains 18,000 huts, which are occupied by cavemen.
it contains 20,000 caverns, which are the equivalent of a dungeon level, or a doom level, or a cave level in skyrim. caverns are generated when you enter them the first time, and saved when you leave. when you return, they are reloaded. caverns repopulate over time.
a new world is generated each time you start a new game - so its random, not hard coded content like skyrim. the entire world is saved when you save a game, so its persistent. and you can build and demolish structures, etc, so its modifiable.
the game also models changes in terrain and occupants of shelters. so berries and fruit come and go with the seasons, vegetation coverage can change over time as the climate changes. cave and rock shelters can change occupants from time to time. new huts can spring up, old huts can be abandoned, as the cavemen move on to greener pastures.
so thats how big the world is.
how it stacks up in other respects:
the game has more than the average number of weapon types - 65 types of weapons - and none are made up. this is partly due to the fact that any tool that can be used as a weapon is considered both a tool and a weapon.
there's no magic in the game, so comparisons in that category can't really be done. the closest thing to magic in caveman is high quality objects, god artifacts, an god effects.
the game includes objects and armor, but in numbers typical for such games (300+, and ~30? respectively).
caveman has over 50 types of "monsters" - so far.
there are no classes, instead there are skills, 48 or so of them. only 5 or so are weapon skills.
you can cook over 30 types of food, and yes there's a cooking skill. since its a person sim, like the SIMS, food is required, unlike skyrim, where its just a weak powerup in a shooter game.
but the place where caveman is REALLY big, is in the actions. like the sims, you can select things and interact with them. i've never calculated the total number of actions in the game since its so large. i know its over 1000, perhaps as high as 10,000 or more. it has over 100 generic action class handlers, each of which handles an entire class of actions (all repair actions, all cooking actions, etc).
by comparison, in skyrim you can open a container, sleep in a bed, open a door, sit, use smelter, use forge, use grindstone, use armor table, use alchemy lab, use arcane enchanter, pull chain / throw lever, and cook. did i forget any? of course this is not a fair comparison, as skyrim is a quest based rpg shooter with some interactive/open world elements, not a person sim.
open world play is another "big" feature of caveman. there are quests, but they are optional. unlike skyrim, the game is primarily about open world play (think original table top D&D role playing, caveman style), not primarily about completing quests. when i evaluated skyrim, open world play was the first thing i evaluated. i found it to be not-so-great. again, not a fair comparision, as skyrim is designed to be an immersive quest based shooter, and caveman is an open world rpg / person sim.
and then there's replayability. Caveman is designed for maximum randomness. nothing is hard coded, everything possible is random. The whole idea is it should be random enough that even i as developer can play and enjoy it without knowing what will happen next in the game. so everything is randomly generated at game start or when you first encounter it (IE cavern level maps). the game does not use hard coded spawn points, except for occupied caves and shelters. all other encounters and events are random.
by contrast, skyrim is entirely hardcoded, same map, same dungeons, same quests, same NPCs - every time you start a new game. while evaluating it, i created new charcters to test each thing. so i made a fighter for open world play, a dragon slayer, an imperial, a mage, a stormclock, etc. by the time i was done evaluating it for work, there was little game left for me to enjoy as a fan of the elder scrolls! sad.png

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php


"Testing can reveal the presence but never the absence of bugs."

true. testing confirms correct implementation. absence of bugs can only be guaranteed by code design, and only under some circumstances.

IE:

void main()

{

}

contains no bugs (obviously) - and lets not get nit picky here folks - i think you get what i'm trying to say.

its only code that you can't tell by inspection if its bulletproof that you can't design to be 100% bulletproof. then you have to rely on testing or theoretical calculations of edge cases, etc. the sad fact is that its not really possible to path test any code with all possible inputs, if that code is of any significant size. and non-deterministic code makes it even more difficult.

and the problem gains an order of magnitude once you get into integration testing. bulletproof by inspection becomes even more difficult. you may be able to account for some cases, but seldom for all.

otoh, if you use a systems approach, you can break down systems into subsystems until they can be made relatively bulletproof (bullet resistant? <g>). Then you can build up systems in a bulletproof manner. this tends to minimize bugs of all types getting through.

bugs are inevitable because hardware and software are made by humans.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php


So if you were the programmer of Ocarina of Time you would have fixed all these bugs before the game shipped?

absolutely! assuming i could find them. when building a complex system like that where every bit interacts with all the rest, you have to take into account the ramifications of their being combined. the more bits you add, the more connections possible, and the more you have to account for. not thinking through all the possible the effects of a feature results in stuff like random town encounters with vampires, cultists, and thieves killing all the merchants in Skyrim - rendering it eventually unplayable. there they had a spawn point level based design, and added random encounters that could kill off required NPCs. so they modeled random encounters that can kill npc merchants, but they didn't model killed merchants getting replaced by new npc merchants - which is required for a balanced simulation that will run correctly long term. in fact i don't think anyone gets replaced if they're killed in skyrim, except guards and dark brotherhood followers, and i'm not sure about the dark brotherhood followers, never lost one. i lay the stripped dead bodies of the merchants in the middle of the street out on display. as a persistent corpse, they can be used as safe containers that don't repop.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php


It may not be a fair comparison, but you game does sound a lot like "Rust" that is in beta on Steam. Visually, Rust is light years beyond your game (maybe your screenshots are super old so my comparison is off) and is cheaper.

it seems the only things in common are that some parts of rust have a paleo-type setting.

rust is a mmo about survival and crafting. it seems to start paleolithic, but goes beyond that apparently. the developers themselves say they're not sure where they'll go with the game. nice rocks though! i'll have to do something like that.

caveman is a simulator. like a flight sim, or a sub sim, or any other full tilt hard core vehicle simulation software. only difference, you're driving a caveman. so its all about simulation realism first, and pretty but unrealistic graphics second. in fact, the game was designed specifically that way. most game devs seem to pick a graphics engine, then build what game they can with the remaining clock cycles. caveman is done the other way. you build the game, and what clock cycles are left, you draw everything as good as you can. an example: caveman uses rigid body animation for monster and npcs, allowing more monsters and npcs onscreen at once. its doubtful that today's hardware could run it if it used traditional skeleton and mesh deformation models. can games draw over 100 npcs onscreen at once with mesh deformation and no slow downs? the most i've ever seen in skyrim is about 20-30 in the rebellion campaign battles. and the bodies disappear as soon as you kill them! if you're quick, you can loot them first. ; )

graphics in caveman are designed to attempt to mimic real life terrain. as such, plant and tree density is much higher than most games. so instead of going for woods effect with a few trees, caveman attempts to draw true dense woods. requirements of the simulation can also dictate how graphics are done. caveman requires sky and cloud graphics that can be driven by the weather engine. also, it has to run on a single low end pc - both the simulation, and the graphics. the graphics still have a long way to go in many respects. but i'm only one guy, and i know where i can, and cannot compete. a graphics arms race can only be won by the dominant player in the market. also, graphics don't make a game, just look at minecraft.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php


Code that 'works' perfectly fine, except it doesn't do what you wanted it to, for whatever odd reason. Unless you're saying logic flaws aren't bugs.

first you write it, then compile it, then run it and test it to make sure it works as intended. this eliminates logic flaws - but only in that bit of code. there's still integration errors to deal with, when you hook up two or more good pieces of code incorrectly. and then sometimes there's odd ball edge cases that nobody ever thought of.

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php


Hidden RDTSC instruction

no good: false positives if they alt tab away - correct?

i have high rez timers on the game loop i could query for excessive frame time (indicating debugging actions going on), but same difference, false positives from alt-tab.

i think there are other way's to detect a debugger. haven't finished reading all the links i saved on the subject. i plan to do a posting to my gamedev info journal, but there's a TON of info!

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php


Runtime functions - Use VirtualProtect (or similar for your OS) and implement it. Is a lot of fun and you'll learn a lot.

so you DL some code to the client, and use:

BOOL WINAPI VirtualProtect() to set PAGE_EXECUTE ?

but they could still get a memory dump, couldn't they? then make their own code?

Norm Barrows

Rockland Software Productions

"Building PC games since 1989"

rocklandsoftware.net

PLAY CAVEMAN NOW!

http://rocklandsoftware.net/beta.php

This topic is closed to new replies.

Advertisement