lawnjellyMember Since 20 Mar 2012
Offline Last Active Yesterday, 08:17 AM
God of programming. Except the hard stuff like 3d geometry. And maths. All these bugs creep in. Godamned floating point. And other hard stuff.
43 years old, going a bit senile. Worked in game dev, now retired.
Basic - Age 9 Spectrum
6502 Machine Code - Age 12 BBC micro
Pascal - Age 19 unix
C - Age 20 PC
C++ - Age 23 or so? PC
Now programming games again, as well as side projects (interactive children's books, DNA stuff, natural language stuff, music software, image editing software). Have dabbled in a huge number of areas, I'm probably more a jack of all trades master of none.
- Group Members
- Active Posts 134
- Profile Views 6,537
- Submitted Links 0
- Member Title Member
- Age 43 years old
- Birthday June 30, 1973
Tearing my hair out debugging 3d geometry.
Semantic networks and natural language processing.
Beating up small children.
Posted by lawnjelly on 15 December 2012 - 09:43 AM
Note pitch: I'd stick with just a note number like MIDI for now, and the 12 note western scale. 99% of music is written like this, and handling other systems is a bit more advanced and something you can tap on later. Storing notes as float frequencies I wouldn't recommend for several reasons : accuracy (say you transpose down, then up later) .. the wavelengths don't have a linear relationship with note number. You might want to do operations based on the relative pitches of notes, or detect chords etc. All of this would be stupidly difficult just trying to store wavelength / frequencies. Besides the fact your source instruments may have different base frequencies anyway and these would need to be compensated for.
Pan: Why limit yourself to stereo pan? What about surround sound?
Channels / instrument info on a note: Would you want the note to determine this, or the track and / or pattern? Having a 'grouping' feature for notes can be useful though. Remember you are going to want to be able to do stuff like edit the instruments you are using quickly and easily, and not change this for every note.
What happens when by accident you set 2 bunches of notes to the same instrument ID (if storing on the notes?) you have then lost their 'individuality'. Better to store something else that then maps to the instrument.
Volume: This is usually key velocity rather than volume (there is midi volume as well, but you wouldn't store this per note, but as a separate event), which in midi is 0-127. There is also release velocity, which may or may not be used by the instrument.
There's also other stuff like pitch bend, aftertouch etc, which you can store as a separate event.
Note name / ID: Why try and store this on the note? If your pattern has e.g. an array or vector of 35 notes, then you know its ID as you access it.
An example to start with might be something like this:
int m_iStartTime; // in PPQN. this could be negative? if you want some notes to start before the official start of pattern
unsigned int m_uiLength; // in PPQN
unsigned int m_uiKey; // e.g. like MIDI have middle C as 60
unsigned int m_uiVelocity; // 0-127?
unsigned int m_uiReleaseVelocity; // 0-127?
Once you have a simple system working then it will become more obvious where to add things.
To reiterate on the notes side of things, don't worry so much about space saving, just concentrate on simplicity. Note data doesn't tend to be that large. It's more when you get to the audio side you need to pay attention to the data structures / bottlenecks.
And rather than just having a struct-like class you can use accessor functions so the actual data underneath can be anything you want.
Posted by lawnjelly on 15 December 2012 - 08:55 AM
As L. Spiro says, don't store you times as floats or something like that, store than as ticks.
Sequencers commonly work on a scale of PPQN (pulses per quarter note), so if you adjust tempo (once off or gradually throughout a song) it just *works*. The PPQN values are usually things like 48,96,192 etc.
Bear in mind that if you are doing 4/4 music that's all good, but if you are using triplets, or groove, you'll want the PPQN divisible by 3, and with enough precision for your 'groove'.
You'll also probably want to store your note timings as offsets from the start of a pattern, rather than the start of the song. This way you can several instances of the same pattern at different parts in the song.
Also instead of storing things as e.g. char to save space, it's probably more sensible just to make them 4 byte unsigned int / ints and keep your structures 4 byte aligned so you (or the processor) aren't faffing about for no reason. You can always compress them on import / export, if you really need to.
Another reason for PPQN is so you easily change the output sample rate (assuming you are going to do some audio instead of purely MIDI).
I've done several audio / sequencing apps and don't think I stored anything as floats. PPQN can be used to calculate the exact sample for an instrument to start / end (and you might precache this kind of info). You could possibly use something more accurate to get within sample accuracy for timing, but I've never bothered myself.
It's really worth using a plugin architecture for different components of a sequencing / audio app, I'd highly recommend it. You can make effects (reverb, delay, chorus etc) plugins, and instruments plugins. You could potentially also use VST plugins or similar if you can work out their interface (you may find some open source apps that have managed this).
I'm currently rewriting a sequencer / audio app I wrote a few years ago, and have actually moved to using plugins for things like quantization / legato / groove / argeggios. Have a think about whether you want to be able to do stuff like 'undo' quantization, keep original values, or have a modification 'stack' applied to notes.
I don't think you'll get the exact structures bang on first time, it's the kind of thing you write a first version, then realise there's a better way of doing it, redo it, etc etc. But it is fairly easy to get something usable. You may also spend as much time on user interface / editing features as the stuff 'under the hood'.
As for APIs, I have so far cheated and don't actually use MIDI input or output (although I have done that in the distant past and it wasn't that difficult I don't think). I have just been writing a MIDI file importer though refreshing my memory lol.
If you want realtime MIDI input you'll have to pay much more attention to latency and the APIs you use. I was just getting by with the old Win32 audio functions for primary / secondary buffers, but the latency is awful, so using direct sound or I think there may be a new API in windows 7 would be better. Sorry can't help yet in that as I haven't researched it myself yet.
Also I'd add, consider using direct3d or (in my case) opengl to accelerate the graphics side. This way you can easily show the position within a song without overloading the CPU and causing stalls and having your audio stutter.
Once you start doing the audio side a bit of SSE / SIMD stuff helps. And you have to think carefully about how you'll structure your tracks / sends to effects, to make it efficient but also customizable.
Posted by lawnjelly on 08 December 2012 - 05:27 AM
When there's no user interaction, there's a lot of shortcuts you can take. Objects may only need to be built with one viewing angle for instance. You don't tend to need physics representations in the same way. And the whole thing can be prerendered with smoother curves and effects. On the other hand the detail expected tends to be higher for movies.
A lot of games have scripted elements. This would be more akin to a movie sequence. Game designers have to choose how scripted a game is (which is good for storytelling), and how much choice there is. Sometimes it can be difficult to force the intended story without giving the player the impression that the game is linear.
As the others say, there is a cutoff point, where your game is so linear, that you might as well make it pre-rendered or a movie, so you can take advantage of those techniques.
Posted by lawnjelly on 30 September 2012 - 10:33 AM
No matter what language you use, there is always something that has to be distributed with it.
With C/C++ programs, you'll have to distribute the proper library DLLs and make sure the user has the right version of the C/C++ runtime installed. Then there are video card drivers. The DirectX user redistributable, etc...
So nothing will change. Just the name of the language!
I see what you are saying, but I'm not sure it's strictly always true, it depends on the type of game / app you are writing, and the market (think e.g. casual games, apps, versus AAA games).
If you write c / c++ and statically link to the runtimes, there are no dependencies due to the language, only core OS dlls (which will always be present).
The need for more and more sprawling dependencies is up to you, your choice of language / tech and what third party stuff you decide to pull in. (Sometimes you can statically link to third party stuff though).
I think azonicrider is right to an extent .. end users are easily put off installing stuff. If your game needs a 15 minute download from a third party and separate installation just to run, they'll probably move on. I know I do. If I see a 'your java needs to be updated' or flash or whatever, I'm like 'forget it'. So there is a good argument for considering your market before deciding what dependencies to rely on.
Posted by lawnjelly on 30 September 2012 - 04:48 AM
I personally find 3d packages incredibly tedious and time consuming, but have recently been converted to 2d vector animation. I'm currently using Anime Studio, but SynFig is another free option.
If you build your art out of vectors, you can then rescale it, animate it (with bones etc), easily change colours etc down the line without needing to do loads of extra work.
As for creating the artwork from line drawings, there is PoTrace in Inkscape and a few other options. In the end I wrote a little app for doing centreline autotracing (from e.g. a scan or photo) which is a bit more what you typically want for easy editing:
The results still require cleanup though, so I tend to use a combination of this and mouse vector editing in Anime Studio.
End result is something like this (character in little 2d game I am currently making) :
Posted by lawnjelly on 25 September 2012 - 06:52 AM
As a caveat I haven't personally dealt with this type of scenario, but here's some guesses:
When it does break down I believe it is usually best to rely on the server being authoritative. If you 'see' on the client you've planted a bomb and trapper another player, but as far as the server is concerned, that player has already moved out of the way (their input is ahead of yours, for example), you have a choice, you either let the server be authoritative and you correct your client, or you wind back the server, and say 'hey this client made a hit' and replay everything from there and send the result to everyone.
I personally wouldn't want to wind back the server, it strikes me as a bit of a nightmare in terms of balancing, potential for cheating, etc .. (what happens if there's a stutter on the client, and the other player is not updated, and thus easier to trap?).
So with an authoritative server you are going to get situations where either the trapper or the trappee is corrected (or both), and it's going to look kind of annoying, like in a FPS when you get shot round a corner or snapped back. Whether this totally mucks up gameplay is a question - some games maybe are not suitable for this sort of multiplayer I guess. You'll just have to try it and see.
It maybe something that works as a splitscreen game (with zero lag), but totally falls down when playing with lag.
The other thing you could do if all else failed, easy solution - don't use client side prediction lol. Then the problem is solved. It just means you'd have to either play with fast connections, or lan games, or try and hide the delay somehow in the gameplay.
There may also be other possible solutions with client-client communication, but I don't know much about that.
Posted by lawnjelly on 24 September 2012 - 07:28 AM
I'm toying with sending inputs in a sort of reliable fashion. It's a queue of inputs, and I keep sending them until they are acknowledged. I also have a bandwidth throttle to keep the bandwidth usage in check.
This is a fairly standard way of doing it, I've been doing it for a while, there's a gaffer on games article describing this I think:
I think (without looking at the code) I send the server calculated position with the ack for the client input. In fact this may all be part of the regular update packet from server to client (with actor positions).
Then when the client gets the ack, it can compare the server calculated position with its own client side predicted, and if it's different, roll back and recalculate the client position based on the input history. And once it has the server ack up to tick 'blah' it only now needs to send input out from tick 'blah' onwards, etc etc.
Posted by lawnjelly on 24 September 2012 - 02:31 AM
A question out to all the code-heads out there. I'd like to get a sense for what you as a programmer like to see out of designers other then cold hard cash to spend a few dozen/hundred hours on writing code. Obviously a good idea is a pretty useful thing but lets get beyond that. When you're reading the classifieds you're looking for something out of a team or individual to instill a sense of commitment and components to indicate a unified design idea but what are those components. In a priority list what matters to you? Art, documentation, pre-recorded audio, etc?
I can't speak for other programmers, particularly beginners (to whom this question might be more relevant).
Being a 'designer' is a very difficult sell to try and start a project yourself. Unless you have prior experience / track record, realistically you are probably going to need money to pay, or have other skills (do you have a good looking girlfriend, who is *really* committed to the project?).
If we take money out of the equation, other skills would be (as AlterOfScience says) things like producing artwork / programming, and *possibly* running a business (prior experience). I would have thought producing artwork is most likely to be the successful avenue, as that is what most programmers usually can't do themselves / aren't interested in doing. To be realistic, if you have any chance of being part of a non-paid project doing purely 'designing' (probably writing scripts or making levels), it's most likely to be joining an existing project rather than starting one yourself (and consequently working on someone else's ideas, that's what a designer does 99% of the time).
I would suggest to anyone who wants to get started in designing to either start making games themselves, with a game toolkit or mod tool of somekind that doesn't require programming, or to learn to produce artwork of some kind, either 3d or 2d, so they have some marketable 'skillz' to bring to the game. For indie games, artwork doesn't necessarily have to be AAA quality, you just have to be prepared to spend the time learning how to do it, and spending the hours and hours and hours on it.
In short, designer is a hard sell. Designer / artist (primarily) and to a lesser extent designer / programmer is a better bet.
Not really. The game world isn't short of ideas. It's short of people who have the skills / time / commitment to do stuff. But I'm sure this has been discussed ad infinitum.
Obviously a good idea is a pretty useful thing
Posted by lawnjelly on 08 September 2012 - 07:02 AM
Yup, that's pretty much how we ended up doing it too! Snap lol. I think it probably ended up as a malloc on a reserved heap on pc build for the level file, and just loading into the prereserved block on consoles.
For the streaming I think I had more chunks (I called them banks), maybe 8 or 16 something like that, then parts had the option to use e.g. 2 banks worth.
For deciding which banks needed to be loaded I used a PVS calculated from the artist's levels and portals, and a potentially loadable set derived programmatically from this. There were areas though I'm sure where the artists had overcooked it and they needed to put in visibility blocks of some kind. I think the tool chain alerted them to this. Worked a charm, especially with decent asynchronous streaming support.
Getting way off topic though there hehe!
Posted by lawnjelly on 08 September 2012 - 01:18 AM
So, I mostly agree with the rest of your post, but this point isn't quite as straightforward as you suggest.
The other is that there is no question over the time taken over a deallocation / allocation. It is determined by your code and can be tightly determined - usually a constant very short time.
Malloc/new are not deterministic. The cost of an individual allocation is generally much higher than that of a garbage collector, and it is not a fixed cost. But you do get the (to my mind, dubious) benefit that the performance cost is incurred at the call site (whereas garbage collection incurs a performance cost at an indeterminate later date).
If you actually need deterministic allocation cost, then you have to go with other solutions (probably ahead-of-time allocation: pool allocators, SLAB allocators, etc.)
Ahha .. this may be where the confusion lies.
I didn't want to suggest 'using malloc / free at runtime is better than garbage collectors'. Far from it... they both have related downsides.
In c++, if you override new, you don't need to use OS calls for memory management. You can use whatever system you want for grabbing memory from wherever you want, then you have the opportunity to call the constructor yourself with placement new.
In addition there is a distinction between one off allocation / allocations at startup, and their corresponding deletion at shutdown, and dynamic use (i.e. the kind of things you might use lots of times in a frame). The second case is what we are interested in here. For actually reserving your memory at startup, you could use whatever you want .. an OS heap, garbage collected system. Ultimately your memory has got to come from somewhere.
(There is also the slightly less stringent case of level load / unload, where you *could* if necessary be a bit more lenient / take some shortcuts on some platforms).
What we are after in games, in an ideal world, for dynamic allocation (things that happen a lot rather than just startup and shutdown) is stability (no failed calls) and constant time (and fast) allocation and deallocation.
Sorry I should have been more clear on this. I would on the whole use things like fixed size memory allocators (and potentially other constant time allocators) for things that need to be created / destroyed dynamically (see my first post on page 1). You can use this for constant time incredibly fast allocations / deallocations, suitable for things like nodes in algorithms, even particle type systems.
For things that are truly variable size (levels etc) the tradeoff can be to prereserve space at startup for worst case, and work with that. Alright you lose a bit from the theoretical maximum, but you gain in simplicity and stability. On levels with not much geometry, you can e.g. add more sound, or more textures, and vice versa. For your level file you can prepack into the best format possible, with zero fragmentation, and make use of the whole of your budget in megs. If you need to use more than this, then you need to support streaming of level data on the fly (this is a whole other topic with similar concerns, guess what, you can use fixed size bank slots for this too!).
You can do this for GPU resources too .. reserve e.g. 5000 verts for a character and then stick to that budget or lower for your artwork, and you can guarantee they will always fit in that 'slot'.
You can also pre-designate blank 'slots' for various items in the level data RAM allotment to give more flexibility, if it seems a better idea than deciding ahead of time the maximum number of item 'blah'. If you do this you get the benefit of zero fragmentation, and best use of memory for that level.
In short there are lots of handy 'helper' bits of functionality offered to programmers, like 'general purpose' heaps, variable size strings etc. There are whole languages dedicated to making things 'easier' for the programmer where these things are a given (basic, php etc etc). In most situations this is a real benefit because it makes you much more productive as a programmer - less code, simpler code, less potential for bugs, and the 'costs' are not going to appear to the user.
It's just that in some situations, particularly time critical applications, and those on limited memory devices, it can become worth it to not use some of the helper functionality. An extreme example would be missile control software. You might have limited memory. If your program crashes, people die. If your program takes too long to faff around restructuring the heap, people die. It's only if it works predictably and as per spec that the right people die.
Other examples where you have to be a bit more stringent include things like financial software, medical software, some engineering software.
Would you want the nuke heading towards your neighbours house programmed in java with garbage collection, or c++ with no external allocations? I know know which one I'd rather have heading towards my neighbours.
(edit) Some good search terms to google in this area are : 'real time programming', and 'mission critical programming'. (/edit)
Posted by lawnjelly on 07 September 2012 - 02:39 PM
Well it would 'be nice' to be lazy and leave everything to a GC,
Sometimes the developer (or team) may not be quite skilled enough to have a choice and I hope to god that if working in a team, I wouldn't have to use some bodgy monstrosity.
In my spare time, I port software to FreeBSD and notice quite a few cases where developers have tried to hand roll their own stuff, nothing flags up bugs in this type of thing better than porting to an entirely new platform (and older version of GCC). So I suggest using a garbage collector unless you really know how to use the language properly.
While I havn't properly touched managed languages for well over 4 years, the Boehm GC works satisfactory on C++.
For software which needs no clever memory management however, the only solution is tr1/shared_ptr!.
Yup, don't get me wrong, in almost the majority of apps I'd be all for using all the tricks in the book to make things simpler. Garbage collection, you name it.
Sorry if I come off as opinionated on the subject, I was a bit unfair on you Karsten .. I've had to deal with the mess caused in the past and it's not been pleasant. It's not very fair when people's jobs are on the line, and their families depending on them etc.
It's just in the specific case of (professional) games, particularly on fixed low memory devices (and some other software on embedded systems), my personal belief is that controlling the memory yourself can be the best option. That doesn't mean it's necessarily the best approach for people learning .. it's more an approach for making a solid professional product.
The two main reasons I would argue for this are:
Stability - no worries about failed allocations .. your game will run each time, every time, no matter how many levels you load, what combinations of objects need to be loaded. There's no, ah but if character B walks round the back of building A, carrying object C and opens the door on level BLAH, then it crashes. Sometimes. Which is pretty much what you don't want to hear about when you are trying to ship something. Or what happens if someone is running such and such a program in the background in a multitasking environment.
Of course it's possible you could get round this to some extent with your Garbage Collection system - if it can allow you to pre-reserve your memory, (depending on its implementation regarding fragmentation), and if you keep a tight handle on your numbers of various objects. But once you get to this extent you are almost doing the work of doing it yourself anyway.
The other is that there is no question over the time taken over a deallocation / allocation. It is determined by your code and can be tightly determined - usually a constant very short time. There's no worry about dropping frames etc. Using a third party allocation / deallocation system leaves you at the mercy of their implementation. That's not to say there aren't good implementations, but there are also bad ones, and worst cases. Windows for example is quite happy to grind to a halt and do some disk swapping when it thinks it's necessary during an allocation / deallocation.
I fully understand that it can be a bit of extra effort (sometimes quite a bit) to manage memory yourself, although it's usually mainly a one off cost setting up your project. But development isn't just the time putting the code together, it's also beta testing, trying lots of different scripts, game levels, combinations of factors. In this situation the more potential problems you can remove the better.
If you are working to a time schedule with milestones and a budget and staff costs to pay, the last thing you want is some vague uncertainty over 'yeah it may take 2 years to beta test this thing'. That's one of the (several) reasons why games get canned / companies go under.
But anyway at the end of the day it's up to whoever is technical lead on a project to make these kind of decisions. Right I'm tired that's enough essaying it's bedtime!
Posted by lawnjelly on 07 September 2012 - 05:55 AM
Using OS calls for allocation and deallocation at runtime in a game is one of the cardinal sins, but GC too? Urggg!! Do you know what these calls do behind the scenes? I wasn't even going to mention it.
If you do look into memory pools, stuff can get quite complicated so sometimes it might be nice to leave it to the GC platform's memory pool.
Well it would 'be nice' to be lazy and leave everything to a GC, but unfortunately there are reasons why people don't tend to use this kind of thing for time dependent stuff. I understand looking after memory is 'an extra bother' and 'complicated' but it's necessary if you want to make fast, stable code. I've also had to spend weeks sorting out problems caused by 'programmers' who thought memory management was 'a bother', and delayed shipping products, and left them bug ridden messes.
Posted by lawnjelly on 07 September 2012 - 04:54 AM
The downsides are you have to (typically) know in advance how many of the objects maximum you will want worst case scenario. In addition the memory preallocated for the pool is not available for other uses.
Upsides are they are blazingly fast, constant time allocation and deallocation, there is no fragmentation, and provided you choose the maximums correctly your program CANNOT crash due to an allocation failure.
You can also implement your own heap with buckets but it's not something I'm a fan of.
You can also (in c++) override new and delete to keep track of your allocations. You can use different heaps / counters for different modules, and budget your memory between them. Very useful on consoles and limited memory devices. This can also report to you any memory leaks on closing, which module and which file they are from.
Other tricks are things like, when you load in a game level, load it as a binary file laid out in usable form in memory. Then fixup the pointers within it from offsets within the file to actual locations in memory. This gives you super fast loading, no fragmentation, and cache coherency. And of course level size etc is one of the biggest 'changables' within a game, so if you can isolate this down to one allocation, you shouldn't really need to do much else in the way of allocation. And even for this you can just pre-allocate a big chunk for the biggest level size, that's what I've tended to do on console-like environments.
Of course this is for game code, where stability and speed are paramount. For tools and apps I'll be a lot more lax, and use dynamic allocation etc (sometimes I don't even override new and delete, when I'm feeling like living life close to the edge ).
It's also worth mentioning that there are some allocations you can't avoid, depending on the OS - API allocations such as directx and opengl. You can of course use pooling systems with your API resources too. In addition on consoles you can often completely avoid this problem by using a resource directly from memory as they may be UMA or give you more control over memory.
Posted by lawnjelly on 07 September 2012 - 02:41 AM
It can be a bit of an effort to pull your finger out and write initially write these tools but once you have them you realise you couldn't live without them.
Posted by lawnjelly on 01 August 2012 - 07:57 AM
But you have to have a playstyle that will work with it, otherwise you end up needing to implement a full collision detection / physics system in addition to the navmesh. I think the projectiles could be the problem, you are going to end up needing either a full collision mesh for the level, or at least building a simplified version of it for the projectile collisions. At which point you have to ask whether it would be better to have full physics main character and be done with it.
An alternative is to tweak your game design so you don't need the full collision mesh. Perhaps instead of using projectiles, only allow your characters melee combat.