Intel sponsors gamedev.net search:   
BrainfoldBy Knarkles      

Blog at http://www.brainfold.org/blog/.

Visit #python3d on the Freenode network.

Page:   1 2 »»

Tuesday, May 30, 2006
Nothing exciting happening with Spineless. I've been writing new tutorials and fixing small things as I encountered them while writing. I would definitely welcome comments on the tutorials. You can get them here.

I noticed that writing tutorials is an excellent way to get effectively a fresh pair of eyes looking at the project. Since you have to explain everything, you encounter things you have actually gotten used to yourself, but which don't make that much sense when trying to explain them.

Every comment gets a free cookie!

Btw. Page hit count on the Spineless web page has quadrupled since yesterday after I released the development snapshot and submitted it to Freshmeat... OMG.

Comments: 0 - Leave a Comment

Link



Monday, May 29, 2006
I rewrote the setup script of Spineless, resulting in much better detection of FMOD, and ODE now being an optional dependency, only needed for collision detection and physics.

I also released a development snapshot of Spineless now that 0.3 is feature complete. Quoting from the news item on the web site:

Quote:
Spineless 0.3.0 is now feature complete. This means "all" that's left to do is testing (and lots of it) and writing decent documentation - finally. As promised, I released a development snapshot. You can find it on the Download page. Note that it hasn't been seriously tested and all documentation is seriously outdated, but at least I could get it built and the tests run on all three platforms (Windows, OS X and Linux). Have fun, and please report any problems or bugs!


Still no screenshots; expect those when I start writing new tests and continue working on the Secret Game Project.

Btw, I found a really stupid bug in my triangle mesh collision test, which resulted in strange (and wrong) looking collisions. Turns out I was rotating the rendered mesh but not the collision mesh... *doh* (note: you don't need to rotate these separately in Spineless, it was just that I had the rotation in the lower level model node and not the higher level node containing the collision mesh and model).

Comments: 1 - Leave a Comment

Link



Sunday, May 28, 2006
Spineless 0.3 is now feature complete. Yayzers! Nothing new to show you, I've only implemented ray picking and done some bug fixes and cleanup since the last update.

Here I come, tests / documentation / bug fixes / setup scripts!

Esteban is still working on the Collada loader, which basically works too.

Comments: 0 - Leave a Comment

Link



Friday, May 26, 2006
Yay, I got triangle mesh collisions implemented in Spineless. Needs a bit more cleanup and optimization though... but it's working well. Secret Game Project now has cubes falling on the terrain as a test. Next: camera.

Screenshots for you to lust on later, got to go now. here:

[deleted image]

Also, geometry is now loaded from Collada files. Materials etc. still to come.

Whee, almost 1200 1300 views.

Comments: 0 - Leave a Comment

Link



Tuesday, May 23, 2006
To make the journal slightly less boring, here's a screenshot of my quickly hacked-together image loading for height maps:

[deleted image]

That's 1 million polygons. Loads slowly as hell, as the code is quickly butchered-together Python and not Pyrex. In actual news, I fixed physics except for the problem in the last entry. That's practically all the features done for Spineless 0.3. Off to tweak mode and documentation writing next!

Comments: 3 - Leave a Comment

Link


Ok, I'm stumped on a problem. The said problem is halving the speed of my physics benchmark and causing fps go up and down a lot, like a little monkey on a bungee cord. Ok, so that was the worst analogy of the week, but that's beside the point. The problem has two parts and I'm going to describe them first.

Collision detection in Spineless uses ODE, and ODE must of course know the location of collision geometry to do its thing. So the Space class, responsible for spatial relations of nodes, keeps track of locations of nodes with collision geometry and forwards them to ODE.

Physics simulation in Spineless also uses ODE, and locations of rigid bodies must, of course, be extracted from ODE to use them in the engine. So each step after physics simulation is updated, the Physics class sets locations of all nodes which contain a body.

Combine the behavior described in the last two chapters with the fact that the ODE geometry and body are already linked and maintain the same location, and you can probably see the problem. That is; for each physically simulated node for each step, location of the collision geometry is set completely in waste, because it's already correct.

Now, an obvious first thought for a solution would be to just disable location updates for collision geometry with connected rigid bodies. This would also prevent the user from changing the location of these nodes, so it's no good. I could set a private flag in the Node for each physics update, check it in Space and not update location if it's set, but that reeks of a stinky hack to me.

Bright ideas, please? I'll give you as many cookies as I'm able to find! :)

EDIT: Esteban (my partner in crim... I mean coding) came up with an excellent solution to the problem: for nodes with physically simulated rigid bodies, physics simulation becomes the "owner" of the node's location, and updating it directly doesn't work anymore. You have to update the rigid body's location instead. IMO this is good because it's making it explicit that you're "cheating" the physics and doing something unusual.

# This doesn't work anymore
node = Node()
node.body = Body()
node.position = Vector(1, 2, 3)

# This is the way to do it from now on
node = Node()
node.body = Body()
node.body.position = Vector(1, 2, 3)



Comments about this solution are of course still welcome.

Comments: 0 - Leave a Comment

Link


I also completely broke physics while doing cleanup on the interface, oops... everything except joints seems to be working now, though I do have to do some more implementation-level cleanup. After the physics interface is nailed down, I'm more or less ready to go to documentation/testing/bug fixing mode for Spineless 0.3. Hoorays!

...not much else happening. I'll post more about Secret Game Project after I have implemented collision detection for triangle meshes (and by extension, heightmaps). As I said, I'll let SGP pretty much drive the development of Spineless now (but of course keeping engine code generic as usual).

Any wishes about which part of Spineless I should discuss next? Resource management perhaps?

Comments: 0 - Leave a Comment

Link



Monday, May 22, 2006
Spineless

I've made some decent progress on Spineless. Cleaning up the physics interface is nearly complete, but it's still buggy. I did some more testing on rendering the height map and noticed that I had hardcoded index buffer element type to unsigned short... so meshes with over 65k vertices caused some interesting problems. I'm fairly sure this is also the reason I got segfaults with some models a long time ago. As a temporary solution, I hardcoded it to unsigned int. :P I will probably make this automatic so that it's chosen based on the number of indices. Anyway, a heightmap with about a million polygons was rendering at a bit over 100fps, so at least Spineless isn't polycount limited now. ;) Granted, any renderer can easily do this with vertex buffers, but it was still nice to see. I'll post more screenshots of the terrain after I generate it with something a bit more intelligent than completely random points (current heightmap implementation was written in about 15min).

EDIT: Discovery: a 3072x3072 texture loads 4 times slower than a 4096x4096 one when using gluBuildMipMaps (around 15 seconds versus around 4 seconds). It's specifically this call which slows it down. I guess the texture is first copied to a 4096x4096 one before building mipmaps? Behold the power of the power of two.

Secret Project

Nothing new for the game project; I've been concentrating on nailing down the Spineless interface (at least for now) so I don't have to constantly adapt the engine changes.

Renderer

I was going to write about the Spineless renderer earlier but I sort of forgot about it, so I'll do that now. Like all scene management in Spineless, rendering is based on arbitrary attributes of nodes. Render operations (ops), which you set as node attributes, roughly correspond to OpenGL calls, eg. "lighting", "projection" and "viewport". Each op has apply(), save() and restore() methods. Rendering is based on walking the scene graph (actually a tree) except for light nodes, which are stored in a list and applied globally (N closest lights to the viewpoint at the moment; it would be easy to change the algorithm). Then, for each node:
1) Save and apply all render ops and node transformation
2) If the node contains a renderable attribute:
- Add it to the list of transparents if it's transparent
- Else render it
3) Descend to child nodes and start from step 1 with them
4) Restore all ops and node transformation

After this, sort collected transparent nodes and render them. Of course, a lot of caching is done with attributes to speed up rendering, and there's more to do. The renderable attribute can be any object with a render() method. At the moment, Spineless comes with two such objects: Mesh and Text. I'm thinking of implementing Billboards as decorators (ie. something like node.render = Billboard(myMesh)), and particle systems will probably also plug in to the render attribute. Later, culling based on hierarchial bounding radius (ie. bounding radius of the whole subtree) will be done before step 1. I'm yet to generalize multipass rendering (technically rendering opaques and transparents could be considered multipass rendering), but I just might borrow Ysaneya's concept of pipes. It seemed like an elegant solution.

As always, comments and questions are more than welcome.

Btw. I can't think of a good name for the attribute containing the actual renderable object. At the moment it's called "visual", but I'm not sure if that's a good name since it's an adjective. "render" could be better, but I'm not sure... "renderable" would be accurate, but the name is clumsy. Suggestions?

Comments: 3 - Leave a Comment

Link



Sunday, May 21, 2006
Behold.

[deleted image]

Yes, it's a crappy model, well, at least the materials are, but OBJ loading is (more or less) working now! I have two conclusions though:
- Model importers of free / cheap (Milkshape 3D) modeling packages suck big time
- Model exporters of free / cheap (Milkshape 3D) modeling packages suck big time

It took far too long to find a model that would:
1) Be correctly imported by Blender, Wings 3D or Milkshape 3D
2) Didn't generate errors in Blender's OBJ exporter (it seems Milkshape 3D's OBJ exporter is decent though)

Can anyone recommend a good 3D model format conversion tool that doesn't cost an arm and a leg?

EDIT: This will eventually be terrain for the Secret Project:
[deleted image]

EDIT 2: I was kinda wondering why rendering the simple height map (100x100 map = 20000 triangles) was only running at 85fps. Python isn't blazing fast and Spineless isn't very fast either, but it shouldn't be that slow. Guess what my monitor's refresh rate is? It's now running at a healthy 800fps with vsync disabled.

Comments: 0 - Leave a Comment

Link


I started writing the OBJ model loader today. Conclusion: Blender's OBJ exporter is crap. It kept spewing errors at me... and now I'm not sure if my model looks funny because of exporter errors or bugs in my code. I need some test data from other modeling packages. Anyway, it looks "mostly right", but not right enough to screenshotify. I'll come back to that later. Also, some (not very large) jpg images seem to take an awfully long time to load. I wonder if it's a bug in Pygame. Or maybe something funny in my image loading code. Needs more research.

In other news, I found out that ODE doesn't support scaling of collision geometry. So no scaling matrices for you, ODE. That means scaling is now only useful for graphical special effects. Oh well. Hmm, Newton does support scaling of collision geometry and seems to be better overall than ODE... so looks like I'll be supporting both. I suppose no one has any clever ideas for hacking scaling of collision geometry in?

I've also fixed a crapload of other things, and done a lot of cleanup. It's actually looking like Spineless will soon be useful again, and much more elegant. Here we come, milestone 0.3. Hoorays!

Comments: 1 - Leave a Comment

Link



Friday, May 19, 2006
Ok, I'm not sure if doing a todo list generator is useful after all... it's really not that difficult to just copy/paste lines and change color of the line when its status changes.

I started converting the geometry package to the system I mentioned in the edit to yesterday's entry, and so far it's looking good. The geometry package won't depend no half the engine anymore, because there will be no geometry package anymore. ;) The major thing I'm worrying about now is ODE integration... it's still going to be hairy.

I'm also going to write a Pong clone before I continue working on my Secret Game Project, I need at least something completed using Spineless. :) Random tidbit about SGP (no, that's not its real project name): it's about big weapons.

I think I'll talk about implementation of my renderer in the next entry.

EDIT: I put the full Spineless todo list on the wiki

Comments: 0 - Leave a Comment

Link



Thursday, May 18, 2006
Ok, so I'm not happy with my geometry package. It's responsible for representing geometric objects, such as boxes, spheres and triangle meshes. Here's how you use geometry at the moment, creating a node rendered as a sphere, and collision geometry of the same shape:

geom = Sphere(1) # 1 is radius of the sphere
geomNode = Node()
# 20 and 20 are the amount of horizontal and vertical slices
geomNode.render = geom.createPrimitive(20, 20).render
geomNode.collisionGeometry = geom.createCollisionGeometry()



This has two problems: you can't change collision geometry after you've created it, and geometry has too many responsibilities (geometry classes should be low-level, not caring about graphics and physics). What I'm thinking now is to:
- Have geometry objects create only vertices; normals and texture coordinates can be created afterwards
- Have a factory function in the physics package for creating collision geometry, which would return instances of collision geometry classes

You would use it somewhat like this (I haven't designed the API yet, just pulling something out of my nonexistent hat):

geom = Sphere(1)
vertices = geom.generateMesh(20, 20)
normals = generateNormals(vertices, smooth=False)
texCoords = generateTexCoords(vertices, type="spherical")
primitive = RenderPrimitive(vertices, normals, texCoords)

geomNode = Node()
geomNode.render = primitive.render
geomNode.collisionGeometry = physics.createCollisionGeometry(geom)



More complicated? Yes. More flexible? Yes! Will I provide a higher level interface too? Yes! Will you use mostly use model loading anyway, instead of geometry objects? Yes! The smooth parameter to generateNormals() would dictate if we are creating vertex or face normals. The type parameter to generateTexCoords() specifies the type of projection (spherical, cylindrical, planar etc). Or there could be separate functions or classes for different projections. I dunno... as I said, haven't designed the interface yet.

Maybe geometry could also optionally create normals and indices, since it's already doing the necessary calculations when generating vertices. Like this: vertices, normals, indices = geom.generateMesh(20, 20, normals=True, indices=True)

Comments? Suggestions? Bestest ideas ever, just for me? Please? Pretty pretty please?

(Hmm... maybe I need to change node.render back to an object with a render method instead of a callable, since later I will need the bounding radius of renderables for frustum culling etc...)

Btw. I'll see how easy it would be to create a HTML/CSS TODO list generator with Python. :) I'll post the results or report of my failure later.

EDIT: Ok, I was truck with a Revelation about this, and I think it would be the cleanest way. I was just thinking... how often do you need to render geometry primitives, and does it make any sense to actually keep render geometry and collision geometry in a single class? I have the answer for you: no it doesn't. So, here's aboutish how it will work:

# Alternative 1
vertices, normals, texCoords = graphics.geometry.createSphere(1, normals=True, texCoords=True)

# Alternative 2
vertices = graphics.geometry.createSphere(1)
normals = graphics.geometry.generateNormals(vertices, smooth=True)
texCoords = graphics.geometry.generateTexCoords(vertices, "spherical")

primitive = RenderPrimitive(vertices, normals, texCoords)

geomNode = Node()
geomNode.render = primitive
geomNode.collisionGeometry = physics.geometry.Sphere(1)



Note that in this example, node.render is an object with a render() method again, not a callable.

Any comments on this? :P

Comments: 0 - Leave a Comment

Link


My Plan:
- Finish release 0.3 of Spineless, needs:
* A bunch of fixes and cleanup
* Model loader for Collada and/or OBJ
* Crapload of documentation and testing
- Start working on my game project "full time" (as in for the full time I have for hobby coding) and let it drive development of the engine, adding features as I need them (and taking requests from anyone else possibly using Spineless)
- Tease people by releasing annoyingly small tidbits about the game project until I have something concrete (figuratively) in my hands

Yes, this I will do.

EDIT: I need a cool/lame TODO list at the top of my journal page because everyone else has one too!

EDIT 2: Also, I think I'll start adding features by first writing small prototypes / demos, then design a nice interface for it and write a finished version. Should keep the engine a bit cleaner from now on...

Comments: 2 - Leave a Comment

Link



Wednesday, May 17, 2006
Ok, so another entry for today. It seems I missed two critical parts: the overall sequence of how signals work in the scene graph, and transforms. So here they are, starting with signals.

I'll give a couple of examples of what happens when you add a new node or change an attribute in a node.

Setting an attribute:
myWorld = World()
myWorld.rootNode.logic = coolUpdateFunction
-> myNode.dirtyAttributeSignal(myNode, "logic", coolUpdateFunction)
-> myWorld._dirtyAttribute(myNode, "logic", coolUpdateFunction)
-> myWorld.space._dirtyAttribute(...)
-> myWorld.logic._dirtyAttribute(...)
-> attribute name was "logic", so if we're not already storing this node, store it so we can update it when myWorld.update() is called
-> myWorld.physics._dirtyAttribute(...)
-> for view in myWorld.views: view._dirtyAttribute(...)
->     view.renderer._dirtyAttribute(...)
->     view.audio._dirtyAttribute(...)



As you can see, setting an attribute results in quite a fuss inside the engine, but as I said in the last entry, setting an attribute is rare compared to using an attribute (Node.__getattr__ is not replaced so it's fast), so this makes sense. If I want to optimize this later, I guess I could implement a registry in the World class, telling which workers are interested in which attributes. At the moment, this is far from being the bottleneck.

Adding a new node to the tree:
myNode = Node()
myNode.render = myGeometry.createPrimitive().render
myNode.collisionGeometry = myGeometry.createCollisionGeometry()
myWorld.rootNode.addChild(myNode)
myWorld.rootNode.newChildSignal(myNode)
myWorld._newNode(myNode)
myWorld.space._newNode(myNode)
...etc



_newNode methods of the various workers will then inspect the node for attributes they are interested in (see the example in last post). That's really all there is to signals in scene graphs. All the rest is the various workers keepin track of interesting nodes and updating them as needed.

I'm also using Signals for event passing in my engine: I don't have a centralized event manager (anymore), and this seems to be a much more elegant solution, and didn't really add any dependencies that weren't there already. I'm littering signals around in various places where I find them useful. Last I added was stepSignal to World, which is called for each step; step size is always constant, and there might be zero to many steps per frame depending on FPS.

About transforms then. Since they are so common an operator, all functions for working with Node transforms are in an external module (they could have been in the Node itself since they are so commonly needed, but I decided I want to keep the Node class clean). You get the world transform of a node by calling transform.getWorldPosition/Rotation/Scaling/Matrix(). They will check if there's a cached world transform available, and calculate one if there isn't. When the node's "position", "rotation" or "scaling" attributes are set, World._dirtyAttribute() in turn calls transform._invalidateWorldTransform(), which just clears the cached world transforms from the Node. An example is probably in order:

myNode.position = Vector(1, 2, 3)
# World transform cache is invalidated, so the following call will calculate it
print transform.getWorldPosition(myNode)
# World transform is now cached, so the following call will use the one from cache
print transform.getWorldRotation(myNode)



There are two transform utility functions at the moment in addition to world transform functions: lookAt() and getVelocity(). lookAt() is just a wrapper for quaternion.lookAt(), and gets a node whose rotation is to be calculated, and a target node to look at. getVelocity() calculates the velocity of a node based on time since last update and the previous position (which it stores itself).

More questions are welcome as always. :)

Btw. I have a reeeally basic heightmap working for my game, but it doesn't even have normals calculated (Spineless doesn't calculate normals automatically yet), so no screenshots for now. :)

Comments: 0 - Leave a Comment

Link


Even though the enthusiasm about the last post about scene management wasn't as overwhelming as the previous one (0 comments instead of the whopping 4), I'm determined to continue. Tonight, I'm going to tell an epic tale about the current implementation of my scene graph.

So, the nodes are in a tree, with only the root node assigned to the World object. A trivial implementation just walked the whole tree each step while each worker processed each node. Slow. A tree is far from the best structure for all the workers, and nodes that are not interesting to a worker shouldn't be processed at all. So, how to keep track of changes to the tree structure and its attributes? Callbacks? No; since a node could potentially be in multiple worlds at a time, a single callback won't do. Signals! Signals are more or less functors implementing the Observer pattern, and here's the trivial implementation and an example of usage:

class Signal(object):
    def __init__(self):
        self.slots = []
        
    def connect(self, slot):
        self.slots.append(slot)

    def disconnect(self, slot):
        self.slots.remove(slot)

    def __call__(self, *args, **keyargs):
        for slot in self.slots:
            slot(*args, **keyargs)

foo = Signal()
foo.connect(bar)
foo.connect(baz)
foo("hi there", 2)



This would result in calls to bar("hi there", 2) and baz("hi there", 2). The Node class has signals for added child nodes, removed child nodes, new (or replaced) attributes and removed attributes. When the World object's rootNode is set, it connects to the node's and all child nodes' signals. When the new node or removed node signals are fired, its signals are connected to or disconnect from, respectively. All signal calls are forwarded to all workers. This is essentially what all caching in the workers is based on, and allows them to have the node in any data structure that best suits their purposes. Here's the code for the added and removed attributes signals, implemented by replacing the attribute setter and getter methods respectively (for those who don't know Python, if you type eg. myNode.foo = 5, it results in a call to Node.__setattr__(myNode, "foo", 5), and attributes whose name starts with an underscore are considered private):

def __setattr__(self, name, value):
    object.__setattr__(self, name, value)

    if not name.startswith("_"):
        self.dirtyAttributeSignal(self, name, value)

def __delattr__(self, name):
    object.__delattr__(self, name)

    if not name.startswith("_"):
        self.removedAttributeSignal(self, name)



Note that no signals are fired for private attributes, since these are used by workers for internal bookkeeping and caching. You might think this is too slow, but modifying the attributes of a node happens far less than using them, so this optimization makes sense. For example, the Space class (as mentioned previously, responsible for spatial relations of nodes) stores nodes in both a list and a space object of ODE for collision detection. Later, it'll be easy to write another space class using an octree or some other more optimized structure without changing the API. Here's how the signal handlers of Space are implemented (I've ripped out some lines irrelevant to the discussion):

def _newNode(self, node):
    self.nodes.add(node)
        
    if hasattr(node, "collisionGeometry"):
        self._updateODEGeometry(node)

def _removedNode(self, node):
    self.nodes.remove(node)

    if hasattr(node, "collisionGeometry"):
        self._removeODEGeometry(node)

def _dirtyAttribute(self, node, name, value):
    if name == "collisionGeometry":
        self._updateODEGeometry(node)
        return

def _removedAttribute(self, node, name):
    if name == "collisionGeometry":
        self._removeODEGeometry(node)

def _updateODEGeometry(self, node):
    odeGeom = node.collisionGeometry
    self._ode.SpaceAdd(self._odeSpace, odeGeom)

def _removeODEGeometry(self, node):
    odeGeom = node.collisionGeometry
    self._ode.SpaceRemove(self._odeSpace, odeGeom)



(I just realized I could optimize this by passing collisionGeometry to the ODE integration methods and save an attribute lookup) The Physics worker doesn't use any structure to store nodes, since everything is handled by ODE. The Audio and Logic workers use simple lists. The Renderer still walks the graph, but it's a bit more optimized now, and I have quite a few optimization ideas left (there's not even frustum culling implemented yet). And this is all there is to it. Nothing magical (well, except that Python's special attributes whose name is like __something__ are sometimes called magic attributes), but it's really flexible with a simple API that doesn't need to change at all when new features are added or old ones reimplemented or optimized.

I hope this was useful to some people, and I'll gladly answer any questions you might have about the details. If there's interest, I'll continue describing other parts of my engine. Well, at least the parts that I'm happy with. :) Would you like to hear more details about the renderer? Or resource management? Or maybe I should post about the identity crisis of my geometry package, as I'd like some feedback on the issue.

Later, folks.

Btw, I started on my game project this morning. Expect fabulous screenshots in a couple of days.

Comments: 3 - Leave a Comment

Link

Page:   1 2 »»

All times are ET (US)

 
S
M
T
W
T
F
S
1
2
3
4
5
6
7
8
9
10
11
24
25
27

OPTIONS
Track this Journal

 RSS 

ARCHIVES
April, 2009
March, 2009
February, 2009
January, 2009
December, 2008
August, 2008
July, 2008
June, 2008
May, 2008
April, 2008
March, 2008
February, 2008
January, 2008
December, 2007
November, 2007
October, 2007
September, 2007
August, 2007
July, 2007
June, 2007
May, 2007
March, 2007
January, 2007
December, 2006
September, 2006
August, 2006
July, 2006
June, 2006
May, 2006