Jump to content
  • Advertisement
Sign in to follow this  
  • entries
    73
  • comments
    131
  • views
    55058

About this blog

Studying and researching things of programming the 3d space.

Entries in this blog

 

Moving on...

I'm canceling my gamedev.net premium membership soon, so if you want to follow my work, please follow my blog over at http://jlongster.com/blog. I still plan on working with Scheme and game programming, and I think it will continue to be really interesting stuff.

I'm not getting much out of this account, and I'm tidying up my budget, so $12/trimonth was nixed.

Cheers!

okonomiyaki

okonomiyaki

 

Farmageddon is released!

Posted from http://jlongster.com/blog/2010/04/03/farmageddon-available/

***



For the past couple months I have been working on a 3d iPhone game in Gambit Scheme. It is called Farmageddon and I submitted it to Apple a couple days ago. It was just accepted and is now live on the App Store! You can find it here.

You can check out a gameplay video here.



I am not sure what projects I will be working on in the future, but be sure that I will be doing more research in 3d graphics and Scheme and will continue posting my research here.

I plan to write up a much longer post-mortem about the whole project, so be sure to follow my blog. I learned so much throughout this whole process and I think it would beneficial for others to here about my experience. Especially when the project is as unique as a 3d iPhone game in Gambit Scheme. In general, Gambit Scheme worked great, with a few minor GC problems on the iPhone 3G because of slow memory.

If you think my game looks fun, or if you just want to support my work, be sure to buy my game. It's only priced at $1.99, and there will be a free "lite" version soon.



Many of you have responded to questions or simply encouraged me in my work, and I thank you for that. Also, sorry if I haven't been responsive with email, I've been pushing out this game recently.

http://farmageddongame.com/

Cheers!

okonomiyaki

okonomiyaki

 

Farmageddon Gameplay

I am coming into the final stages of my iPhone game, Farmageddon. I made a video showing about 2 minutes of gameplay and posted it to youtube here:

">

Please check it out and let me know what you think! I really wish gamedev let me embed videos.

Farmageddon is a game where animals have gone insane. They have turned against humans and fling themselves towards YOU, and you must be the one to stop them with your power of lightning before they destroy you (and crack your precious iPhone's screens)!

Chickens, ducks, sheeps, pigs, and cows are all part of the fight. Pigs and cows are especially dangerous because of their large, heavy bodies which explode into pieces that still hurt you.

There is a rumor that a few humans have been able to disguise a nuke as an animal and throw it into the fray. If you find these nukes and explode them, you will find that they destroy all the existing animals. Supposedly, the radioactive material in the nukes give the animal a greenish appearance, so be on the lookout.

If you would like to follow the progress of Farmageddon, which should be released on the App Store next week, be sure to follow this twitter account:

http://twitter.com/farmageddongame

There is a website too:

http://farmageddongame.com/

okonomiyaki

okonomiyaki

 

A look at my iPhone game: Farmageddon

Posted from http://jlongster.com/blog/2010/02/23/farmageddon/:

I thought it would be valuable to, for the first time, show some screenshots of the iPhone game I've been working on.

The game is called Farmageddon. Farm animals have turned against humans and the world is entering an apocalypse. If you stand by idly, you will be destroyed. You must use your power of lightning to explode the animals which are bent on assulting you by flinging themselves through the air and cracking your screen.

There are several levels, each having a specific goal. You might have to kill 5 animals without hurting any people, or you might just simply have to survive to 15 seconds. You must not kill humans which have been captured and thrown in to the air by the animals.

The game is very close to being done, but I'm still designing the levels so I will save a detailed explanation of the game until I release it. Until then, enjoy these screenshots, and please let me know what you think! As I've said before, I need help marketing this game, so if you like what you see let your friends know about it!

I also wanted to thank all of you who responded to my previous post about marketing ideas. I learned a number of things from various people, and I will try to reciprocate your generosity by posting my marketing plan in the future.


















okonomiyaki

okonomiyaki

 

A look at my iPhone game: Farmageddon

Posted from http://jlongster.com/blog/2010/02/23/farmageddon/:

I thought it would be valuable to, for the first time, show some screenshots of the iPhone game I've been working on.

The game is called Farmageddon. Farm animals have turned against humans and the world is entering an apocalypse. If you stand by idly, you will be destroyed. You must use your power of lightning to explode the animals which are bent on assulting you by flinging themselves through the air and cracking your screen.

There are several levels, each having a specific goal. You might have to kill 5 animals without hurting any people, or you might just simply have to survive to 15 seconds. You must not kill humans which have been captured and thrown in to the air by the animals.

The game is very close to being done, but I'm still designing the levels so I will save a detailed explanation of the game until I release it. Until then, enjoy these screenshots, and please let me know what you think! As I've said before, I need help marketing this game, so if you like what you see let your friends know about it!

I also wanted to thank all of you who responded to my previous post about marketing ideas. I learned a number of things from various people, and I will try to reciprocate your generosity by posting my marketing plan in the future.




img src="/media/images/farmageddon/farmageddon_8aaa.jpg" alt="/media/images/farmageddon/farmageddon_8aaa.jpg" />





okonomiyaki

okonomiyaki

 

The painful state of fonts with OpenGL(ES)

Posted from http://jlongster.com/blog/2010/02/08/fonts-ugh/:



Fonts. Ugh. I knew it would be a pain to integrate text rendering in
my OpenGLES-based graphics framework. I was able to avoid it for long
while using textures, but now that I'm about to publish an iPhone
game, there's really no avoiding it any longer.

I begrudgingly opened up google and typed in "opengl font rendering".
The first result was A Survey Of OpenGL Font Technology
straight from opengl.org.

"There is no native font support in OpenGL. The ARB has rarely
ever even discussed fonts." Yes, I knew that, so what options do I
have? That page lists a smattering of font rendering libraries,
most of them completely unusable due to some major restriction.
The only one with potential was
FTGL.

Searching for a solution

FTGL turned out to be my best bet, but the only problem was that
it targeted OpenGL and I'm working with OpenGLES, which lacks many
features of OpenGL. Luckily, it looks like David Petrie ported
FTGL to OpenGLES with the cleverly-named project
FTGLES.



I'm very thankful for David's work on the port, but I was
disappointed to find that I simply couldn't get the library to
work. Another minor gripe is that FTGLES only comes with an Xcode
project and not autoconf/automake scripts.

So I forked the damn thing and got to work.

At this point, I was frustrated. With all of the games being made
on mobile devices in OpenGLES, you would think there would be more
options or at least a clear and straightforward solution to text
rendering. What are all of you guys doing? And why aren't you
releasing or documenting it? Am I missing something? There
are a couple good articles on this such as
Adding Font support in OpenGL - Acorn Heroes,
but that's far from a good reusable library.

FTGL itself could use better documentation. I only found an FTGL User Guide
which only contains a tutorial for the basic functionality. There
are also a few API reference docs out there, but there really
should be readable, helpful, and thorough documentation for a
library that seems to be so critical for OpenGL applications (it
really seems to be one of the few solutions for font rendering).

We aren't even talking about smart textual layout yet. Writing an
OpenGL backend for Pango is
really interesting but there's only a proof of concept
finished at this point. Supposedly Pango might be too slow for
OpenGL applications, so I'll digress. Do games even integrate a
textual layout system or do developers do everything manually?

FTGLES

Back to my fork of FTGLES.

I integrated the FTGLES source code with the original FTGL
autoconf/automake build system so you can do the standard
./configure && make install. I also fixed a few bugs in
the source which make it behave more predictably. I plan to
contact David to see where he wants to take it from here, as I
don't really like maintaining a separate fork of a project.

FreeType 2 and compiling for iPhone

FTGLES uses FreeType 2 to load TrueType font files, so we need to
compile FreeType 2 for the iPhone. At this point I realized that
people might need to compile many libraries for the iPhone
(requiring cross-compilation to ARM), and it's really annoying to
always remember how to configure the project to build correctly
for the iPhone.

I created the configure-iphone
project to help with this. It is basically a single shell script
which wraps around your ./configure call and appends the correct
arguments to link with the right iPhone SDK and possibly
cross-compile. Please see the project README for more info.

With the configure-iphone utility script, I was able to compile
FreeType 2 like so:


wget http://download.savannah.gnu.org/releases-noredirect/freetype/freetype-2.3.11.tar.gz
tar xvzf freetype-2.3.11.tar.gz
cd freetype-2.3.11
configure-iphone 3.1.2 -t os
sudo make install


By default, configure-iphone installs libraries in
/usr/local/iphone, but you can override that with the -d
options. Our libraries would end up in /usr/local/iphone/iPhoneOS3.1.2.

Please note that there are articles such as this one
which describe tweaking parts of FreeType 2 in order for it to
compile; you do NOT have to do this anymore as far as I'm
concerned. Maybe they fixed that in that last couple minor
versions, but the above commands compiles FreeType 2 for the
iPhone just fine.

Compiling FTGLES

Compiling and installing FTGLES uses similar commands as compiling
FreeType 2. You must tell FTGLES where your FreeType 2 library is
located. Here is how it would look:


git clone git@github.com:jlongster/ftgles.git
cd ftgles
configure-iphone 3.1.2 -t os -- --with-ft-prefix=/usr/local/iphone/iPhoneOS3.1.2
make
sudo make install


Now just make sure you link with the ftgl library and add
/usr/local/iphone/iPhoneOS3.1.2/include and
/usr/local/iphone/iPhoneOS3.1.2/lib to your header and library
paths. The version and target may change, of course (target being
either OS or Simulator).

Using FTGLES

For general information about using FTGL, read through this
FTGL User Guide.

The only thing specific to FTGLES is that you must call
ftglInitImmediateModeGL to setup the vertex buffers correctly
every frame before rendering any of the fonts. It took me a while
to figure that out.

Extruded, pixmap, and polygon fonts are not yet supported in the
FTGLES port.

The following is some pseudo C code which assumes init is called
at the beginning of the program and render is called every
frame. It renders the text "Hello, World" with the Batang
TrueType font at the screen position (10, 0).



#include "FTGL/ftgles.h"

FTGLFont *font;

void init() {
font = ftglCreateTextureFont("Batang.ttf");
}

void render() {
// ... draw stuff ...

ftglInitImmediateModeGL();

glMatrixMode(GL_PROJECTION);
glOrthof(0.0f, width, 0.0f, height, -10.0f, 10.0f);
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();

glTranslatef(10.0f, 0.0f, 0.0f);
ftglSetFontFaceSize(font, 72);
ftglRenderFont(font, "Hello, World");
}





Scheme bindings

I have also writting some Scheme bindings, of course, if anyone is
interested. Finally, I have OpenGL fonts in Scheme! And of course,
my tweening functionality
works just fine with fonts.



Roll over, boy!

http://www.youtube.com/v/h_mul5bFRBU

Woohoo!

okonomiyaki

okonomiyaki

 

Sneak Peek of My 3d Scheme Game

I am in the final development stages of my Scheme game that will initially be released for the iPhone. It's been an incredible adventure to leave my job and pursue this full-time. I have learned so much and I really hope I can continue doing this.

This is one of a few posts I will be making in which I will one, begin showing you my game and two, ask for your assistance. I will be brief this time, but soon I will show you the whole game and explain more of my vision for Scheme and game development.

Sneak Peek

First, here is a video of just one aspect of my game, which is fully written in Gambit Scheme. I couldn't resist sharing it. I won't disclose many details of my game yet, but yes, those are cows, and yes, you are exploding them by shooting them with bolts of lightning!

Unfortunately gamedev.net doesn't allow youtube embedding, so please watch it here: ">. Here's a teaser screenshot:



My game should hopefully be released on the iPhone in the next couple weeks. I will post a much more detailed overview of my game soon when I start my big marketing push.

A Request for Help

Speaking of marketing, that's where I need some assistance. I'm a one-man shop right now with little connections in the game industry. It's going to be extremely difficult to make this game successful. I'm asking for your support by either buying my game when it's released (it will be cheap on the iPhone App Store), or helping me market my game in any way possible. You can either email me ideas or simply tell your friends about it (I'm mostly interested in how to gain wide exposure on the app store).

Why should you support me? Well, you may want to support small businesses or indie game companies. Or, you may see that I really strive to give back to the game community. I really want more independant people developing games, and I want to make it easy to do so. I will frequently update my blog with useful information and release as many open-source libraries as possible. (I'm even considering releasing the source to my game when I'm done.)

So far, I have written articles about writing apps for the iPhone in Scheme, ">debugging games interactively, tweening techniques, and more.

I want to expound on these ideas. Although my focus is on Scheme, I hope to offer some cool ideas to the game development community that is good for anyone to think about. But I can only continue doing that if enough people buy my games, and I need help getting the word out!

okonomiyaki

okonomiyaki

 

Tweening and Interpolating OpenGL Objects with Sch

Posted from http://jlongster.com/blog/2010/01/29/tweening/

--



I want to share something I just finished implementing for my 3d
graphics framework in Scheme which I am building my iPhone game
off of. Keep in mind that I am not intentionally building a
separate 3d graphics framework as a general platform, but I am
building only what is needed to get my game out the door, so it is
rough and the API is awkward and not well thought out.

Given my custom scene objects which abstract away OpenGL
rendering, I built a system for interpolating these objects'
properties (such as scale and color) over time, or "tweening." This turned out to be
incredibly useful. Here's the story.

(Note: I'm not extremely confident I'm using the word "tween"
correctly, but if not, I'm re-defining it.)

What's the problem?

The problem was that I needed a system for presenting the title
screen, level titles, and a 2d overlay for my 3d iPhone game.
These 2d elements should smoothly fade in and out of the screen
over time. I don't need a full GUI toolkit, but I'm more
interested in smooth animations when levels change, text appears,
etc.

This turned out to be a huge bear in my old system: because of the
primitive nature of my Scheme graphics libraries, I had to
manually do all the fading every frame for every object. Horrible.

Establishing the Tween

I started re-hauling parts of my system. It started small, only
supporting alpha fading of 2d objects. However, as I worked
on more and more, I realized all properties should be tweenable.

So I set out to implement tweening for any object properties, such
as position, rotation, scale, color, and anything else. And why
not let 3d objects be tweened as well? No reason not to!

It turns out that this is a good way to specify simplistic
behaviors of objects. I'm sure anyone experienced in developing
games is used to this. I've seen various forms of tweening in game
frameworks, especially particle systems. I'm not sure if I've seen
it expressed this way though. Is there a formal or established
name for this, and any examples of how most engines do this?

Once the system was in place, I implemented many types of tweens
such as "linear," "quadratic", "cubic", and "bounce", each
defining a different interpolation function based on time. More
details about these types, including graphics of each function,
can be seen in
this documentation.

Examples

Here are some examples from my tweening system. I will show you
code and a video of what it does. I will not go into details about
the code because it isn't public yet, and it's really rough, but
hopefully it will help expose what I'm trying to achieve.

Types of Tweens

Here I show you all the types of tweens my system currently
supports. A quick list would include linear, quadratic, cubic,
bouncing, and all the variations of applying them at the end or
the start of the interpolation.



(begin
(define (tween-type type)
(scene-list-clear!)
(scene-list-add
(make-tween

(make-2d-object
2d-perspective
position: (make-vec3d .1 (- .5 .025) 0.)
scale: (make-vec3d .05 (/ .05 1.5) 1.)
color: (make-vec4d 1. 1. 1. 1.))

position: (make-vec3d .65 (- .5 .025) 0.)
color: (make-vec4d (+ (* (random-real) .5) .5)
(+ (* (random-real) .5) .5)
(+ (* (random-real) .5) .5)
(+ (* (random-real) .5) .5))
scale: (make-vec3d .2 (/ .2 1.5) 1.)
length: .7
type: type)))

(tween-type 'linear))





">

Queued Tweens

You can queue up tweens as well so that one fires off as another one ends.


(begin
(scene-list-clear!)

(define tx (image-opengl-load "survived.png"))

(let ((obj (make-2d-object
2d-ratio-perspective
position: (make-vec3d (- .5 .025) .1 0.)
scale: (make-vec3d .05 .05 1.)
color: (make-vec4d 1. 1. 1. 1.)
rotation: (make-vec4d 0. 0. 1. 0.)
texture: tx
)))
(scene-list-add
(make-tween
obj
position: (make-vec3d (- .5 .025) .75 0.)
scale: (make-vec3d .4 .4 1.)
color: (make-vec4d 0. 1. 0. .5)
rotation: (make-vec4d 0. 0. 1. 90.)

type: 'ease-out-bounce
length: 1.5

on-finished:
(lambda ()
(scene-list-add
(make-tween
obj
type: 'ease-inout-cubic
position: (make-vec3d .8 .1 0.)
scale: (make-vec3d .1 .1 1.)
color: (make-vec4d 1. .5 0. 1.)
length: 1.))
#f)))))





">

3d Tween

There's no reason you can't apply tweens to a 3d object as well.



(begin
(scene-list-clear!)

(let ((obj (make-mesh-object
3d-perspective
mesh: sheep-mesh
position: (make-vec3d 0. 0. 10.)
rotation: (make-vec4d 0. 0. 1. 0.)
scale: (make-vec3d 2. 2. 2.))))
(scene-list-add
(make-tween
obj
scale: (make-vec3d 6. 6. 6.)
rotation: (make-vec4d 0. 0. 1. 90.)
length: 1.
type: 'ease-out-cubic
on-finished:
(lambda ()
(thread-sleep! 1.)
(scene-list-add
(make-tween
obj
scale: (make-vec3d .7 .7 .7)
type: 'ease-inout-cubic
on-finished:
(lambda ()
(scene-list-add
(make-tween
obj
position: (make-vec3d 0. 5. 10.)
type: 'ease-out-bounce
on-finished:
(lambda ()
(scene-list-add
(make-tween
obj
type: 'ease-out-bounce
position: (make-vec3d 0. -5. 10.)))
#f)))
#f)))
#f)))))





">

Improvements

I'm still wrapping my head around how to implement a full motion
system with this. As of now, if you specify a cubic tween between
point (x, y) to (x1, y1), it will perform a cubic interpolation on
both the x and y as a function of time. The result is that the
object moves in a straight line from point A to point B, but the
speed of the movement is cubic.

For example, here's a cubic curve:



Even though it is a curve, if I apply cubic interpolation to
positions, which are 3-dimensional vectors with an x, y, and z
property, the object moves between point A and point B in a
straight line, with cubic-varying speed. If we tracked the
movement, it would look like this:



What if I wanted the object to curve when moving from point A to
point B? Surely I can express that with interpolations? The
problem lies in the fact that I've bound my interpolations of
multi-dimensional vectors to be constant across each dimension. I
would need to support different kinds of interpolations across
each x, y, and z axis. This sounds really neat, and I will have to
extend my system to support this later.

Now that I think about it, I think the special case of
interpolating movement should be handled with
[splines](http://en.wikipedia.org/wiki/Spline_\(mathematics\)). In fact,
it's probably downright inappropriate for me to interpolate
position this way. Splines are on my todo list for research.

Conclusion

Having this simplistic tweening system will make it easy for me to
apply smooth animations consistently across my iPhone game. It's
feels a lot more polished when I use this to fade in items and
slide them off the screen. I wouldn't really have thought of this
unless I forced myself to build a real game.

Building a game from scratch has forced me to learn
all kinds of things, and I find this experience invaluable. I'm
not building something simply because the idea of it sounds cool,
but I *have* to build my framework with certain features so that
my game will not only *run* but look good too. Having this dictate
my development forces me to drastically alter my ideals quickly as
I learn how games are laid out.

Scheme is a huge support in this kind of simplistic agile
development. Between its dynamic type system and functional
simplicity, I can constantly re-shape huge parts of the system as
if I'm a sculpter removing and adding huge chunks of clay. On top
of that, The Lisp methodology of interactive/iterative development
allows me to add details extremely quickly. The turn around time
for feeling reward from intense coding sessions is *really* fast
and extremely motivating!

okonomiyaki

okonomiyaki

 

Request for .max - .obj conversion

This is definitely not an interesting post, but rather a request. Does anyone have 3ds Max that would be willing to convert a model from the .max format into something more standard? .obj would be the best, but .3ds is fine too.

I have found a few good free 3d models online for my game, but they all come in .max format unfortunately. It's unfortunate that this is a proprietary format for something far beyond my budget. Here's the one I found:

http://www.3dxtras.com/3dxtras-free-3d-models-details.asp?prodid=6912

I hope this kind of request isn't in bad taste. I am certainly not a modeler and that one looks great.

Just PM me and I'll send you the model I downloaded if you're willing to help. Thanks!

EDIT: Nevermind, I was able to get the trial version of 3ds Max and use it once.

okonomiyaki

okonomiyaki

 

On Spit and Polish, and Emacs Remote REPL

Posted from http://jlongster.com/blog/2009/09/9/spit-and-polish-emacs-repl/:

------------------------------

On Spit and Polish, and Emacs Remote REPL

September 09, 2009

Here I am, several weeks into a serious project, and what's happening? I'm getting distracted. Why is it so hard to finish projects? I think I know the answer to that question, but I don't like it.

**I hate spit-and-polish work.**

I love *low-level* work, even to the point of dealing with memory management and optimizing assembly code. It's the spit and polish, the sanding of the surface, everything that makes a product *look* and *feel* right that gets me. And I know why: not only is it menial work, I never anticipated it in the first place!

Jeff Atwood recently discussed this difficulty using Stack Overflow as an example. It's easy to oversimplify software; actually, it's *really* easy to *super extremely* simplify software.

I started coding games in Scheme over a month ago. I spend about 3 days a week on it. Now, that doesn't seem like much, but I fiercely believed that there *had* to be simple games I could make. My initial target would be the iPhone. Unfortunately, I'm not very interested in 2d, but there has to be at least one simple 3d game I can make, right?

I'm well on my way to finishing that game. However, I just want to make it clear how badly I underestimated the amount of work it would take. I started out strong; I ported Gambit Scheme to the iPhone, and whipped up some helpful tools for it.

But suddenly, weeks would go by seemingly without much progress. I thought about where my time was going, and realized all that's necessary for a complete game: I have to market myself, find/contract all of my art and models, write different levels for the game, make a menu screen, keep track of the score, write game instructions, make it look pretty, etc.

Anyway, I am not frustrated. I have simply re-learned the same mantra I've learned so many other times: software is not simple.

This all hit me when I stayed up all night trying to figure out how to add fur to one of my sheep models in Blender. I spent 5 hours, and all I came up with is this!



Suddenly I realized that I am *not* a modeler and I can't afford to spend time doing this. If I'm not careful with how I'm spending my time, I'll end up 6 months down the road with nothing to show but a creepy sheep wearing a bear coat.

So, how about some real content, you say? Here are a few things I've done recently.

iPhone game
Despite the tone of the opening paragraphs, I have successfully built most of my first iPhone app. It is completely written in Scheme, of course. I'm not going to disclose everything yet, but here's an early screenshot (without spit and polish, of course...):



Remote REPL in Emacs
I wrote a previous post about using a remote REPL for debugging/interacting with your apps. Many people have asked me if I have integrated it with Emacs yet, as that's the next logical step. Indeed I have.

This is just a teaser, though. I'm not releasing my Emacs package because Marc Feeley (creator of Gambit Scheme) and I have talked and he wants me to finish his work on SLIME integration instead (also, it's rather buggy). I agree with him. Using Gambit Scheme under SLIME will give much more power, and we've already done a lot of the work. Please stay tuned for the release of it.

Until then, here's the kind of stuff you can do with it:

">(click to view video)

Montreal
Speaking of Marc Feeley, I made a trip up to Montreal a couple weeks ago and visited the guys working on Gambit Scheme. I also went to the Scheme conference in Boston and met many other guys working on Scheme. It was a very inspirational trip. There are some *really* smart guys working on Scheme, and I hope to work closely with them over the next few years.

Marc even invited me to present at MSLUG, which was a real honor! He videotaped the meeting and if I can get my hands on it maybe I'll post it here. I presented my recent work of using Gambit on the iPhone and discussed developing native apps in Gambit Scheme using Gambit's portable C compilation and the remote REPL. More importantly, we all went out for beer afterward.

Many thanks also to Christian Jaeger who let me stay at his place.

Snow Leopard and OpenCL
I've been keeping an eye on OpenCL and I now have a chance to play with it! I just upgraded to Snow Leopard, so I have a system fully capable of running OpenCL. I'm really look forward to playing around with it. If you haven't already seen it, you should read Arstechnica's great review of the new OS, specifically the sections about Grand Central Dispatch and OpenCL.

I'm thinking of testing matrix multiplications with OpenCL. It may be more optimal to manage a Scene Graph by pushing off all of its hierarchical matrix transformations off to the GPU. All of that depends on how much overhead there is transferring the data, of course. I can't wait to blog about my results!

Until then, I'm going to sit down and enjoy a nice some good coffee.

okonomiyaki

okonomiyaki

 

Physics and 3d models in Scheme

Posted from http://jlongster.com/blog/2009/08/3/iphone-physics-and-models/:

-------------------------

Physics and 3d models in Scheme

August 3, 2009

I finished two major parts of my Scheme 3d graphics framework this past week, and got it running on the iPhone: basic physics simulation and importing 3d models.

* Disclaimer

Apple's been getting some really terrible press about the iPhone App Store recently, from Google Voice being taken down to general crapness in their treatment of developers. I'm still confident that I'll be able to get my Scheme apps through the App Store, but please note that most of my work here is applicable for any Scheme game, whether it's on Windows, OS X, or whatever. I look forward to working on full PC games when I have enough resources.

* Physics

">(Click here if you don't see the video above)

In this video, you can see my current physics environment. I implemented velocity and acceleration, which lets me simulate gravity by applying a global acceleration downward of about 9.8 m/s. I proceed to toss balls up in the air randomly, and gravity takes care of the rest.

I then use the procedure `kick` to kick around balls. It simply pushes all the objects in the scene by adding a velocity vector to each object, giving them instant acceleration in the specified direction.

Lastly, I change gravity so the balls gravitate to other areas in the world. Pretty fun stuff.

I'm able to develop interactively using the technique described in my previous post about using remote REPLs.

* Models



I wrote a OBJ file parser so that I can finally import 3d models. It wasn't really that hard, and I think I will just post it here. It's a work in progress, of course, and will be tracked in my git project in the `lib` directory.

Honestly, I didn't do a whole of research before I wrote this, but first, it didn't seem like there was a simple OBJ loader written in C out there, and second, it's nice to have this native in Scheme anyway. It sure looks a lot prettier than this loader written in Objective-C.

I also found this blog post, which suggests using a Blender script to output... C header files with embedded data? That's crazy! I can't imagine having to re-compile my program every time I changed a model.



(define (read-map #!optional f)
(unfold (lambda (x) eof-object?)
(lambda (x) (if f (f x) x))
(lambda (x) (read))
(read)))

(define (enforce-length name len lst)
(if (eq? (length lst) len)
lst
(error name "assert-length failed")))

(define (obj-parse-vertex)
(enforce-length "vertex" 3 (read-map exact->inexact)))

(define (obj-parse-normal)
(let ((v (vec3d-unit
(apply make-vec3d
(enforce-length "normal" 3
(read-map exact->inexact))))))
(list (vec3d-x v) (vec3d-y v) (vec3d-z v))))

(define (obj-parse-face)
(enforce-length "face" 3 (read-map (lambda (n) (- n 1)))))

(define (obj-parse-line obj line)
(define (appendd lst lst2)
(append (reverse lst) lst2))

(with-input-from-string line
(lambda ()
(let ((type (read)))
(case type
((v)
(obj-vertices-set!
obj
(appendd (obj-parse-vertex)
(obj-vertices obj))))
((vn)
(obj-normals-set!
obj
(appendd (obj-parse-normal)
(obj-normals obj))))
((f)
(obj-indices-set!
obj
(appendd (obj-parse-face)
(obj-indices obj)))))))))

(define-type obj
id: 9F8E77AF-49A7-4974-A071-C91616CD1DD0
constructor: really-make-obj
num-vertices
num-indices
vertices
normals
indices)

(define (make-obj)
(really-make-obj #f #f '() '() '()))

(define (obj-load file #!optional avoid-c-vectors?)
(define (convert data)
(list->vector (reverse data)))

(define (make-c-vectors mesh)
(if (not avoid-c-vectors?)
(begin
(obj-vertices-set! mesh
(vector->GLfloat* (obj-vertices mesh)))
(obj-normals-set! mesh
(vector->GLfloat* (obj-normals mesh)))
(obj-indices-set! mesh
(vector->GLushort* (obj-indices mesh))))))

(with-input-from-file file
(lambda ()
(let ((mesh (make-obj)))
(let loop ()
(let ((line (read-line)))
(if (not (eof-object? line))
(begin
(obj-parse-line mesh line)
(loop)))))

(obj-num-vertices-set! mesh (length (obj-vertices mesh)))
(obj-num-indices-set! mesh (length (obj-indices mesh)))
(obj-vertices-set! mesh (convert (obj-vertices mesh)))
(obj-normals-set! mesh (convert (obj-normals mesh)))
(obj-indices-set! mesh (convert (obj-indices mesh)))
(make-c-vectors mesh)
mesh))))









Usage:


(obj-load "mesh.obj")


There are a few caveats to my loader: it doesn't index normals (assumes every vertex has a normal), requires triangulation (faces can only have 3 vertices), and doesn't support texture mapping yet. I will add texture mapping soon, but as to the other caveats, I'm not sure if I want to complicate my loader when I can simply tell Blender to export things properly. Maybe someday though.

** Optimizing

Now, it's a bit slow to load a model with about 3000 vertices and 2000 indices. Why not use Gambit's nice serialization features to speed this up?

Folks, *this* is why I use Scheme, and Gambit Scheme specifically. Gambit Scheme provides two interesting procedures: `object->u8vector` and `u8vector->object`. These procedures convert objects to byte vectors and back. Byte vectors can easily be written to files, too!

So, by adding the following two procedures to our OBJ loader, we immediately support a very efficient binary model format. `compress` basically takes the constructed mesh, converts it to a `u8vector` and saves it out to disk. `decompress` does the opposite.



(define (compress filename mesh)
(with-output-to-file filename
(lambda ()
(let* ((v (object->u8vector mesh))
(len (u8vector-length v))
(len-u8 (object->u8vector len))
(boot (u8vector-length len-u8)))
(write-u8 boot)
(write-subu8vector len-u8 0 boot)
(write-subu8vector v 0 (u8vector-length v))))))

(define (decompress filename)
(with-input-from-file filename
(lambda ()
(let* ((boot (read-u8))
(len-u8 (make-u8vector boot)))
(read-subu8vector len-u8 0 boot)
(let* ((len (u8vector->object len-u8))
(v (make-u8vector len)))
(read-subu8vector v 0 len)
(u8vector->object v))))))



I created a quick script to compress meshes:



;;; saves it with the ".gso" extension

(define (main filename)
(compress (string-append filename ".gso") (obj-load filename #f #t)))


I modified `obj-load` to take an extra parameter indicating if the file is compressed or not. So, lets see what we gained with this no-effort improvement:



james% du -h logo.obj*
1020K logo.obj
800K logo.obj.gso

(obj-load "logo.obj") => 1.05s
(obj-load "logo.obj.gso" #t) => .215s



We got a 32% compression and 5x load time speed increase!

okonomiyaki

okonomiyaki

 

New look and new future for jlongster

Posted from http://jlongster.com/blog/2009/07/31/new-look-jlongster/:

***



New look and new future for jlongster

July 31, 2009

I spent some time redesigning this site this past week. The initial design was only temporary until I figured out how I was going to lay everything out. After putting up some content for a couple weeks, I think I have a feel for where I want to take this site.

I fleshed out all of the projects in the software section so they now have individual pages. Of particular interest is the project page for the Scheme iPhone Example which will serve as a hub for all of my work on Scheme iPhone development. Please let people know about it!

As you can see, this blog received a few improvements as well, mainly a better way of viewing previous posts.

The future of this blog will be to track my progress as an independent game developer trying to make it in the vicious world of gaming. I am inexperienced in the gaming business, so this will be interesting. I feel that I have something new to bring to the table, and that is Scheme. I will be developing all of my games in Scheme.

Currently, I am spending 2-3 days a week developing iPhone games in Scheme. I am not entirely sure my games won't be rejected by Apple (can anyone be sure?), but it's the only market I have a chance at success in. If I can build a few successful iPhone games, I will have the resources for spending more time on bigger products. I hope to build a competitive 3d engine in Scheme someday.

As I said before, although I have been working with 3d graphics for a while, I am new to this business. Please let me know if you have any tips or want to help out in any way!

P.S.

I'm glad to see Haskell on the iPhone, but it's too bad that it takes 19 patches to GHC! Luckily, it's a good bit simpler to get Gambit Scheme running on the iPhone.

okonomiyaki

okonomiyaki

 

Barcamp Presentation on Scheme on the iPhone

Posted from http://jlongster.com/blog/2009/07/27/barcamp-presentation-scheme-and-using-it-iphone/.

---------------------------------

Barcamp Presentation on Scheme on the iPhone

July 27, 2009

I presented at the Chattanooga Barcamp this past weekend. In the former half, I introduced Scheme and attempted to explain why it's a good language for development. The latter half presents some cool practical examples, specifically showing what I've been working on with using Scheme on the iPhone.

It was warmly received; thanks to all who came! I'm glad you enjoyed it.

(looks like I can't embed it here)

Writing iPhone apps in Scheme.

okonomiyaki

okonomiyaki

 

Touch and accelerometer functionality available fo

Posted from http://jlongster.com/blog/:

--------------------------------------------

Touch and accelerometer functionality available for Scheme on the iPhone

July 20, 2009



I have finished the Gambit Scheme libraries for touch and accelerometer support. The green smudge was produced by my fingers, and the orange smudge was produced by tilting the phone.

I've introduced a basic events system. To capture a an event, you define an event handler using DEFINE-EVENT-HANDLER. Currently, the only events implemented are "touches-began", "touches-moved", "touches-ended", "touched-cancelled", and "did-accelerate". The arguments of the event are the same as they would be in Objective-C.

Here's an example, taken from the application shown above:

(define-event-handler (touches-began touches event)
(for-each (lambda (touch)
(let ((loc (UITouch-location touch)))
(add-point (car loc) (cdr loc))))
touches))

(define-event-handler (touches-moved touches event)
(for-each (lambda (touch)
(let ((loc (UITouch-location touch)))
(add-point (car loc) (cdr loc))))
touches))

(define-event-handler (did-accelerate device accel)
(define (shift n weight)
(+ n (* weight 70)))

(add-point (shift 150 (UIAcceleration-x accel))
(shift 100 (- (UIAcceleration-y accel)))))

** Download **

I'm still polishing up a few things in these FFIs, but they are ready to use. Soon, I will create a separate github project for Gambit Scheme FFIs. For now, they are included in my gambit scheme on the iphone example project.

Specifically:

lib/ffi/osx.scm
lib/ffi/osx#.scm
lib/ffi/iphone.scm
lib/ffi/iphone#.scm
lib/events.scm

Don't forget to setup a few hooks in your Objective-C app, as seen here (accelerometer) and here (touch).

You can view this iphone application included in my github project as well for example usage.

okonomiyaki

okonomiyaki

 

Remotely debugging your iPhone Scheme apps with a

Posted from http://jlongster.com/blog/2009/07/5/remotely-debugging-iphone-scheme/.

------------------------------------

Remotely debugging your iPhone Scheme apps with a REPL

July 5, 2009

View the remote debugger source

In an earlier blog post, I showed you how to develop on the iPhone
with Gambit Scheme. Although that is cool, what about debugging?
Haven't we just cut ourselves off from using either GDB (we aren't
using Objective-C) or Gambit's debugger (we aren't running the program
from a console)?

A debugger is an essential tool, especially in dynamic languages like
Scheme. It may not be as important in languages like Haskell because
of type-checking, but even then you want to watch "data flow" as the
program is running. I think we can say that using Scheme on the iPhone
is impractical until we can use Gambit's debugger for our iPhone apps.

Fortunately, Marc Feeley, implementer of Gambit Scheme, recently wrote
a "proof of concept" remote debugger. It allows you to push REPLs from
a running Gambit application (the client) to a different machine (the
server).

You guessed it. The client will be our iPhone application and the
server will be our main computer, allowing us access to REPLs on our
computer which are running inside the iPhone application.

I think this is best explained in the screencast below. First, let me
explain the philosophy of the remote debugger a little more.

Marc wrote the remote debugger in response to someone asking about
debugging multiple Gambit Scheme threads at once on the Gambit mailing
list (view original message). With the remote debugger, you can tell
Gambit to create a remote REPL for errors on each thread. So if two
threads errored, two separate debugging REPLs would open up on the
server.

Now we have facilities for debugging Scheme iPhone apps
remotely. We can run Scheme apps on either the iPhone Simulator or
the iPhone itself, and when an errors occurs we get a real Gambit REPL
on our computer for debugging the problem.

What's even cooler is that we have everything we need for running
a full Scheme REPL to *interact with our iPhone app real-time*! Check
it out in the screencast below.

Please note that this library is still "proof of concept," and there are
more things to figure out and bugs to fix. I am not publishing it standalone
yet, but you can find it included in the gambit iPhone example program on github or in
Marc's reply.

You can also see how I integrated the debugger library in the example
iPhone app here. A couple lines further down and you can see how I
fire up a main REPL.

(I apologize for all the static. It was raining and I didn't realize
it would pick it up.)

">

Next up is support for touch events and access to the accelerometer!

View the remote debugger source

okonomiyaki

okonomiyaki

 

Writing apps for the iPhone in Scheme

Posted from http://jlongster.com/blog/2009/06/17/write-apps-iphone-scheme/

----------------------------------------------------------

Writing apps for the iPhone in Scheme

June 17, 2009

View the source for my example application.

Apple seems to have a tight leash on how you can develop for the iPhone. The User Agreement essentially states that you can only program in Objective-C. Now, let's see... Objective-C is a superset of C, correct? So C code is valid, right? If we only had something which could generate C for us... it would take code in the language of our choice and compile it to C without violating the User Agreement.

Luckily, my language of choice is Scheme, and it is well-versed in this kind of song and dance. Several Scheme implementations compile Scheme code to C code for portability and speed. This approach has many wonderful benefits, such as allowing us to write full applications in Scheme and compiling them into valid apps ready for the iPhone app store.



My Scheme implementation of choice is Gambit Scheme, which uses a Scheme->C compiler. I wrote the above app in Gambit Scheme (view it in graphics.scm), and I will show you how to get your Scheme apps running on the iPhone. We need to compile the Gambit run-time for the iPhone first, then we can start writing our apps.

Note that, in the end, we are simply compiling a bunch of C code together to form a resulting binary. It just so happens that there is quite a bit of C code in there which implements a few things like continuations, lists, and other helpful stuff.

Compiling Gambit for the iPhone

Gambit is insanely easy to compile for the iPhone. This was very shocking to me; I was expecting at least some portability problems, and I haven't met any yet. It does not need to be ported since it already supports the ARM architecture of the iPhone.

So, where do we start? There are two target platforms we need to set environments up for: the iPhone simulator and the iPhone OS. The main difference between the two is that the former uses the x86 architecture on your computer and the latter uses the ARM architecture on the iPhone.

iPhone simulator

First, download Gambit. I'm using v4.4.4.

Now we need to start poking around Xcode to find out what happens when you build something for the iPhone simulator. The most useful thing in Xcode is the "Build Results" window found under the "Build" menu. In this window, there are a few small icons in the bottom left. One of these icons looks like a scribbled document. If you click this icon, a window will pop up showing you exactly what commands Xcode is issuing to the system. Using this, we can "reverse engineer" the iPhone build process.

The first thing you'll notice is that it's using a different instance of "gcc". In fact, everything seems to be coming from an environment named iPhoneSimulator.platform in "/Developer". This is achieved by calling the separate "gcc" instance with the "-isysroot" flag.

So it appears we need to setup Gambit to call this instance of "gcc" with this special environment. "configure" respects $CC as the C compiler and $CFLAGS as any additional flags. I discovered that "configure" sometimes calls the C compiler without the additional flags, however, so lets just set the C compiler to the following. The specific paths were copied from the Xcode "Build Results" window.


$ export CC='/Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc-4.0
-isysroot /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator2.2.sdk'


Note that we are using the "iPhoneSimulator2.2.sdk." Change this to the appropriate SDK you want to use; since version 3.0 of the iPhone software just came out, "iPhoneSimulator3.0.sdk" should even be available. Now we can actually compile Gambit and install it somewhere we can recall later:


$ ./configure --prefix=/usr/local/Gambit-C/iPhoneSimulator
$ make
$ sudo make install


I'll make a quick note that Gambit highly encourages using the "--enable-single-host" configure option, as the code will be much faster at the expense of compile time.

Now we have a Gambit which uses the environment for the iPhone simulator instead of your default system. The two important things are the "libgambc.a" in the "lib" directory, which is what we will link to when building the app for the simulator, and "gsc" in the "bin" directory which we will use to generate C files.

iPhone OS

This process is very similar to the iPhone Simulator build. The interesting difference is that we are actually cross-compiling Gambit for the ARM architecture. Important note: I had some troubles with re-compiling the same Gambit source even after a "make clean", so if you run into any problems, just download a fresh copy of Gambit. I'll be submitting a bug report about this.

First, configure with the correct CC environment variable. I discovered this cross-compiler much the same way I discovered the iPhone Simulator compiler (switch your Xcode project to target the iPhone OS, click build, and watch the results). The only difference was that I had to manually expand "gcc-4.0" to "arm-apple-darwin9-gcc-4.0.1" which was sitting right beside it.


$ export CC='/Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/arm-apple-darwin9-gcc-4.0.1
-isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS2.2.sdk'


Again, note that we used "iPhoneOS2.2.sdk". You should change this to whatever you want; version 3.0 of the iPhone software would use "iPhoneOS3.0.sdk." Next, configure and make it. Note the extra option "--host" which tells it to cross-compile to ARM.


$ ./configure --host=arm --prefix=/usr/local/Gambit-C/iPhoneOS
$ make
$ sudo make install


See the previous section about other recommended options to "configure".

Now we have the Gambit library compiled for the ARM architecture.

Using Gambit in iPhone apps

So, after all that work, how do we use it? We basically need to embed Gambit in an iPhone app now and link with the appropriate library we just built.

I'll show you with an example. My example application is a 2d OpenGL app. Let's use the "OpenGL ES Application" project template as a starting point (seen in Xcode when creating a new project). By default, this project renders a rotating colored square.

First, you have to modify your "main" procedure to embed Gambit, which is the most complicated part. Usually, Gambit provides a "main" entry point method for your compiled executable which does all this itself. However, since we want our own "main" method, we need to setup Gambit's runtime.

Go ahead and view my application's main.m if you want to go ahead and see what it looks like. One important note is that we set the debug settings to use stdout for REPL output, which is where error messages are sent (we don't really have access to a REPL yet). This means that Gambit errors will be spit out to the console file, just like normal output from an iPhone app. There's no room here for me to fully explain Gambit, so I have to leave it at that.
Second, we need to write any FFIs (foreign function interface) required for accessing iPhone functionality. For my application, I need access to OpenGL ES, so I wrote a few functions in Scheme which call the appropriate C functions. Here's how you would write you "glTranslatef" function:


(define glTranslatef (c-lambda (GLfloat GLfloat GLfloat) void "glTranslatef"))


"c-lambda" is the important form here. This function is simple to interface. Other ones, such as "glVertexPointer," are more difficult since they use C arrays. You can view my complete FFI in gl.scm. It's a bit large because of requiring access to C arrays, and you can view the most important parts here. As a side note, a few folks are working on automatically generating FFIs for Gambit.

Third, you write your application code in Scheme. For example, to draw a box with my OpenGL ES FFI, I would use this code:


(define vertices
(vector->GLfloat* (vector -.05 -.05
.05 -.05
-.05 .05
.05 .05)))

(define (render)
(glLoadIdentity)

(glVertexPointer 2 GL_FLOAT 0 vertices)
(glEnableClientState GL_VERTEX_ARRAY)

(glColor4f .2 .5 .5 1.)
(glTranslatef 0. .5 .0)
(glRotatef .1 0. 0. 1.)
(glDrawArrays GL_TRIANGLE_STRIP 0 4))


To view my full application (as shown in the screenshot), checkout graphics.scm.

Fourth, generate C procedures which you call from the Cocoa world at the right times, serving as "hooks." For example, during "drawView" in the OpenGL view, I call "render" which looks like a C function, but actually calls into the Scheme world (see that in EAGLView.m).

Here's how I turned the Scheme function "render" into a C function:


(c-define (c-render) () void "render" ""
(render))


"c-define" is the important form here. It does the opposite of "c-lambda": it wraps a Scheme function in a C function. You give it the function signature and you can be sure that it'll be available for other C code.

You can view this hook in init.scm.

"get-title" is another Scheme function available to C which is used for the value of the label. In Scheme, it just returns "Hello, World. - Gambit Scheme" (in graphics.scm).

Fifth, compile your Scheme code to C. The way I did this was to have one Scheme file which includes all of my modules: init.scm. Then, I compile it to C using Gambit's compiler. Gambit will spit out 2 C files: init.c and init_.c.


$ /usr/local/Gambit-C/iPhoneSimulator/bin/gsc -link init.scm
$ ls init*
init_.c init.c init.scm


"gsc" also supports the "-debug" flag which makes it produce good error messages. During development, you should always use it.

We used the "gsc" which came with our iPhoneSimulator build. Technically, when simply generating C from Scheme, I don't think it would matter which "gsc" we used, which is why it's fine to use the above code when targeting the iPhone OS. Choosing the right "gsc" becomes important when you use it to automatically compile your code into dynamic libraries, which is a topic for another post.

Sixth, configure the build environment and compile the application. In Xcode, add those 2 generated C files to your project. You need to add the search paths: "/usr/local/Gambit-C/iPhoneOS/include" for header files, and "/usr/local/Gambit-C/iPhoneOS/lib" for linking. Now we need to link Gambit: simply add the "-lgambc" flag to the linker. The one last thing you need to do is add an additional C flag: "-D___LIBRARY", which tells Gambit not to generate a "main" function. My project comes with an example Xcode project.

In addition, you may need to add another C flag: "-x objective-c", which forces everything to be compiled as Objective-C. This is useful if you are writing FFIs for Cocoa Touch API's.

Now you should be able to compile and run it! Note that we hardcoded "iPhoneOS" in the paths. If you built Gambit the way I described earlier, you can change that to "iPhoneSimulator" to build the program for the simulator. I'm not an Xcode expert, but I'm sure you can setup multiple built environments so that each device has the appropriate flags.

Here is my program running on my iPhone:



What next?
Although we got it working, it still feels a bit crude. Debugging is a little awkward, there seems to be issues with standard output, etc. There are certainly things that need to be smoothed out, but there should be no problem in doing so. It should be somewhat easy to implement a remote REPL when an error occurs, meaning that I would be using REPL on my mac which is debugging problems real-time on the iPhone OS across the network.

I haven't tested everything, such as initiating TCP connections, opening files, etc., but there's no reason why it shouldn't work. I also haven't run any serious benchmarks yet, and I can't say anything yet because I haven't optimized anything. But Gambit is good at optimizing, and it should be fast enough.

One last note is that to fully comply with the User Agreement, you might have to compile Gambit without the interpreter, meaning "eval" wouldn't be available. I'm not sure if it's simply that you can't use it, or if it's that the actual existence of something which could interpret code violates it.

It's exciting to be able to develop iPhone apps in Scheme! For example, now I can do something really cool: when developing on the simulator, use Gambit's "load" procedure. "load" takes a file and loads it at runtime. If it's compiled, it will load it as a dynamic library. If it's not compiled, it will interpret the code. Using "load", I only have to install the application once, and then I can simply edit and/or compile files being loaded at runtime.

There are many more things to do, such as remote REPL debugging, more FFIs, etc., so expect more blog posts about this. I think we've done something very cool here, and it's all thanks to Gambit.

View the source for my example application.

okonomiyaki

okonomiyaki

 

Emacs font locking: highlighting comment lines

Posted from http://jlongster.com/blog/2009/06/9/font-locking-highlighting-code-section-headers/.

----------------------------------------

Emacs font locking: highlighting comment lines

June 09, 2009

I recently tried an package called Quack which adds a bunch of commands for writing and running Scheme code in Emacs. Although I don't use it anymore, one thing I really like was how it highlighted comment lines with a depth of 3 or more (meaning ";;;" in Scheme, or "///" in C or javascript). It turns out this was really nice, since it ends up highlighting file-level comments at the top and section headers throughout the code.



I think this looks much nicer than other techniques, such as repeating the comment character 40 times to indicate a line (e.g. ";;;;;;;;;;;;" over and over). If everyone on a project wrote code this simple way, each person would get to choose how to view those kinds of comments.

I wanted to separate this functionality into its own package so that I could use it in languages other than Scheme. I have never written any Emacs Lisp before, so although it was pretty brutal, I ended up with something that works rather nicely. Use it if you want.



(defconst jwl-title-face 'jwl-title-face)
(defface jwl-title-face
'((((class color) (background light))
(:background "DarkSeaGreen1" :foreground "grey25"))
(((class color) (background dark))
(:background "DarkGrey")))
"Face used for titles.")

(defun jwl-generate-highlight-keywords (comment-char)
`((,(concat "^\\([ \t]*"
comment-char
comment-char
comment-char
"[ \t]*"
"\\("
"[^\r\n]*"
"\\)"
"\r?\n?\\)")
1 jwl-title-face prepend)))

(add-hook
'scheme-mode-hook
(lambda ()
(font-lock-add-keywords
nil
(jwl-generate-highlight-keywords ";"))))

(add-hook
'python-mode-hook
(lambda ()
(font-lock-add-keywords
nil
(jwl-generate-highlight-keywords "#"))))

(add-hook
'js2-mode-hook
(lambda ()
(font-lock-add-keywords
nil
(jwl-generate-highlight-keywords "/"))))




okonomiyaki

okonomiyaki

 

new site: jlongster.com

I just launched http://jlongster.com/. I will be actively posting projects and blog entries related to 2d/3d graphics development, and hopefully game programming. I'm a Scheme programmer, so a lot of it will concerning Scheme. You might be interested in stuff there though.

Instead of canceling GDNet+, I'm going to try copying my blog posts here. I think this is a good community and I hope some of you keep track of my work. I will soon be working part-time on high-performance graphics work in Scheme.

To kick it off, here's one of my first blog entries:

-----------------------------------------------------

http://jlongster.com/blog/2009/05/25/mona-lisa-genetically-drawn-scheme/

Mona Lisa genetically drawn from Scheme

May 25, 2009

Remember the inspiring project which drew Mona Lisa using a genetic algorithm (or, according to some people, stochastic hill climbing)? I decided to implement this technique in Scheme. I've always been fascinated by genetic algorithms and this is a good project to learn them.

I call the project "Genetic Canvas" (github). It is written in Gambit Scheme on OS X and uses OpenGL.

I explored various methods of helping the genetic algorithm, from edge detection to color analysis. In the end, most of this work ended up hurting its ability to solve the problem. The basic process uses a population size of 10, randomly creates and mutates polygons on the screen, and analyzes each solution by summing the color difference of each pixel. We then use elitist selection to pick the best one, and rinse and repeat.

There were points in the project which I felt hatred towards genetic algorithms. I underestimated the flaky and irrational behavior of the whole process, which is something close to a temperamental child. All of your mutation rates, population sizes, mutation and fitness procedures, etc. have to be exactly right, and there's no way to provably solve which settings are correct. You have to eye the solution yourself and tweak the settings as you see fit.

In the end, I still find genetic algorithms interesting. This application of them is simply not very interesting, as it has no practical use, and the problem set is rather difficult to solve generally (large and small images, various compositions, etc.).

I wanted it to find an acceptable solution quickly. The following images were rendered in about 30 minutes.




With borders...



And finally, taking about 3 hours...


okonomiyaki

okonomiyaki

 

A Sad Day

Hey guys,

I'll be canceling my GDNet+ account soon. I'm moving my blog to smallnum.blogspot.com. I'm in the process of porting over my recent posts there.

It's been fun to be around here, and I hope I can find a way to export all of my posts (there's some really old ones that are great reads if I need to laugh). If anyone knows anything about export posts, let me know... I'm pretty pessimistic about it.

Anyway, even though I'm getting into game development, I'll probably be focusing more on Scheme and other things which most people around here aren't really interested in. And I simply like the blogger software better :)

Cheers
- James

okonomiyaki

okonomiyaki

 

SILex & lexing

I've been playing around with SILex recently. It's a very sweet lexical analyzer, essentially a port of Lex to Scheme. Now, I'm no experienced lexer (never even used Lex), so I'm not going to compare Lex and SILex. But I do want to write about something cool SILex let me do.

I need to write an FFI (foreign function interface) for each C library I want to use from Scheme. Possibly the most important one is OpenGL. To give you a taste of the FFI I'm working with in Gambit-C Scheme, here's an example:


(define glRotatef
(c-lambda (GLfloat GLfloat GLfloat GLfloat) void "glRotatef"))


This lets me call `glRotatef' from Scheme. Now, if you think writing this for every single C function would be monotonous, repetitive, and error-prone... you're right! There's no reason why we can't automatically generate this. Well, there are some difficulties with FFI's in general, but we can get close.

The most straight-forward way is to parse the OpenGL header, gl.h, and generate something of worth from it. Technically, we will first "lexically analyze" the text in the file, and then "parse" the tokens into something we can use.

Lexical Analysis

Lets use SILex for some lexing. There's a postscript manual for SILex in the download if you want to know more about it. From what I can tell, however, it's remarkably like Lex in the way it works, so you'll recognize this if you're familiar with Lex. Lex's manual describes the basic process.

You give SILex a specification file which contains a mapping of regular expressions to tokens. It then uses this to generate a "lexer" program, or something that takes a program as input and produces a set of tokens.

In order to focus more on lexing/parsing, lets simply try to parse all of the preprocessor constants defined in the OpenGL header gl.h. You should be able to parse all of the function definitions too, but that involves a few tricky nuances that would just be distracting.

I knew that Chicken Scheme already has a general FFI generator, and I noticed that it's using SILex as well. So I grabbed it's SILex specification file which tokenizes C and parts of C++/Objective-C and started there. Here it is:


; easyffi.l -*- Scheme -*-
;
; Copyright (c) 2000-2004, Felix L. Winkelmann
; All rights reserved.
;
; Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following
; conditions are met:
;
; Redistributions of source code must retain the above copyright notice, this list of conditions and the following
; disclaimer.
; Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following
; disclaimer in the documentation and/or other materials provided with the distribution.
; Neither the name of the author nor the names of its contributors may be used to endorse or promote
; products derived from this software without specific prior written permission.
;
; THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS
; OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY
; AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDERS OR
; CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
; CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
; SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
; THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
; OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
; POSSIBILITY OF SUCH DAMAGE.
;
; Send bugs, suggestions and ideas to:
;
; felix@call-with-current-continuation.org
;
; Felix L. Winkelmann
; Unter den Gleichen 1
; 37130 Gleichen
; Germany


letter [a-zA-Z]
digit [0-9]
digit16 [0-9a-fA-F]
digit8 [0-7]
space [ ]

%%

\\[ ] (yycontinue)
\\?\13?\n (if pp-mode
(begin
(set! pp-mode #f) 'pp-end)
(yycontinue) )
{space}+ (yycontinue)
\9+ (yycontinue)
\13+ (yycontinue)
\12+ (yycontinue)
"//" (let loop ()
(let ([c (yygetc)])
(if (or (eq? 'eof c) (char=? #\newline c))
(begin
(if pp-mode
(begin
(set! pp-mode #f)
'pp-end)
(yycontinue) ) )
(loop) ) ) )
"/*" (let loop ([c (yygetc)])
(cond [(eq? 'eof c) (parsing-error "unexpected end of comment")]
[(char=? #\newline c) (loop (yygetc))]
[(char=? c #\*)
(let ([c2 (yygetc)])
(if (eq? #\/ c2)
(yycontinue)
(loop c2) ) ) ]
[else (loop (yygetc))] ) )
"enum" 'enum
"typedef" 'typedef
"extern" 'extern
"static" 'static
"___fixnum" 'fixnum
"___number" 'number
"___symbol" 'symbol
"___bool" 'bool
"___pointer" 'pointer
"___u32" 'u32
"___s32" 's32
"___s64" 's64
"int64_t" 's64
"__int64" 's64
"bool" 'bool
"___safe" 'callback
"___declare" 'declare
"___scheme_value" 'scheme-value
"___scheme_pointer" 'scheme-pointer
"___byte_vector" 'byte-vector
"C_word" 'scheme-value
"___abstract" 'abstract
"___specialize" 'specialize
"___byte" 'byte
"___discard" 'discard
"___in" 'in
"___out" 'out
"___inout" 'inout
"___mutable" 'mutable
"___length" 'length
"size_t" 'size_t
"int" 'int
"unsigned" 'unsigned
"signed" 'signed
"float" 'float
"double" 'double
"short" 'short
"long" 'long
"char" 'char
"void" 'void
"struct" 'struct
"union" 'union
"const" 'const
"class" 'class
"public" 'public
"protected" 'protected
"private" 'private
"volatile" 'volatile
"namespace" 'namespace
"virtual" 'virtual
"explicit" 'explicit
"inline" 'inline
"using" 'using
"@interface" 'interface
"@implementation" 'implementation
"@end" 'end
"@class" 'objc-class
"@protocol" 'protocol
"@public" 'objc-public
"@protected" 'objc-protected
"@private" 'objc-private
"@encode" (list 'id "@encode")
"@defs" (list 'id "@defs")
"@selector" (list 'id "@selector")
"..." 'dots
^[ \t]*#[ ]*define (begin (set! pp-mode 'define) 'pp-define)
^[ \t]*#[ ]*include (begin (set! pp-mode 'include) 'pp-include)
^[ \t]*#[ ]*import (begin (set! pp-mode 'import) 'pp-import)
^[ \t]*#[ ]*ifdef (begin (set! pp-mode #t) 'pp-ifdef)
^[ \t]*#[ ]*ifndef (begin (set! pp-mode #t) 'pp-ifndef)
^[ \t]*#[ ]*elif (begin (set! pp-mode #t) 'pp-elif)
^[ \t]*#[ ]*if (begin (set! pp-mode #t) 'pp-if)
^[ \t]*#[ ]*else (begin (set! pp-mode #t) 'pp-else)
^[ \t]*#[ ]*pragma (begin (set! pp-mode #t) 'pp-pragma)
^[ \t]*#[ ]*endif (begin (set! pp-mode #t) 'pp-endif)
^[ \t]*#[ ]*error (begin (set! pp-mode #t) 'pp-error)
^[ \t]*#[ ]*undef (begin (set! pp-mode #t) 'pp-undef)
# '(op "#")
"if" 'if
"else" 'else
@?\" (let loop ([cs '()])
(let ([c (yygetc)])
(cond [(eq? 'eof c)
(parsing-error "unexpected end of string constant")]
[(char=? c #\\) (loop (cons (yygetc) cs))]
[(char=? c #\")
(list 'string (list->string (reverse cs))) ]
[else (loop (cons c cs))] ) ) )
\'\\{digit}{digit}{digit}\' (list 'char (string->number (substring yytext 3 5) 8))
\'\\0\' '(char #\nul)
\'\\a\' '(char #\alarm)
\'\\b\' '(char #\backspace)
\'\\f\' '(char #\page)
\'\\n\' '(char #\newline)
\'\\r\' '(char #\return)
\'\\t\' '(char #\tab)
\'\\v\' '(char #\vtab)
\'\\.\' (list 'char (string-ref yytext 2))
\'.\' (list 'char (string-ref yytext 1))
({letter}|_)({letter}|_|{digit})* (list 'id yytext)
0(x|X){digit16}+ (list 'num (string->number (substring yytext 2 (string-length yytext)) 16))
0{digit8}+ (list 'num (string->number (substring yytext 1 (string-length yytext)) 8))
[-+]?{digit}+(\.{digit}*)?([eE][-+]?{digit}+)?
(list 'num (string->number yytext))
" (if (eq? pp-mode 'include)
(let loop ([s '()])
(let ([c (yygetc)])
(cond [(eq? 'eof c) (parsing-error "unexpected end of include file name")]
[(char=? #\> c)
(set! pp-mode #f)
`(i-string ,(list->string (reverse s))) ]
[else (loop (cons c s))] ) ) )
`(op ") )
"(" 'open-paren
")" 'close-paren
"[" 'open-bracket
"]" 'close-bracket
"{" 'open-curly
"}" 'close-curly
"," 'comma
";" 'semicolon
"*" 'star
"."|"+="|"-="|">>="|"|"*="|"/="|"%="|"%"|"&="|"|="|"^="|"+"|"-"|"/"|">="|"|"=="|"|">>"|"&&"|"||"|"&"|"|"|">"|"|"^"|"~"|"?"|"::"|":"|"="|"!="|"!"
(list 'op yytext)
> (begin (set! pp-mode #f) 'stop)
> (lexer-error (yygetc))






The chunks of code may be a bit confusing. The second statement of each entry is a piece of Scheme code which, when evaluated at a specific point in the text, will produce the appropriate token.

So now we're ready to lex! First, we need to generate our lexing program.


Gambit v4.3.2

> (include "silex.scm")
> (lex "autoffi.l" "autoffi.scm")
#t


This took in the specification file "autoffi.l" and generated "autoffi.scm". Now we can use this program for lexing C code, such as gl.h!

Here's a simple program using our lexer:


(include "autoffi.scm")

;; Token generators depend on these definitions
(define pp-mode #t)
(define (lexer-error c)
(display "*** ERROR *** invalid token: ")
(write c)
(newline)
(exit 1))

(define (lex-gl output-port)
(lexer-init 'port (open-file "gl.h"))
(let loop ()
(let ((tok (lexer)))
(write tok output-port)
(newline)
(if (not (eq? tok 'stop))
(loop)))))

(lex-gl (current-output-port))







It basically opens the file "gl.h" and passes it off to the C lexer. Then we read all of the tokens and write them to standard output. The output is this:


pp-ifndef
(id "__gl_h_")
pp-define
(id "__gl_h_")
pp-ifdef
(id "__cplusplus")
extern
(string "C")
open-curly
pp-endif
pp-end
typedef
unsigned
int
(id "GLenum")
semicolon
typedef
unsigned
char
(id "GLboolean")
semicolon
typedef
unsigned
int

... snip ...

pp-define
(id "GL_POLYGON_MODE")
(num 2880)
pp-define
(id "GL_POLYGON_SMOOTH")
(num 2881)
pp-define
(id "GL_POLYGON_STIPPLE")
(num 2882)
pp-define

... snip ...

(id "glGetTexParameteriv")
open-paren
(id "GLenum")
(id "target")
comma
(id "GLenum")
(id "pname")
comma
(id "GLint")
star
(id "params")
close-paren
semicolon
extern
void
(id "glHint")
open-paren
(id "GLenum")
(id "target")
comma
(id "GLenum")
(id "mode")
close-paren
semicolon

... (thousands of more lines, of course)







That's great! We have "lexed" the OpenGL header gl.h (I just facepalmed myself for using the word "lexed").

Parsing

Now we need to write a parser which will take the tokens and construct somewhat of an AST for us. Since we're only interested in the preprocessor constants, it's pretty easy. Here's what I came up with. This program will pick out most preprocessor statements (and typedefs too, just for fun):



(define (parser-error err)
(display err)
(exit 1))

(define (parse input-port output-port)
(define (writer node)
(write node output-port)
(newline))
(let loop ((mode #f) (tokens '()))
(let ((t (read input-port)))
(case t
((pp-end)
(if (pair? tokens)
(writer (reverse tokens))
(parser-error "invalid preprocessor statement: pp-end"))
(loop #f '()))
((pp-define pp-include pp-if
pp-ifdef pp-ifndef
pp-else pp-endif
pp-undef pp-import
pp-pragma pp-error)
(loop 'pp (list t)))
((typedef)
(loop 'typedef (list t)))
((semicolon)
(if (not (null? tokens))
(writer (reverse tokens)))
(loop #f '()))
((stop)
#t)
(else
(loop mode (if mode
(cons t tokens)
tokens)))))))

(parse (current-input-port)
(current-output-port))






Running this program, given the input of our lexing program, spits out:


(pp-ifndef (id "__gl_h_"))
(pp-define (id "__gl_h_"))
(pp-ifdef (id "__cplusplus"))
(pp-endif)
(typedef unsigned int (id "GLenum"))
(typedef unsigned char (id "GLboolean"))
(typedef unsigned int (id "GLbitfield"))
(typedef signed char (id "GLbyte"))
(typedef short (id "GLshort"))
(typedef int (id "GLint"))
(typedef int (id "GLsizei"))
(typedef unsigned char (id "GLubyte"))
(typedef unsigned short (id "GLushort"))
(typedef unsigned int (id "GLuint"))
(typedef float (id "GLfloat"))
(typedef float (id "GLclampf"))
(typedef double (id "GLdouble"))
(typedef double (id "GLclampd"))
(typedef void (id "GLvoid"))
(typedef long (id "GLintptr"))
(typedef long (id "GLsizeiptr"))
(pp-ifndef (id "GL_TYPEDEFS_2_0"))
(pp-define (id "GL_TYPEDEFS_2_0"))
(typedef char (id "GLchar"))
(pp-endif)
(pp-ifndef (id "GL_GLEXT_LEGACY"))
(pp-endif)
(pp-define (id "GL_LOGIC_OP") (id "GL_INDEX_LOGIC_OP"))
(pp-define (id "GL_TEXTURE_COMPONENTS") (id "GL_TEXTURE_INTERNAL_FORMAT"))
(pp-define (id "GL_VERSION_1_1") (num 1))
(pp-define (id "GL_VERSION_1_2") (num 1))
(pp-define (id "GL_VERSION_1_3") (num 1))
(pp-define (id "GL_VERSION_1_4") (num 1))
(pp-define (id "GL_VERSION_1_5") (num 1))
(pp-define (id "GL_VERSION_2_0") (num 1))
(pp-define (id "GL_VERSION_2_1") (num 1))
(pp-define (id "GL_ACCUM") (num 256))
(pp-define (id "GL_LOAD") (num 257))
(pp-define (id "GL_RETURN") (num 258))
(pp-define (id "GL_MULT") (num 259))
(pp-define (id "GL_ADD") (num 260))
(pp-define (id "GL_NEVER") (num 512))
(pp-define (id "GL_LESS") (num 513))
(pp-define (id "GL_EQUAL") (num 514))
(pp-define (id "GL_LEQUAL") (num 515))
(pp-define (id "GL_GREATER") (num 516))
(pp-define (id "GL_NOTEQUAL") (num 517))
(pp-define (id "GL_GEQUAL") (num 518))

... snip ...







Now we're getting somewhere. This isn't quite an AST, but lets call it that. Now we need to analyze the AST and generate our preprocessor constants interface. We'll need a program that identifies tokens and their sub-parts, and is able to pull data out of each token. Here's this program:



(define (parser-error . args)
(for-each
(lambda (x)
(if (not (string? x))
(write x)
(display x)))
args)
(newline)
#f)

(define (num-token? token)
(and (pair? token)
(eq? (car token) 'num)
(eq? (length token) 2)))

(define (make-num token)
(cadr token))

(define (id-token? token)
(and (pair? token)
(eq? (car token) 'id)
(eq? (length token) 2)))

(define (make-id token)
(string->symbol (cadr token)))

(define (constant-token? token)
(and (pair? token)
(eq? (car token) 'pp-define)
(eq? (length token) 3)))

(define (make-constant-expr token)
(let ((id-token (cadr token))
(val-token (caddr token)))
(if (not (id-token? id-token))
(parser-error "invalid id: " id-token)
(let ((id (make-id id-token))
(val (cond
((id-token? val-token) (make-id val-token))
((num-token? val-token) (make-num val-token))
(else (parser-error
"invalid constant value: "
val-token)))))
(and val
`(define ,id ,(if (symbol? val)
`(lambda () ,val)
val)))))))

(define (parse input-port output-port)
(let loop ()
(let ((token (read input-port)))
(if (constant-token? token)
(begin
(write (make-constant-expr token) output-port)
(newline)))
(if (not (eq? token #!eof))
(loop)))))

(parse (current-input-port)
(current-output-port))






Now, give the output of our parser to this, and this generates:


(define GL_LOGIC_OP (lambda () GL_INDEX_LOGIC_OP))
(define GL_TEXTURE_COMPONENTS (lambda () GL_TEXTURE_INTERNAL_FORMAT))
(define GL_VERSION_1_1 1)
(define GL_VERSION_1_2 1)
(define GL_VERSION_1_3 1)
(define GL_VERSION_1_4 1)
(define GL_VERSION_1_5 1)
(define GL_VERSION_2_0 1)
(define GL_VERSION_2_1 1)
(define GL_ACCUM 256)
(define GL_LOAD 257)
(define GL_RETURN 258)
(define GL_MULT 259)
(define GL_ADD 260)
(define GL_NEVER 512)
(define GL_LESS 513)
(define GL_EQUAL 514)
(define GL_LEQUAL 515)
(define GL_GREATER 516)
(define GL_NOTEQUAL 517)
(define GL_GEQUAL 518)
(define GL_ALWAYS 519)

... snip ...







And we have a list of OpenGL preprocessor constants available to us in Scheme! Notice how we aren't actually using any of the FFI mechanisms. Since all boundaries between Scheme and C land are typed, we can't really pull out preprocessor constants from C. We aren't trying to make this FFI compatible across versions of header files either. The point of automatically generating these interfaces is so that it's easier to re-generate them against any version/platform.

I hope someone else found this as interesting as I did. I will be working on parsing typedef's and function declarations as well, which is a natural extension to the above programs. Having such technology will make it almost painless to be working in Scheme, where I'm usually cut off from all of the math/graphics/etc. C libraries out there.

okonomiyaki

okonomiyaki

 

typing & building

Merry Christmas Eve!

There are three things I've been thinking a lot about recently: module systems, build systems, and type systems. These disparate technologies are related in funny ways: a build system inherently falls out of a module system, and type systems must be aware of module boundaries. I mainly want to talk about build systems and how I plan to develop my project (some talk of type systems will probably creep in).

Right now, I'm handwriting Makefiles (for GNU make). I wonder how many people just cringed. My project is really small right now, so it's suitable. It's also really easy to write the Makefile, because I don't specify dependencies.

You see, I'm using a language called Scheme, which is on the level of Python in terms of "higher-level"-ness. I don't really even need a build system, since I could load up all of the code at runtime and interpret it. However, I'm using an implementation of Scheme called Gambit-C which compiles to C code, and ends up running really fast when compiled. So I want some way of compiling code automatically.*

It turns out that it's really easy as long as I stick with Gambit-C Scheme for the whole project. I just need to use the procedure (LOAD name). At runtime, LOAD looks for the compiled dynamic library under that name, loads it if it finds it, and otherwise tries to load in the source code file for interpretation. I write my project using individual scheme (.scm) files as modules, and load in any dependencies like so:


--- render.scm ---

(load "opengl")
(load "scene")

(define (render-it thing)
(glBegin GL_TRIANGLES)
...)


Now, I can choose whether or not to compile a module (a scheme file) into a dynamic library. If I don't compile it, Gambit will simply load in the source and evaluate it.

Since the LOAD procedure happens at runtime, if I do compile a module, none of its dependencies are compiled with it. This means that I don't need any special mechanism of recompiling a module when I change one of its dependencies, since it will always be loaded at runtime.

I really like this method. Marc Feeley, author of Gambit-C, obviously wrote LOAD with this kind of usage in mind. It's like compiling individual C files into dynamic libraries and loading them at runtime - but that would be more difficult for a few reasons (I don't think you can reload dynamic libraries, wouldn't that cause duplicate symbols to conflict?).

The problem is that I don't think Gambit-C Scheme is going to cut it performance-wise for my whole project. I plan on developing in it for a while, but eventually at least part of the core system will need to be rewritten in C or PreScheme. When that comes, I'll need to figure out how I want to develop the system. I'm enjoying the flexibility of dynamically loading in libraries, since I can reload them at runtime, compile them separately, etc.

I guess I want to avoid creating a large C codebase that takes hours to compile, but maybe I'm over-analyzing it. After thinking through the big difference it would make, I realized it comes down to this: static typing versus dynamic typing.

C, C++, and even PreScheme are statically typed, meaning the compiler can reason about the type of every piece of data at compile-time. This has a huge impact on the coding style and how its built. Technically, it's the reason why header (.h) files exist. And it's the reason why it's easy for build times to increase as the codebase size increases, even for small changes. Whenever an interface changes, you have to rebuild all modules depending on it (to re-verify all the types).

Scheme, on the other hand, is dynamically typed. There are no assertions about types at compile-time; rather, every piece of data is tagged with a type at runtime. This gives you enormous flexibility, such as being able to do what I described above with LOAD.

I favor both systems. Static typing lets you prove something about the whole program at compile time, which can be a huge time saver when finding bugs. When I use dynamic typing, I tend to write a huge amount of unit tests in an effort to cover every execution path. Static typing also enables many optimization techniques. On the other hand, dynamic typing gives you a lot of flexibility. I've discovered that dynamic typing is fine AS LONG AS you have an appropriate debugger, and I tend to develop faster when using it.

I predict that I'll have a mix when my project gets bigger. I like to prototype with Gambit-C Scheme since it is really fast to develop with. However, for optimized code that needs to be maintained (probably parts of the core libraries), I'll probably end up writing them in a statically-typed, faster language. When this comes, it'll be interesting to figure out what kind of a build system I want to use.

My first couple posts will be thought dumps. I'd like to hear if anyone else is using a higher-level language like Scheme, Python, or something else for high-performance graphics development. Have you enjoyed it, or do you miss C or C++? How do you typically develop (in what IDE, what build process, etc.)?

* I also need to compile scheme files that contain C code. Gambit-C has mechanisms for inlining C code or interfacing into external C code, and this code has to be compiled for obvious reasons.

okonomiyaki

okonomiyaki

 

Hacking

Haha, boy I love hacking. It excites a strange feeling of rebellion.

I am working on making my development environment as comfortable as possible. My thoughts an IDE's and workspaces are this: do whatever makes you happy. It's tempting to evangelize and build up an attitude of "being right" (see the Emacs/VI holy war). But whatever; you're the one doing the work, and you should choose how to do it.

With that said, I can't stand IDE's. I think they enforce an extremely specific workflow and style of development, and also make it freaking annoying to tweak many parts of the build process. I'm an Emacs guy. I thought about writing up a formal review of current IDE's and why they don't work for me, but I don't really feel like doing it anymore. Let's just say I have used both Visual Studio and Xcode, but I've never been happier with Emacs. Maybe I can show you why in the next couple of posts.

I'll explain my opening statement at some point, I promise. I suddenly feel like I should attempt to describe Emacs, however, since much of what I will work on involves customizing it. Emacs is a text editor written mostly in Emacs Lisp. Obviously, ELisp was built for Emacs, and the whole language is exposed to you. You are allowed to redefine any part of the text editor itself, and as you use Emacs, you begin tailoring it to your specific needs. The result is two-fold: there are several packages out there you can install which turn Emacs into some amazing things, and you get to transform your text editor into something perfect for you.

My current focus is making my development environment in Emacs as fun and intuitive as possible. Why don't I start with a simple program, and see how I want to develop it from there?

I want my first program to:


Use Cocoa's application framework and open up a window
Embed a Gambit-C Scheme function which prints out some text on load
Be invokable on the command-line, such as `./main' (easy management of stdout/stderr, also can use Gambit-C's REPL)


There's not much code to do this, but it's certainly a start in figuring out issues with using Cocoa outside of Xcode, Gambit-C Scheme integration, and more. Here's the program and the output:


--- main.c --------------------------------------
#import
#define ___VERSION 403002
#include "gambit.h"
#include "engine.h"
#include "stdio.h"

#define LINKER ____20_link__

___BEGIN_C_LINKAGE
extern ___mod_or_lnk LINKER (___global_state_struct*);
___END_C_LINKAGE

___setup_params_struct setup_params;

int main(int argc, char* argv[]) {
___setup_params_reset (&setup_params);
setup_params.version = ___VERSION;
setup_params.linker = LINKER;

___setup(&setup_params);
print_something();
int ret = NSApplicationMain(argc, argv);
___cleanup();
return 0;
}

--- engine.h --------------------------------------

void print_something();

--- engine.scm --------------------------------------

(c-define (print-something) () void "print_something" ""
(display "Hello, World!"))

--- Makefile --------------------------------------

gsc -debug -link -o link_.c engine.scm
gcc -o main main.c link_.c engine.c
-D___LIBRARY
-I/usr/local/Gambit-C/current/include
-lgambc
-framework Cocoa -framework OpenGL
-sectcreate __TEXT __info_plist Info.plist

---------------------------------------------------

(there's also a supporting .nib file and Info.plist which Cocoa
reads to build the window from)









Sweet! There's one big problem which the screenshot doesn't show: Ctrl-C doesn't kill the program. After some research, it looks like Gambit-C takes control over the interrupts, and since I never call back into Gambit it doesn't have a chance to respond. This is perfectly reasonable, and should be solved once we're calling Gambit in the rendering code.

My second goal was to have some triangles on the screen, rendered from Scheme code. It's pretty much the same program above, but some more Cocoa integration to initialize OpenGL and such. The only interesting part is probably the Scheme code, so I'll post it:



(define medium #f)

(c-define (init-engine) () void "init_engine" ""
(set! medium (obj-load "/Users/james/projects/scheme/graphics-gambit/resources/medium.obj")))

(define x 0.0)

(c-define (run-frame) () void "run_frame" ""
(set! x (+ x 1.5))
(glLoadIdentity)
(glRotatef 90.0 1.0 0.0 0.0)
(glRotatef x 0.0 0.0 1.0)
(glScalef 0.02 0.02 0.02)
(glClearColor 0.2 0.3 0.4 1.0)
(glClear GL_COLOR_BUFFER_BIT)
(glColor3f 1.0 0.9 0.8)
(glBegin GL_TRIANGLES)
(for-each
(lambda (triangle)
(glVertex3f (vector3d-x (triangle-v1 triangle))
(vector3d-y (triangle-v1 triangle))
(vector3d-z (triangle-v1 triangle)))
(glVertex3f (vector3d-x (triangle-v2 triangle))
(vector3d-y (triangle-v2 triangle))
(vector3d-z (triangle-v2 triangle)))
(glVertex3f (vector3d-x (triangle-v3 triangle))
(vector3d-y (triangle-v3 triangle))
(vector3d-z (triangle-v3 triangle))))
(obj-data-triangles medium))
(glEnd))






INIT-ENGINE is called at the beginning of the program, and RUN-FRAME is called every 1/60th of a second. I wrote a .obj file loader so that I can have some stuff to test with, so OBJ-LOAD just returns a list of triangles from a mesh inside a .obj. Here's the result now:



Even sweeter! There's no lighting, so it looks bad, but it's a rotating 3d mesh spelling "medium".

Interrupting the program works now, although I'm getting a strange crash when the program exits. I'm shutting down the Gambit system wrong somehow. Need to fix.

My third goal was to learn how to track mouse and keyboard events with Cocoa. As I got into this, however, I realized that I can't do this. The windowing system can't possibly dispatch any user input events on the window. The process is still seen as a command-line process, a child of my current shell. The shell still receives all the mouse movements and key presses. It looks like I'll have to invoke the program as a windowed application.

On OSX, this is achieved by creating a bundled application and using `open'. So instead of `./main', I will do `open Main.app'. The reasons why I initially tried to avoid this is that it is no longer a child process, and now I have to track down where stdout/stderr are going and there's no chance of using Gambit's REPL.

In reality, I'll be invoking the app in several different ways, compiled with several different options, depending on how I intend to use it. Debugging scene rendering problems, cocoa, gambit code, etc. are all different which will require different tools. It'll be good to build in all the options of compiling & invoking the app.

Anyway, this is a lot more than I intended. I didn't even get to talk about configuring Emacs. But essentially, anything that happens on the command line can be hooked into Emacs, so my most common actions are only a keyboard shortcut away. I hope to show some example of this later.

ALSO, in reference to my opening statement, I was playing around with the idea of dynamically loading in Cocoa and keeping Gambit in control. It's a huge hack, since Cocoa is meant to handle the entry point and be statically compiled down to an executable. But essentially, I compiled a gambit program which wraps around Cocoa's NSApplicationMain procedure into a dynamic library. I then proceeded to load it and try to invoke it, which would hopefully create a window, but let me stay in Gambit land. I got a nice little error from Cocoa though; it couldn't find the preferences file which was compiled in as the __TEXT section. I'm guessing there's something I'd have to do to locate it in a dynamic library (it would be ok in a flat executable).


[19:07] james:~/projects/scheme/graphics-gambit
james% gsc -cc-options 'cocoa-entry.m glview.m glwindow.m -framework Cocoa -framework OpenGL -sectcreate __TEXT __info_plist Info.plist' entry.scm
cocoa-entry.m: In function 'cocoa_entry':
cocoa-entry.m:28: warning: passing argument 2 of 'NSApplicationMain' from incompatible pointer type

[19:07] james:~/projects/scheme/graphics-gambit
james% gsi
gsi
Gambit v4.3.2

> (load "entry")
2008-12-18 19:07:14.945 gsi[37765:10b] No Info.plist file in application bundle or no NSPrincipalClass in the Info.plist file, exiting





Oh boy, I think I just dumped a bunch of stuff here at the end. Forgive me. I'm still getting the hang of decent writing (I probably should have split this up).

okonomiyaki

okonomiyaki

 

Testimony

Hi. My name is James. I'm an optimization junkie. I enjoy blazing across the bare metal of computers' circuitry, dangerously soaring through the lands of computational existence and virtualized memory. I once even saved a kernel from panicking.

I haven't written C++ for 3 years. Phew that feels good. Although I have used C for various smaller tasks, I have generally been able to stay away from lower-level languages. At times, the withdrawal was difficult. Virtual machines? Dynamic types? anonymous functions? Is my program going to be excruciatingly slow, or simply slow? It seemed like those were the only options.

The intervention came when I was forced to program for the web. Suddenly I couldn't justify making code harder to write, read, or maintain for the sake of efficiency. Efficiency usually didn't really matter. I hated giving up my favorite hobby of optimizing, but I started writing shorter code faster, in languages I would have never considered. First came PHP and javascript, and then I started venturing into the lands of Lisp and Scheme. And that's when my world exploded.

Suddenly I was learning about compilers, evaluation, macros, and all sorts of other seemingly magical things. I started learning all sorts of new ways of solving problems. Expressing everything in terms of cons, car, cdr, and fold. And suddenly, as if completing a circle, I started learning about ways of optimizing these higher-level Scheme programs. Writing compilers became trivial, and various techniques allowed me to generally apply optimizations to scheme programs.

Lisp and Scheme, as awesome as they are, are still just tools. Probably the most important thing I've learned in the past couple years is this: use the right tool for the right job. No programming language is a silver bullet. But here's the question: how do you know what the right tool is? I think it takes experience. But even a newbie should be encouraged to think this way, to carefully evaluate what should go into his toolbox. It makes it much easier to manage and improve a project when you have a clear reason why you are doing it this way.

So here's a practical example. I think many people use C++ for graphics development simply because it's the industry standard. I wish more people would play around other languages which can perform just as well, if not better. My choice for low-level programming is PreScheme, a dialect of Scheme which compiles to highly efficient pure C (no virtual machine needed). I wish more people would use type inferencing systems instead of C's horrid type system. etc. etc.

So, that's the goal of this blog. I want to explore using efficient Scheme systems for programming intense graphics. Gambit-C Scheme will be used as the language; it's one of the fastest Scheme implementations out there, with full support for R5RS which includes closures, lists, continuations, and everything else. PreScheme will be used in various places where Gambit-C doesn't quite cut it performance-wise. Hopefully I'll inspire at least one or two people to avoid huge C or C++ codebases for graphics.

okonomiyaki

okonomiyaki

 

Atmospheric scattering

Hey guys, glad to be programming and posting again!

I have something to show too. Yay. I've been studying atmospheric scattering. I'm actually writing my senior paper on it, so I'll be looking at different implementations all semester. Over the past couple days I implemented Preetham's introduced in "A Practical Analytic Model for Daylight." It's a very easy one to implement, and hard to get looking right. But still, I've impressed my friends with these pictures:









If I was to seriously use this it would require a ton of tweaking. But I'm off to study other implementations because 1) this one sucks to be honest and 2) I need to for my paper anyway.

You can track my progress here:
http://okono.radicand.org/SIP/

okonomiyaki

okonomiyaki

Sign in to follow this  
  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

We are the game development community.

Whether you are an indie, hobbyist, AAA developer, or just trying to learn, GameDev.net is the place for you to learn, share, and connect with the games industry. Learn more About Us or sign up!

Sign me up!