• Create Account

## Inventing on Principle - amazing video!

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

66 replies to this topic

### #21tstrimple  Prime Members

Posted 20 February 2012 - 09:57 PM

I like the point he is trying to get across, but his examples to me seem to be more gimmick than practical for software development. As others have stated, you have to build a program specifically for being able to tinker with code in real time like that and it is game or engine specific. I would have loved to have had this functionality while tweaking my WP7 game, and I probably could have had it with a few days to a week of development, but my time was better spent tweaking and running for an hour to get it close enough and moving on.

Posted 20 February 2012 - 10:04 PM

You try doing that with a scripting language. Go on, fire up Python/C#/JavaScript, and try to do what he's showing there.

From what I saw he had the ability to load up the last 2 seconds of gameplay, either by having each object keep a stack of information for 2 full seconds, or by using a negative value for his simulation delta. Then show ghost forwarding of the simulation by calling the simluate function a bunch of times and saving those into another stack. Its a cool idea, but I didn't see anything that is unique other than that design. I'm not sure why you feel no one could do that or that it would be remotely impossible for me or anyone else here to implement that. I watched the tree and 2D game. The tree is really just a normal tool editor "I can use control variables at run time to change my tree." I didn't get that part at all.

### #23Hodgman  Moderators

Posted 20 February 2012 - 10:15 PM

I like the point he is trying to get across, but his examples to me seem to be more gimmick than practical for software development. As others have stated, you have to build a program specifically for being able to tinker with code in real time like that and it is game or engine specific.

I think his point is that you should build said code for your specific game or engine ;)
His examples are more geared towards how adding visualisation functionality worked for him -- the initial example with the tree is something that he's personally interested in (programmed art).

The 2nd example shouldn't be news for any serious game developer -- we're always adding developer visualisations to our games, simply because his point about being able to see what you're doing is valid. The specific example of diplaying several frames worth of simulation data is just an example -- every game I've worked on has some kind of visualisation modes in it though.

The 3rd example should also be familiar with any game programmer -- it's Visual Studio's "watch" window, but in a different incarnation.

### #24swiftcoder  Senior Moderators

Posted 21 February 2012 - 12:17 AM

I'm not sure why you feel no one could do that or that it would be remotely impossible for me or anyone else here to implement that.

WTH did you think I meant? Of course its trivial to implement. The point is the it *isn't* implemented.

Tools should offer this sort of thing out of the box. Editors should popup little variable editor panels, then auto-recompile the function, and patch it into the running executable. Game engines should offer replay functionality, with the ability to observe shadows of previous/future states.

Nothing he shows is impossible, or even very difficult (he implemented it, after all). But in general, people aren't writing this kind of tool, even though it can make one's life so much easier.

Tristam MacDonald - Software Engineer @ Amazon - [swiftcoding] [GitHub]

### #25Mussi  GDNet+

Posted 21 February 2012 - 04:50 AM

I think his point is that you should build said code for your specific game or engine ;)

I think his point is that you could build said code for your specific game or engine ;D

### #26Antheus  Members

Posted 21 February 2012 - 05:15 PM

WTH did you think I meant? Of course its trivial to implement. The point is the it *isn't* implemented.

It's not implemented because you need to fight constant factors, halting problem or NP-class problems at every step.

For a typical wordpress app, tool like that would need to make by-value copies of deployment and database for each editor.
For algorithms, table looks amazing. Until the user types fib(MAX_INT). Or Acekrman(4,4).

Tools should offer this sort of thing out of the box. Editors should popup little variable editor panels, then auto-recompile the function, and patch it into the running executable.

Not with languages we have.

Consider this:
template < size_t N > struct OptimizedTexture {
int getWidth() { return N; }
}
Texture is hard-coded to size N by N and pipeline supports that. In a tool I now change N from 512 to 2048 - to see how much better it looks. The simple recompilation now means entire asset repository needs to be rebuilt.

Nothing he shows is impossible, or even very difficult (he implemented it, after all). But in general, people aren't writing this kind of tool, even though it can make one's life so much easier.

A system like that exists. (insert obligatory comment about Lisp doing it before everyone).

As with everything in CS and SE, what can be done has been done by mid-70s.

But due to commercial factors, legal reasons and general direction of the market, such open-ended computational devices failed. They also tend to be suited to computational tasks, not IT or consumer devices.

Building a fully introspectable system today is also impossible. Between proprietary OS layer, the opaque hardware and unpublished hardware, all one has left is a tiny sandbox to play in.

Theory of such systems is old. It's the original design of systems, where code = data and vice versa. For example, you take a window, send it over network and it pops up on another machine, fully functional. For simpler unix-like versions see also Plan9 and sam/acme editor.

As for use with today's tools? Not possible anymore. The two most popular OSes are completely gated through legal restrictions making development of such tools impossible and Linux seems to be competing on just how badly it can fragment, making development on anything more advanced that GNU make collapse under its own weight of incompatibilities.

CS ended in 1970s. There has not been a single meaningful advance since then. All systems in use today are merely poor attempts at replicating everything that has been done back then.

But if you have an idea on how to improve ad conversions by 0.3%, then Google or Facebook will pay you billions for it. It can be done in Visual Basic, it doesn't really matter.

### #27swiftcoder  Senior Moderators

Posted 22 February 2012 - 12:17 AM

For a typical wordpress app, tool like that would need to make by-value copies of deployment and database for each editor.

I said we need tool support. APIs are tools too. So what if we have to retool database layer to allow on-the-fly shallow-copy-modification of data?

For algorithms, table looks amazing. Until the user types fib(MAX_INT). Or Acekrman(4,4).

And then our tool displays "infinite recursion", or some such. Better to provide UI for finite cases, than not at all.

Not with languages we have. Consider this:

template < size_t N > struct OptimizedTexture { int getWidth() { return N; } }
Texture is hard-coded to size N by N and pipeline supports that. In a tool I now change N from 512 to 2048 - to see how much better it looks. The simple recompilation now means entire asset repository needs to be rebuilt.

That's an architectural issue, not a language issue. Not the compiler's fault that your asset pipeline only supports a single size of image.

A system like that exists. (insert obligatory comment about Lisp doing it before everyone).

It's a Turing machine, we get it. But quite frankly, whether in software or hardware, a Turing machine is still a Turing machine.

Building a fully introspectable system today is also impossible. Between proprietary OS layer, the opaque hardware and unpublished hardware, all one has left is a tiny sandbox to play in.

Just as well I didn't suggest a fully introspective system then. Our little sandbox allows us to examine the contents of memory, and replay functions with varying inputs - all you need for any of the items shown in the video.

As for use with today's tools? Not possible anymore. The two most popular OSes are completely gated through legal restrictions making development of such tools impossible and Linux seems to be competing on just how badly it can fragment, making development on anything more advanced that GNU make collapse under its own weight of incompatibilities. CS ended in 1970s. There has not been a single meaningful advance since then. All systems in use today are merely poor attempts at replicating everything that has been done back then. But if you have an idea on how to improve ad conversions by 0.3%, then Google or Facebook will pay you billions for it. It can be done in Visual Basic, it doesn't really matter.

Wow. You've exceeded even my tolerance for jaded old-timer ranting. I'm going to say good night...

Tristam MacDonald - Software Engineer @ Amazon - [swiftcoding] [GitHub]

### #28Antheus  Members

Posted 22 February 2012 - 08:24 AM

I said we need tool support. APIs are tools too. So what if we have to retool database layer to allow on-the-fly shallow-copy-modification of data?

That's an architectural issue, not a language issue. Not the compiler's fault that your asset pipeline only supports a single size of image.

Let's say my pipeline generates height at some fixed resolution using texture color. Then I plug in Google Maps as texture source. How do you reduce that to a shallow copy, when changing parameters requires full processsing of a petabyte of data or so?

It's a computational complexity problem, we simply don't have enough processing power even for small datasets, let alone those used daily for the most boring of CRUD apps.

And then our tool displays "infinite recursion", or some such. Better to provide UI for finite cases, than not at all.

To determine whether it's finite or infinite, you need to solve halting problem.

I suppose that's "simple".

You've exceeded even my tolerance for jaded old-timer ranting.

You asked why nobody is building tools like that.

Because we can't. Let's say I want to build a code analysis tools for Windows - I don't have access to WinAPI or compiler, since building Windows applications requires proprietary extensions and VC is closed. I'm only allowed to use existing DLLs to use whatever information they provide.

Heck, I'm not even allowed to set up a build farm for VC, since licensing terms of compiler and SDK forbid it.

### #29swiftcoder  Senior Moderators

Posted 22 February 2012 - 09:21 AM

Let's say my pipeline generates height at some fixed resolution using texture color. Then I plug in Google Maps as texture source. How do you reduce that to a shallow copy, when changing parameters requires full processsing of a petabyte of data or so?

Certain data sources/sinks are immutable - that's a given. Pretty much anything that comes from outside the source code, in fact.

You change a parameter relating to an immutable source/sink, and you get an error/undefined behaviour (exactly as happens if you make the same change today). The only difference is that this error message pops up as you work, rather than in a separate compilation step.

To determine whether it's finite or infinite, you need to solve halting problem.

Of course it isn't possible to find the limit in the general case, but it's simple enough to cap our tool at 100 stack frames, or 100 bytecode instructions, or whatever arbitrary criteria proves reasonable.

However, I'm not advocating for a generalised solution. A practical solution for a limited subset of problems is perfectly acceptable to me.

I'd like the ability to tweak constants at runtime, and this is eminently achievable. Hell, there's even a GDNet article that manages to set that up in C++ using some unholy macros. Give me IDE integration of those, and I'd be pretty happy.

And I'd like the ability to repeatedly execute isolated functions in a user-defined context, with output of variables at each step. That's trivial to do with a scripting language, and certainly not impossible to do with a compiled language. Obviously it only is of benefit in a certain class of algorithm development, but that's a lot better than nothing...

Tristam MacDonald - Software Engineer @ Amazon - [swiftcoding] [GitHub]

### #30Antheus  Members

Posted 22 February 2012 - 11:09 AM

I'd like the ability to tweak constants at runtime, and this is eminently achievable. Hell, there's even a GDNet article that manages to set that up in C++ using some unholy macros. Give me IDE integration of those, and I'd be pretty happy.

And I'd like the ability to repeatedly execute isolated functions in a user-defined context, with output of variables at each step.

*nix anything + scripting languages + emacs.

This isn't jaded old timer talk. Why are there so many people that are rabid about that combination and won't hear anything about an IDE. To them, IDE is a puny little toy for 3 year olds. And they're right.

But it does require one to completely turn around the notion of what a program is and instead treat everything as computation.

And needless to say, it's limited mostly to shuffling text, simply because drivers and other parts of OS aren't sufficiently exposed.

The obvious caveat is that if a feature is missing, "implement it yourself". But unlike the usual OSS stance, in this case implementing it yourself is just a few lines of code that solve a very specific problem with a very specific part of application. Making it reusable is somewhat counter-productive since it. by definition, limits what can be done.

For C and C++, we lack a sufficiently accessible compiler. Due to sheer complexity, they are a black box. But in ideal case, such compiler would be just a bunch of scripts. Want to monitor a variable?
x86mov.prototype.write = function() { ... }
Change register order? Just foo.replace[(ebx, eax), (eax, edx), (edx, abx)]. Replace code? Just take process.image.replace("foo", "bar").

Obviously, constructing a C++ compiler with such capabilities exceeds current capacities (cost/demand) of software engineering, but it's theoretically doable and would bring all the scripting language flexibility to compiled languages. LLVM is a tiny step into this direction. Compilation is essentially just a special case of lazy evaluation.

And again, we cannot ignore the fact that OS vendors then forbid, through legal means, use of such compilers by restricting access to OS APIs, restricting us to basic console/socket applications. Which is much bigger reason why nobody is doing it, outside of Apple who has immediate need for it in form of XCode and ARM/x86 cross compilation.

### #31Hodgman  Moderators

Posted 22 February 2012 - 06:18 PM

You asked why nobody is building tools like that.
Because we can't.

No, actually, we do. In every (good) video game engine, we do build tools like this.
If you can't edit content (code+data) and immediately see the new results on your screen, then your game engine needs an upgrade.

However, if you interpret the aim of such tools to be as general and broad and impossible as you can, then yes, you're right, it's definitely unpossible. Any time anyone brings up the halting problem, they're probably off in some academic la-la land instead of actually doing something useful in reality, like upgrading my damn game engine so I can see what I'm doing.

### #32Antheus  Members

Posted 23 February 2012 - 08:06 AM

You asked why nobody is building tools like that.
Because we can't.

No, actually, we do. In every (good) video game engine, we do build tools like this.
If you can't edit content (code+data) and immediately see the new results on your screen, then your game engine needs an upgrade.

But you need to build the tool that will fit into whichever project you're currently building. You don't have a tool first, then as you go along, it shapes the result. That's the idea behind these concepts.

An engine inevitably limits you to whatever it was designed for. Compare this to tree concept, which starts as bunch of lines, then experimentally moves into day/night scenery of a tree in wind.

instead of actually doing something useful in reality, like upgrading my damn game engine so I can see what I'm doing.

There are more articles from same author, including one of them mentioning the reason for these experiments.

As long as designer (whoever) needs to call onto a developer to do a tweak or do some technical bit, there's a huge impediment to creativity. Pipe dream goal of such experiments is to present tools that vanish out of sight.

In the same way one no longer needs a programmer to read mail, write docs or edit spreadsheets.

Imagine writer trying to write a new book using some text editor:
W: "I'm trying to present John as sarcastic, but the paragraph keeps breaking"
P: "Yea, the TLB is getting messed up, you've only got 4GB of RAM"
W: "Can you fix it?"
P: "Well, I could work around it if you reduce allegory to only 2 pages"
W: "But those sections define the premise"
P: "Could you rework the childhood suffering motive?"
W: "..."

Text editing is a blank canvas which doesn't limit the content written. But existing programming tools do, or at very least, we've exhausted creative potential they can realize. It's less that we cannot do interesting things with them, it's that the reach has plateaued.

### #33Hodgman  Moderators

Posted 23 February 2012 - 09:55 PM

An engine inevitably limits you to whatever it was designed for. Compare this to tree concept, which starts as bunch of lines, then experimentally moves into day/night scenery of a tree in wind.

No, the tree demonstration would map exactly as is onto a modern game engine. One viewport shows you your scene, the other shows you the design behind the scene, which you can tweak with realtime feedback. Playing with these controls leads to discovery.

As long as designer (whoever) needs to call onto a developer to do a tweak or do some technical bit, there's a huge impediment to creativity. Pipe dream goal of such experiments is to present tools that vanish out of sight.

Exactly! This is what we do, and that's the trend that game development has been following for 10 years. If many tasks are dependent on a coder it indicates your engine is more of a programmer's SDK rather than a modern game developer's toolbox.
We build the kind of stuff shown in these experiments every day - this is what game engines have become, so the examples in the video should not be surprising or "impossible/impractical" at all!

### #34Sirisian  Members

Posted 24 February 2012 - 12:17 AM

Just watched that video. The circuit editor is the most impressive thing. As someone that's tried to design an analog circuit he perfectly captures how complex it is to visualize the effects of small changes to a design. Especially if you compare it to what the market has for analog circuit editors which look like 1994 VB6 applications.

### #35way2lazy2care  Members

Posted 24 February 2012 - 08:32 AM

Exactly! This is what we do, and that's the trend that game development has been following for 10 years. If many tasks are dependent on a coder it indicates your engine is more of a programmer's SDK rather than a modern game developer's toolbox.
We build the kind of stuff shown in these experiments every day - this is what game engines have become, so the examples in the video should not be surprising or "impossible/impractical" at all!

Just a note, but doesn't Unreal let you change/tweak most everything in at least some runtime sense? I don't think you can edit the full game at runtime, but stuff like art I believe can be shown in engine as you are editing it iirc. I think it's also pretty trivial to hop into any point in a game to test while you edit (one button I think?). I'd assume CryEngine is similar, but have no experience with it. I know runtime tweaking of existing gameplay behavior is nothing taboo in the projects I've been on outside these two engine's as well--maybe underutilized, but the functionality has been there.

Looking back it seems like the problems he's talking about, while flashy, are already somewhat solved. It almost sounds like, "I'm going to make a tech demo of a less fully featured version of Unreal Kismet and brand it as my own innovation."

### #36Antheus  Members

Posted 24 February 2012 - 08:40 AM

Exactly! This is what we do, and that's the trend that game development has been following for 10 years. If many tasks are dependent on a coder it indicates your engine is more of a programmer's SDK rather than a modern game developer's toolbox.
We build the kind of stuff shown in these experiments every day - this is what game engines have become, so the examples in the video should not be surprising or "impossible/impractical" at all!

Where is my time traveling debugger then? Just like the guy moves forward and backward, where's a tool for that? Because that idea didn't exist until a few years ago, whether in games or elsewhere and has since been patented into oblivion.

Of course purpose-specific tools have been built since day 1. Presentation is more about further reaching ideas, about concepts that really are not available today.

And considering I do have all the latest tools, I don't see anything that has moved even an inch beyond the Turbo Pascal era of tooling. But apparently, VS 2011 will be gray instead of purple?

Obviously, I can built something myself, from scratch, specifically tailored to specific problem being solved. There is definitely an incredibly advanced tree editor. But it was and always will be nothing but a tree editor. Very useful, proven in day-to-day, but still just a tree editor.

Nobody is denying that increased hardware power made dynamic programming more accessible or that specific domains have state-of-the-art tools. Ironically, programming has always been about dynamic and interpreted languages, there was just a brief phase where compiled languages offer sufficient performance edge to dominate.

Experiments like these are more about thought process that long transcends the notion of tool itself. Just like there is a universal compiler instead of one being forced to build one specifically for each individual project, or one focused on tiny subsections of problems.

Whether they're useful or not will more depend on individual project. True creativity is in very short demand these days.

The circuit editor is the most impressive thing.

Tons of other examples, including further reasoning behind experiments.

Based on those, the guy started thinking about these concepts while working for Apple, where he realized designers were constrained by tools and still depended on programmers. I'd doubt that Apple of all things has problems keeping up with latest and greatest when it comes to design, prototyping or production. Thought it doesn't necessarily mean those experiences transfer to every other domain.

### #37way2lazy2care  Members

Posted 24 February 2012 - 09:04 AM

There is definitely an incredibly advanced tree editor. But it was and always will be nothing but a tree editor. Very useful, proven in day-to-day, but still just a tree editor.

It's not a tree editor. It's a rectangle, multiple splotchy circle, jagged line, single large circle editor that happens to also be able to make a tree with mountains/moon.

### #38Antheus  Members

Posted 24 February 2012 - 09:17 AM

It's not a tree editor. It's a rectangle, multiple splotchy circle, jagged line, single large circle editor that happens to also be able to make a tree with mountains/moon.

I was referring to SpeedTree or equivalent modules. Vastly superior to the demo, but in idea just a tree editor, built with disproportionate effort over past 10 years.

The demonstration suggests equivalent free form tool.

Kinda like the old spice guy. Look up, it's a tree, look down, now it's lines, look back up, it's waving in the wind. It's night. Now it's day.

### #39NoisyApe  Members

Posted 25 February 2012 - 10:33 AM

That was a very cool video, I like his ideas and he's a good presenter.

### #40MarkS  Members

Posted 25 February 2012 - 11:38 AM

Just watched that video. The circuit editor is the most impressive thing. As someone that's tried to design an analog circuit he perfectly captures how complex it is to visualize the effects of small changes to a design. Especially if you compare it to what the market has for analog circuit editors which look like 1994 VB6 applications.

There are programs like Spice that do circuit simulations already, although I do like his graphical waveform representation. However, some of his comments lead me to believe that he doesn't understand the purpose of a schematic. The symbols are there to show how the components connect together, not merely as a means of drawing a circuit on paper. I could take that schematic, and without knowing its purpose or how it works, get the components, wire them together and the design will (hopefully) work as the designer intended. His resulting "schematic" is all but useless. If I were trying to make a PCB out of that, I wouldn't know where to start!

Old topic!

Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.