Thanks for the insightful replies guys, I've got some thinking to do about this. Need to make my ideas more concrete.

On my Google+ post, someone asked me to describe a sample scenario of how this approach might be used. Here's what I wrote:

[quote name='shurcool']This is somewhat off the top of my head, but here's a hypothetical sample scenario of how this approach might be used to create a new app by an app designer.

I think the best metaphor to describe the process would be similar to a sculptor sculpting a shape out of some material. He makes changes to it until it looks like his target object.

You start by creating a new app template project with a window. The app is running so you can see how it looks like and make changes to it. You decide the default background color is no good and change it to be green (via some interface, suppose a voice recognition engine). Then you add 2 buttons and describe what clicking on each one does.

Assuming the implementation for all the features you request is already present and is found, you don't have to write any code. If not, you do.

The difference between this approach and what you can already create using Visual Studio or Xcode is that for a designer to create the above program it would take 3-5 minutes and it doesn't require any coding from his part. The coding gets outsourced to coders who write it from scratch or find existing work in a library. The designer doesn't have to know or go through the API to make changes to his app, he just describes them in natural language (or via drawings, or some other means).

I'm really making this up as I go, so it is what it is. Perhaps the vision described here is too hard to create today, but I care more about whether the vision is good and as long as it can be created eventually... In other words, I'm working on the design first (before implementation). ;)

Edit: I realize the above scenario is an oversimplification... It describes a very static app. What makes most apps hard to code is their highly dynamic behaviour. What happens when a button is pressed might vary wildly depending on a complex state. Perhaps these difficulties can be overcome by having the system interpolate/fill in the blanks/guess the missing behaviour (and the app designer can correct it if it's wrong).[/quote]

Well, there is the whole field of Software Architecture in academia that tries to deal with this

Let me give you a few papers you might be interested in. First, the discipline was founded by Perry & Wolf in Foundations for the study of software architecture. There are some interesting concepts, like views (4+1 seems to be the most popular), styles (and older classificaiton), and ADLs (yet another popular classification). You briefly touch on what model-drivendevelopment is - which is implemented by EMFand used in eclipse 4.

Now, most of these papers are paid. If you are in university, chances are you can get them for free via institutional login. Now let me expand a little bit on this Most of the links will be to the free Wikipedia. The introductory article on software architecture seems to give a good overview on the topic, with links to free content.

If you study SA or Software Engineering (SE), chances are, you will be amazed how much different academic and industrial approaches are. I sometimes feel that scientists are like children playing in the sand (assuming a perfect world), but that is OK I guess.

Let me start first with what you call separation of "design" and "implementation". There has been a lot of work on that. One of the accepted ideas is that you can attack the problem by explicitly describing your architecture. You do that in an Architecture Description Language. They differ in shape and form, but most commonly there is a high-level programming language, which can describe the basic architectural items: components and connectors. Ideally, they would also support describing varying styles and/or "dynamic" vs "static" architectures. Then you somehow invoke a compiler that verifies the syntax, and runs some validation checks to confirm that there are no logical problems with your architecture. Since ADLs are formal languages, formal analysis can be performed on the model, thus possibly discovering race conditions or other conflicts.

Now, there are different opinions as to what follows ADLs, or how they should be implemented.

One view is that you should couple architecture and implementation. You design the architecture, and then write the implementation. You can have run-time and compile-time checks for architectural conformance. This makes certain that the two are kept in synch, as they reside in the same codebase, thus share compiler and runtime environment. The most notable example in academia for this is called ArchJava (paper (free), web site), which is specific to to Java, but there are attempts to port it for other languages.

Another view is that you should keep them separate. You then generate your code (or at least a skeleton) from the architectural model. An expansion of that view is Object Management Group (OMG)'s Model Driven Architecture (MDA). These are the same guys that did UML and CORBA. You design your application, and then a tool, through a series of transformations and using predefined templates, generates the source code for you. Very notable example here is EMF. It seems that IBM favors this approach, as it is used in eclipse - a real-world piece of software - that many people use on a daily basis. I never thought anybody will go that route, but it seems that if there is will, there is a way. I guess this comes somewhat close to what the thread opener had in mind.

Yet another view on ADLs is that you should keep them separate, and limit their use to just checking the saneness of your model, making sure you don't make trivial mistakes. The idea is to take advantage of more accessible form of formal methods. The new wave of ADLs, most notably byADL (paper - paid, site), focus on you creating DSL specific to your project or area (e.g. web programming). The advantage is that it won't force you to describe your architecture in terms that don't make sense, and it will cover your specific needs.

What many people seem to be getting at is called architectural styles and patterns in academia. The waters are murky as to what is the difference between styles and patterns, as some authors say they are mostly interchangeable, while others point out that patters are bigger in terms of scale and abstraction. Anyhow, MVC is sometimes classified as a style, and as is probably the most well-known and widely used style in programming. Some people mentioned MVVC, which is most notably featured in WPF/Silverlight.

Now, as to my own opinion on the subject... I just don't like the idea of automatically generated code. I don't think it is reasonable to specify the architecture up-front (or to expect it to be correct). It usually evolves, and keeping some description up to date is IMO waste of time and effort better spent on another iteration of that evolution. There are many tools that automatically extract (UML) class diagrams from your code, so I don't see the benefit of yet another description. The main problem I see with the academic approach is that they mix high-level and low-level concepts. A lot. They work on tiny projects (tested on 20-30k LOC - that is less that what a cart + check out workflow is); work is done in unrealisic ways (some papers change the freaking code of the Java standard library - hello distribution nightmare), and plain childish attempts are made (look up AliasJava for an example of what I mean - unleash the complexity of C++ onto Java).

From my point of view, you can, and should, briefly document your architecture and doc-comment the code. In your nightly builds, build the automated documentation (javadoc for Java/sandcastle or ndoc for .NET) and generate a few automated class diagrams. That, plus a modern IDE, would be enough for a competent developer to cope with even quite big code bases.

What if we found some way to cleanly separate these two things, such that the implementation could be interchanged while keeping the design untouched. So a user wouldn't even notice any change from their perspective (aside from potentially increased performance, etc.).

As others have mentioned, what you're describing more or less is the MVC pattern. However, I think most definitions of MVC are either flawed or incomplete. Conceptually, a View layer is purely for presenting and user interaction, but too often 'back-end' details end up bleeding into the UI (in my experience, anyway). Also, I think most people get too hung up on abstract conflicts such as: Does this functionality belong in the model or the controller? Should I have a skinny model or a fat model? Etc.

Like I said, it seems incomplete. In order to make the idea of MVC useful, in my opinion, the UI should not be treated as not just a separate 'component' of the application, but rather a separate application unto itself which communicates with a back-end application via a clearly defined 'front-end' API. Conceptually, this results in leaner applications which are easier to reason about.

@shurcool: Did we answer your question? Or help you arrive at an answer? Just wondering.

Alpha_ProgDes, this thread has been extremely insightful and helpful to me. I wouldn't say it has "answered" the question, simply because the question is too broad to be completely answered.

That said, I wanted to add something.

Some people have suggested that basically what I'm describing is a "high-level language". Yes and no. I think the key difference between my pitched idea and a typical high-level language is that in a HL language, you don't really have control over how something happens on the low-level. You just say, "get this done" and hope the language figures it out in a way that's appropriate.

Sometimes this works, if the compiler/language is smart, and if you don't really care how something gets done.

However, with my approach, you can explicitly specify the low-level implementation for any high-level construct, if you wish (sometimes, you have no choice but to do it). "Get this high-level thing done, and do it this way."

Does that offer an advantage over the standard HL languages where you have no such control?

Some people have suggested that basically what I'm describing is a "high-level language". Yes and no. I think the key difference between my pitched idea and a typical high-level language is that in a HL language, you don't really have control over how something happens on the low-level. You just say, "get this done" and hope the language figures it out in a way that's appropriate.

Sometimes this works, if the compiler/language is smart, and if you don't really care how something gets done.

However, with my approach, you can explicitly specify the low-level implementation for any high-level construct, if you wish (sometimes, you have no choice but to do it). "Get this high-level thing done, and do it this way."

What you're describing is broadly similar to the idea of using inline assembly in C or C++, which is something games in particular used to do a lot back in the days of 16-bit DOS. As in that scenario, I would argue that the high-level and low-level aspects of the language are actually using two separate languages with different idioms and ways of approaching the problem.

Does that offer an advantage over the standard HL languages where you have no such control?[/quote]

For what you're trying to do, not really. Consider that the objection in the OP to the idea of source code was that design and implementation should be distinct. If you're "falling back to the lower levels" to implement a particular feature, then you're blurring the line between the high- and low-levels - between the design and the implementation, which is exactly the opposite of what you're trying to enforce. If it becomes idiomatic to specify the low-level implementation for a high-level construct in any given program (and for performance-critical programs like games, it will), then idiomatically the "design" and "implementation" will not be distinct the way you want, which means that practically speaking your language will offer no advantages over the existing solutions you object to in the OP.