• Advertisement
Sign in to follow this  

Is Python underestimated with what it can do?

This topic is 1629 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

 


Interpreted languages and VM languages are not intended to replace their lower-level counterparts.

I don't see any particular reason why not.

 

Microsoft research wrote an experimental OS entirely in managed code, and there are rumours that they may be commercialising it.

 

If your OS isn't written in native code, why should anything be?

 

 

What did they write the VM in?

Share this post


Link to post
Share on other sites
Advertisement

Interpreted languages and VM languages are not intended to replace their lower-level counterparts.

I don't see any particular reason why not.
 
Microsoft research wrote an experimental OS entirely in managed code, and there are rumours that they may be commercialising it.
 
If your OS isn't written in native code, why should anything be?

"Interpreted" languages are a bit different from "managed" languages though.
e.g.
C# is generally compiled down to an intermediate assembly language, which isn't interpreted, but JIT compiled to real assembly code. Likewise, PyPy and LuaJIT do the same.

Regular python or regular Lua though, they use a little VM loop that sits spins through the intermediate bytecode (or textual code), interpreting it as it goes, each time, which obviously has different performance characteristics.

However, managed C# code can also be compiled to native assembly completely ahead of time, which is what the Singularity OS did... :/
So actually, that OS is running on native code.
Which makes it the same as C, C++, etc, how they're generally compiled AOT to native code...
So what is actually being compared/contrasted here is a dangerous/vaguely specified language like C, compared to a more abstracted/safe language like C#.
There is no compiled vs interpreted argument present. The "managed" part in the name doesn't have anything to do with how it's compiled or run, it refers to the level of abstraction presented to the programmer -- one where they can't stomp memory willy nilly.
 
As for why you'd not write OS internals in Python or Lua; they don't have the capabilities that are required of low-level systems language, notably the ability to idiomatically (or at all) directly control memory layouts of your data easily on a byte-by-byte level, and for a trained user to be able to guess which native instructions their high-level code will compile down to. The memory access patterns of Lua in particular are absolutely horrid (even with LuaJIT to produce decent native assembly code, the memory access patterns still cause ridiculous amounts of cache misses and branch mispredictions, with no way to mitigate these problems), and the language lacks the tools for a programmer to address these low-level problems.

Having worked on a bunch of current gen console games written in Lua, there are huge and obvious problems that present themselves by the end of a project -- basically that your hardware requirements are higher... which is a problem when you've got fixed hardware.
The Lua VM is careless with memory layouts (with no tools to mitigate this, unlike in a low-level language. e.g. memory layouts can be controlled in endless ways in C++ while the code remains idiomatic) which leads to huge amounts of fragmentation, which leads to the game crashing if left on for 24 hours due to total-memory-free being high, but largest-unallocated-block being small, leading to tens of thousands of dollars wasted on a submission failure, followed by overtime for a programming team in crisis mode! Most VM languages that I've used are also extremely poorly designed when it comes to writing multi-core systems (which is a requirement on current hardware). I've seen more than one games company using python point at the global interpreter lock, and blame all their problems on it :/
Modern performance relies on good memory layouts and sensible multi-core usage, so when trying to write a HPC system, you need a language with good tools in those areas.

Most game code doesn't fall into the HPC realm, so any old language is sufficient -- preferably one that gives good programmer productivity.
 
C# does have the tools required of systems programming and HPC, but I don't consider them to be idiomatic: C#'s normally terse syntax becomes extremely verbose when you start trying to address low-level issues.
C on the other hand is just uniformly verbose ;)
 

What did they write the VM in?

It's quite common for a compiler to be written in the language that it compiles -- e.g. a C compiler written in C.
You need another compiler to initially build your code, but after that, your code can build itself smile.png
Singularity's C# compiler is actually written in C# biggrin.png Edited by Hodgman

Share this post


Link to post
Share on other sites

This last discussion reminds me of how every so often there will be always somebody trying to make a hobby OS that outright ditches native code and uses exclusively bytecode for the programs running on it (though generally it gets JIT'd at load time to prevent the massive performance loss). In theory, the advantages are being able to get away with heavy context switching (since you can be guaranteed the program is sandboxed and won't run unauthorized opcodes) and that since it's recompiled you can optimize the code to the current CPU (potentially making it faster than native code). Not sure if that actually ends up happening in practice.

Share this post


Link to post
Share on other sites

 

What did they write the VM in?

It's quite common for a compiler to be written in the language that it compiles -- e.g. a C compiler written in C.
You need another compiler to initially build your code, but after that, your code can build itself smile.png
Singularity's C# compiler is actually written in C# biggrin.png

 


Okay, I'll grant you that, since it's C#. What I mean is that the wiki page linked there says that the lowest level components are written in assembly and C++. Obviously the guys that designed the thing wouldn't argue that we should do away with C++, or that it has no place.

 

You made the relevant points, though. C# doesn't give the low-level control that's sometimes needed. What I'm trying to say here is that there's no reason to argue between language A and language B, especially when it's between languages like Python and C. They both have their place, and they can work together to give the best of both worlds. I have no problem with the Android method, where most things are in Java, but I do note that it's possible to write things in C++ and get better performance for critical sections of the code. It's relevant that this is something that happens a lot with video games.

In the end, I'd prefer if we could get a language that gives the control of C/C++ and the convenience of the higher level languages. I actually think it would be worthwhile to put some of the energy that's being spent on updating C++ into just developing a whole new language that gives a closer approximation to that blend. When I can get something that's very easy to code in and highly portable, but manages to be clear enough that I can predict what the native bytecode will look like, then I'll feel like we're making progress.

Share this post


Link to post
Share on other sites


every so often there will be always somebody trying to make a hobby OS that outright ditches native code and uses exclusively bytecode for the programs running on it... Not sure if that actually ends up happening in practice.

Android?

 

Sure, the kernel, the Dalvik VM, and some low-level functionality is written in C/C++, but pretty much everything else is in Java. Of course, game developers kvetched till they were given access to native code, but that's neither here nor there...

Share this post


Link to post
Share on other sites

 


every so often there will be always somebody trying to make a hobby OS that outright ditches native code and uses exclusively bytecode for the programs running on it... Not sure if that actually ends up happening in practice.

Android?

 

Sure, the kernel, the Dalvik VM, and some low-level functionality is written in C/C++, but pretty much everything else is in Java. Of course, game developers kvetched till they were given access to native code, but that's neither here nor there...

 

Well, it gets jitted, and cached usually...

 

Still, android is a damn slow platform most of the time, its very annoying. Someone went through and rebuilt the OS with pre-jitting etc. and it was significantly faster than the one that normally ships on phones.

 

Mind you, its not a problem with JIT or interpreted languages or anything like that. Its simply that JIT + embedded systems don't really go hand in hand very well (they work, but its usually not the best...idea).

Edited by Washu

Share this post


Link to post
Share on other sites

This last discussion reminds me of how every so often there will be always somebody trying to make a hobby OS that outright ditches native code and uses exclusively bytecode for the programs running on it (though generally it gets JIT'd at load time to prevent the massive performance loss).

I would absolutely love to see a useful top half ISR written in an interpreted language, even if it's JITted at runtime.

I guess the idea would be an OS kernel written in a native-targeted language like C, with a user runtime written in an interpreted language.

Share this post


Link to post
Share on other sites

In the end, I'd prefer if we could get a language that gives the control of C/C++ and the convenience of the higher level languages. I actually think it would be worthwhile to put some of the energy that's being spent on updating C++ into just developing a whole new language that gives a closer approximation to that blend. When I can get something that's very easy to code in and highly portable, but manages to be clear enough that I can predict what the native bytecode will look like, then I'll feel like we're making progress.

Actually, the Ada programming language predates the C++ language. It's lack of widespread success was due to the combined factors of tight centralized control (no vendor variants were allowed, so you could not get a subset for DOS) and the lack of underlying hardware support for some of its mandated features (for example, support for concurrently on processors that did not provide atomic operations). Oh, and it's incompatibility with existing libraries written in C.

It was like being handed the keys to a brand new Cadillac but all you had in your garage was a Vespa. Everyone rode their Vespa, and now people are debating about whether it's better to ride a Vespa or a Honda. The Cadillac is still out there, rusting away.

Share this post


Link to post
Share on other sites

This is no longer really relevant to beginners, so I'm going to close it (unless some other moderator wants to reopen it and move it to a more advanced forum).

Share this post


Link to post
Share on other sites

This topic is 1629 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Guest
This topic is now closed to further replies.
Sign in to follow this  

  • Advertisement