So I've coded up a version that I *think* will work on a linux system with NX. Unfortunately I don't have one of these so I was wondering if someone with a linux machine with NX would test for me?

On Windows you should use VirtualAlloc and VirtualProtect to make sure the memory you allocate is executable (msdn)

Quote:Original post by Vectorian
On Windows you should use VirtualAlloc and VirtualProtect to make sure the memory you allocate is executable (msdn)

Already working on it ;)

Quote:Original post by smr

Quote:Original post by Vectorian
On Windows you should use VirtualAlloc and VirtualProtect to make sure the memory you allocate is executable (msdn)

Already working on it ;)

OK, I've got it working under windows. I need to clean it up a bit before I can release it. I think I am setting up the execute access properly, but I don't have a machine with NX, so I can't be sure.

I'll release what I have as soon as I can get the setup script to work properly on linux and windows. This is my first go at creating a setup script. The package will build and everything works on Windows, but on linux I'm having some problems getting the libraries to go in the right places.

I've uploaded the python module, dynamic libraries for Windows and Linux and example program to my gdnet page. This isn't an installable setup package. I couldn't get that to work on Linux. The setup script on linux copies the libraries to my package's dist-packages folder, but I can't seem to figure out a portable way to get the path to these libraries unless I hard code them. I'll work on this later.

For now, you can copy the tcc.py module and the lib folder into your application's directory and everything should work on Linux and Windows. This does not work on Mac OS X. It shouldn't be difficult to get working for osx, I just couldn't get Tiny C to build on my Mac.

This module should work on systems using NX protection. I can't test it myself because I don't have a system with NX. Would someone with NX mind testing this for me and letting me know whether it works or not? To test, first try running example.py without any modifications. If that works, set tcc.TCC_TEST_NX = True directly after importing the tcc module. If you receive a page fault with TCC_TEST_NX set true but no page fault when reset False, then then everything is working the way it should.

Quote:Original post by wodinoneeye


My question would be :

What use would this be put to -- on-the-fly C lisp-like Lambda functions ???

It would have to be some kind of dynamicly defined heavy number-crunching (code reused alot to offset the overhead of the initial code-building and compilation stage, over a less efficient code interpretor (possibly a data option control or byte code system working for a limited parameterizing scheme).


If its something that changes constantly (code being rebuilt frequently) the overhead might make the non-native compilation more efficient.

Since the code is building C code it has to have finite/limited logic to do that operation (unless the code is being made from use input fragments, ie- equations) and even that could be a wizard that build bytecode or activated precanned flavors in static code. Again unless if is build once and run ALOT, so to amortize the problem->C code->native code conversion overhead.

The C code is intended to be only built once and the Tiny C compiler is very fast. Have a look at the TCC website to see some metrics. And you are correct, there is quite a bit of overhead when marshaling across the C / python border, so you should always profile before you decide to convert some algorithm from python to C. And when you do replace some python code with C, you should think hard about how you'll marshal the data and how often. In many circumstances it will be more efficient to wad up a big chunk of data to be processed with C code once rather than iterating over it in python, crossing the python-C barrier for each item.

Though building this module has been a fun experience, I'll admit that I'm not entirely sure how useful it will be. There's no debug support for the C code, so for anything non-trivial you'll probably have to code and debug with standard C tools anyway then copy-paste the code into python. Since you've already gone through all of that trouble, you might as well just compile your code as a dynamic library and load it with ctypes. Any other compiler will probably produce better code than TCC because TCC only performs a handful of optimizations, trading fast code for fast compiling.