# Unity Run-time compiled C in python scripts. Works.

This topic is 3083 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

## Recommended Posts

I've written a module that uses ctypes and Tiny C to compile C source code from within a python program and makes it callable within python. Is this something the community would be interested in having? If so I can put up a page and post the code on my gamedev.net space for all to have. The application I'm currently working on has some performance critical sections where python just wasn't cutting it. It's easy to extend python with C using ctypes, but it's not very convenient when you only need to write very small amounts of performance-critical code in C. I found a few libraries that would suit this purpose one way or another, but I felt that they'd be too cumbersome for what I am doing. I did run across a very old, seemingly abandoned project called "C in Python" that used the Tiny C compiler and ctypes to compile C code from strings at run-time. It didn't work, but I thought it was a great idea, so I've rewritten it so that it works with the latest TCC and python 2.6. For now I've only tested it on linux 32 bit. It would need some modifications to work on Windows, but it shouldn't be difficult to do. It would probably work on Mac OS X also, but I couldn't get Tiny C to build on my mac. Here's a working example:
import ctypes
import compiler

# Defining functions that will be exported to python from the C source.
# compile expects a list of two-tuples containing first the name
# of the exported function in the C source code and the ctypes prototype
exports = (('fib', ctypes.CFUNCTYPE(ctypes.c_int, ctypes.c_int)),
('test_callback', ctypes.CFUNCTYPE(None)))

# defining a function to call back into from the C code
def callback():
print "Calling back into python WORKS!!!!"

# You can optionally make your python code available to the C program.
# compile expects a list of three-tuples containg first the name
# to use in the C source, the ctypes prototype, and the actual
# python function object to be called.
imports = (('callback', ctypes.CFUNCTYPE(None), callback),)

# The C source code
code = """
int fib(int n)
{
if (n <= 2)
return 1;
else
return fib(n-1) + fib(n-2);
}

void test_callback() {
callback();
}
"""

# compile returns a dictionary of callables. imports is optional.
lib = compiler.compile(exports, code, imports)

print "c fib result:", lib['fib'](30)
print "should see the same text repeated twice:"
callback()
lib['test_callback']()

Tiny C also supports importing functions from a dynamic link library so I might implement that also, but I haven't needed it yet. [Edited by - smr on December 1, 2009 2:40:26 PM]

##### Share on other sites
that looks awesome

Thanks!

##### Share on other sites
Cool stuff! I'd love to see more.

##### Share on other sites
I think I'll make this available once I get it working under Windows and Mac OS X. I'm also considering automatically grabbing the exported functions by analyzing the C source code. More to come later.

##### Share on other sites
Quote:
 Original post by smrI'm also considering automatically grabbing the exported functions by analyzing the C source code.
Doesn't ctypes already build a list of functions from the DLL/so/dylib symbol table?

##### Share on other sites
Quote:
Original post by swiftcoder
Quote:
 Original post by smrI'm also considering automatically grabbing the exported functions by analyzing the C source code.
Doesn't ctypes already build a list of functions from the DLL/so/dylib symbol table?

It does, but this compiles into a chunk of memory. There's no library to load, so I have to grab the symbols and define the function arguments myself.

##### Share on other sites
Quote:
Original post by smr
Quote:
 Doesn't ctypes already build a list of functions from the DLL/so/dylib symbol table?
It does, but this compiles into a chunk of memory. There's no library to load, so I have to grab the symbols and define the function arguments myself.
Hmm, that seems efficient, but I am not sure how portable it is. How are you making sure the generated machine code ends up in an executable page?

My original thought on how to do this was to mmap a file, compile into it, and then load that with ctypes.

##### Share on other sites
Quote:
Original post by swiftcoder
Quote:
Original post by smr
Quote:
 Doesn't ctypes already build a list of functions from the DLL/so/dylib symbol table?
It does, but this compiles into a chunk of memory. There's no library to load, so I have to grab the symbols and define the function arguments myself.
Hmm, that seems efficient, but I am not sure how portable it is. How are you making sure the generated machine code ends up in an executable page?

My original thought on how to do this was to mmap a file, compile into it, and then load that with ctypes.

Aww, shucks... I guess I was so excited about the idea of this that I hadn't thought that part through. I have to give the tiny c library a place to put the code after it's compiled, which I'm doing now by allocating an array in python of the correct size then passing tiny c the address of its buffer. I suppose I could find a way to ensurr that I get a chunk in an executable page and also properly aligned, but that would be more work than just compiling into a mmap'ed file.

##### Share on other sites
Quote:
 Original post by smrI suppose I could find a way to ensurr that I get a chunk in an executable page and also properly aligned, but that would be more work than just compiling into a mmap'ed file.
In theory at least, you can configure it so that the mmap'ed file shouldn't ever touch the physical disk, so the performance difference should be very much. Plus, you get ctypes to do the legwork of symbol table passing.

I would love to take a look when you have a working implementation - might fit right in with my planet renderer...

• 33
• 15
• 23
• 10
• 19
• ### Similar Content

• Ok, firstly, Hi.

This is my first post on this forum. I am an Indie Dev making my first game so bear with me when I say dumb stuff, I'm on a huge learning curve.

My first question is about inventory systems for unity. I am trying to make a survival type game with crafting. I have purchased Inventory manager pro by devdog from the unity asset store and it seems like a pretty powerful assett but for an intermediate coder its a little tough to use.  I'm beginning to wonder if it was the right purchase.
So my question is.... does anyone have any experience of inventory plugins / systems for unity and can anyone reccomend a system to me?
It needs to have the following: Loot system, crafting system, character sheet, blueprint system,  character stats system. Ideally with as little coding as possible.

Thanks

• I've got a bug with my brick breaker style game. The bricks move down one line at a time ever 1.5 seconds. What appears to be happening is occasionally the ball will be just about to hit the brick when the brick moves down a line, and now the ball is behind it. I'm not sure how to fix this. I have two ideas but I'm not sure of implementation. 1 solution would be to check where they were and where they are going to be before rendering the frame. Then if they crossed paths, then register the brick as hit. Solution 2 would be change how the bricks move. I could maybe slide them down line by line, instead of a jump down. I'm not sure of this will fix the issue or not. Any ideas?
• By Pixeye
I wrote an extension for unity inspector that allows to group/fold variables.
Available on github  , cheers!

• I'M interested in programming tools (For animation, UI, etc). Can anyone suggest me the resources where I can start learning or which technologies I need achive it.

Thanks,
Rakshit
• By Paszq
Group photo of some of the characters and creatures currently living in Arpago