Python C API: Undefined functions in module

Started by
1 comment, last by TheComet 7 years, 6 months ago

def test():
    pass

def apply(frame):
    test()

This is the file I compile and evaluate using the following code (omitting error handling for readability's sake):


void ScriptInstance::reload()
{
    QFile file(fileName_);

    file.open(QIODevice::ReadOnly);
    QByteArray bytes = file.readAll();
    file.close();

    // these are all borrowed references
    PyObject* main = PyImport_AddModule("__main__");
    PyObject* globals = PyModule_GetDict(main);
    PyObject* locals = PyModule_GetDict(module_);

    PythonObject<> compiled = Py_CompileString(bytes.data(), fileName_.toLatin1().data(), Py_file_input);
    PythonObject<> eval = PyEval_EvalCode(compiled.get(), globals, locals);

    applyFunction_ = PyObject_GetAttrString(module_, "apply");
}

module_ was previously created using PyModule_New().

The problem is when I call applyFunction_ I get the error:



call to apply() failed: name 'test' not defined

What am I doing wrong here?

Here's the code I use to call the function:


PyObject* args = PyTuple_New(0);
PyObject_CallObject(applyFunction_, args);
"I would try to find halo source code by bungie best fps engine ever created, u see why call of duty loses speed due to its detail." -- GettingNifty
Advertisement

I tried this (without caring at all about reference counts), and got a compiler warning.


// gcc -I/usr/include/python2.7 x.c -lpython2.7
//
#include "Python.h"

const char *text = "def test():\n"
"    pass\n"
"\n"
"def apply(frame):\n"
"    test()\n";

int main(void)
{
    PyObject *module_ = PyModule_New("mymodule");

    // these are all borrowed references
    PyObject* main = PyImport_AddModule("__main__");
    PyObject* globals = PyModule_GetDict(main);
    PyObject* locals = PyModule_GetDict(module_);

    PyObject *compiled = Py_CompileString(text, "text.py", Py_file_input);
    PyObject *eval = PyEval_EvalCode(compiled, globals, locals);

    PyObject *applyFunction_ = PyObject_GetAttrString(module_, "apply");
    PyObject* args = PyTuple_New(0);
    PyObject_CallObject(applyFunction_, args);
    return 0;
}

x.c:19:38: warning: passing argument 1 of 'PyEval_EvalCode' from incompatible pointer type [-Wincompatible-pointer-types]
     PyObject *eval = PyEval_EvalCode(compiled, globals, locals);
                                      ^
In file included from /usr/include/python2.7/Python.h:136:0,
                 from x.c:1:
/usr/include/python2.7/eval.h:10:24: note: expected 'PyCodeObject * {aka struct <anonymous> *}' but argument is of type 'PyObject * {aka struct _object *}'
 PyAPI_FUNC(PyObject *) PyEval_EvalCode(PyCodeObject *, PyObject *, PyObject *);

Am I doing something wrong, or are you providing an incorrect object type?

Edit, changing the type of "compiled" to PyCodeObject gives a warning at that line:


In file included from /usr/include/python2.7/Python.h:127:0,
                 from x.c:3:
x.c: In function 'main':
/usr/include/python2.7/pythonrun.h:65:37: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
 #define Py_CompileString(str, p, s) Py_CompileStringFlags(str, p, s, NULL)
                                     ^
x.c:20:30: note: in expansion of macro 'Py_CompileString'
     PyCodeObject *compiled = Py_CompileString(text, "text.py", Py_file_input);

The API changed, in 3.x the function PyEval_EvalCode() takes a PyObject* instead of a PyCodeObject*. Probably because someone decided that maybe code objects aren't the only type of object that can be evaluated? Who knows. Py_CompileString() will return a PyCodeObject* so you're safe to cast it.

I was able to solve it with some help from the #python IRC channel: Apparently, passing the "globals" dict from the __main__ module to PyEval_EvalCode() will cause whatever functions are in the code object to be registered globally to the __main__ module, but will still be unavailable to the local module. The documentation is extremely sparse and I have to admit I don't understand why the following code works. What exactly happens with "globals" and "locals" in the call to PyEval_EvalCode(compiled, globals, locals)?

Anyway, here's the code that works:


void ScriptInstance::reload()
{    /*
     * Need to clear everything that is currently in the module's dictionary,
     * so it's not possible to accidentally call old functions that no longer
     * exist.
     *
     * In order for the builtin functions to be defined, copy the __main__
     * module's dictionary into the new module's local dictionary
     */
    PyObject* locals = PyModule_GetDict(module_);
    PyObject* main = PyImport_AddModule("__main__");
    PyObject* globals = PyModule_GetDict(main);
    PyDict_Clear(locals);
    PyModule_AddStringConstant(module_, "__file__", "");
    PyDict_Merge(locals, globals, 0);


    /*
     * Evaluate the code into this module's local dictionary.
     */
    PythonObject<> eval = PyEval_EvalCode(compiled.get(), locals, locals);


    flushFunction_ = PyObject_GetAttrString(module_, "flush");
}

Explanation:

Upon reloading, I have to clear the module's local dictionary of all items, to make sure it's impossible to call functions that are no longer defined in the script.

This isn't enough, because now the dictionary is empty, which means any scripts that are evaluated using this dictionary won't have access to any built-in commands like "print()" or "import". To fix this, I get the dictionary from __main__ (which defines all of these things) and merge it with the local dictionary.

Once that's done, I pass "local" twice to PyEval_EvalCode(). That fixes it.

Like I said: I don't fully understand why this works and would appreciate some explanation.

"I would try to find halo source code by bungie best fps engine ever created, u see why call of duty loses speed due to its detail." -- GettingNifty

This topic is closed to new replies.

Advertisement