Boost.Python as engine's scripting language

After further tinkering I'm feeling like I have a good starting location. I have the BaseBehavior inheritable class exported to the Python side for utilization, and I have my cpp side calling the particular test script that contains a class inheriting from the BaseBehavior and executing it.

However, now I am trying to figure out how to collect / extract that particular class after the script has been executed on the cpp side. This is what I'm trying to do as a simple example, but I'm not sure how to specify what the result to look for is, besides specifically. All I know is that it inherits from the BaseBehavior class.

This is my test.py script:

import Engine

print("Hello?")

class TestPiece(Engine.BaseBehavior):
  def OnUpdate(self):
    print("Hello From TestPiece?")
    
test = TestPiece()
test.OnUpdate()

This is my cpp side locating the "test" object from the main_namespace BP::dict, which works.

try
  {
    PyImport_AppendInittab("Engine", &PyInit_Engine);
    Py_Initialize();

    // '__main__' is the name of the scope in which top-level code executes
    BP::object main_module = BP::import("__main__");

    // we create a dictionary object for the __main__ module's namespace
    // m.x = 1 is equivalent to m.__dict__["x"] = 1
    // Special read-only attribute: __dict__ is the modules namespace as a dictionary object
    BP::dict main_namespace = BP::extract<BP::dict>(main_module.attr("__dict__"));

    auto result = BP::exec_file("../Debug/test.py", main_namespace, main_namespace);
    BaseBehavior& base = BP::extract<BaseBehavior&>(main_namespace["test"]);

    base.OnUpdate();
  }
  catch (...)
  {
    PyObject *ptype, *pvalue, *ptraceback;
    PyErr_Fetch(&ptype, &pvalue, &ptraceback);

    if (pvalue != nullptr)
    {
      std::string error = BP::extract<std::string>(pvalue);
      std::cout << error << std::endl;
    }
  }

How would I make it so it retrieves the TestPiece class for execution on the cpp side, without having to manually create an object on the python side for the system to know what to search for? Upon further thought, I definitely do need a way to associate the script to a particular object such as it searching for the initial "test" object with the BP::dict object, I just need a way for the system to be able to identify all those objects, creating the initial object at the end of the python file doesn't seem right.

Creating a C++ base class for your Python code to derive from only really makes sense if you plan to have both C++ and Python implementations of that class. If you only have Python implementations then the inheritance scaffolding is just extra overhead. Just store a boost::python::object and call methods on that directly.

If you only have one python source file, then using exec_file in the main namespace would probably work. However, if you have multiple source files, then global name bindings could get very screwy. For example. you may try calling one function in your code and end up calling a different one with the same name in a different Python file. This can be especially annoying if someone defines a global with the name of a built-in. I'd give some serious thought to putting your source files on sys.path and just using import on them.

To avoid creating a global variable just for your C++ code to find, you can access the Python classes and functions inside the the script with the Python names.

BP::object test_module = BP::import("test");
BP::object test_class = test_module.attr("TestPiece")
BP::object test_object = test_class()

Thank you for the insight, I definitely like the idea of being able to separate each piece into their own individual modules, there will definitely be the possibility of variables and function of the same names between each of my python files.

I'm currently trying to append my working directory, the location where my python files are located on my sys.path. However I am encountering some problems.

I first tried:

boost::filesystem::path workingDir = boost::filesystem::absolute("../Debug").normalize();
PyObject* sysPath = PySys_GetObject("path");
PyList_Insert(sysPath, 0, PyBytes_FromString(workingDir.string().c_str()));

Which I believe it does work, however, when I get down to my :

BP::object test_module = BP::import("test");
BP::object test_class  = test_module.attr("TestPiece");
BP::object test_object = test_class();

BP::object test_module throws an exception, a "TypeError" with the message : 'str' does not support the buffer interface. Interestingly, if I don't change the sys.path it seems to successfully import test, I'm not sure if it's the right test module, but then it throws an exception on BP::object test_class with the message : 'module' object has no attribute 'TestPiece', which from there I'm concluding it's not the right test that's being imported.

I then tried to manually append the sys.path by doing :

std::string path;
path += "import sys\nsys.path.append('";
path += workingDir.string();
path += "')";
BP::exec(path.c_str(), main_namespace, main_namespace);

But that unfortunately causes the same error to output for the BP::object test_module.

On a side note, once this is imported successfully and the BP::object test_class is found, how do I call / use the particular functions / data. I'm thinking something like this ? :

BP::object test_module = BP::import("test");
BP::object test_class  = test_module.attr("TestPiece");
BP::object test_object = test_class();
test_object.attr("OnUpdate")(); // call the OnUpdate function with operator()

----------
[ Edit 0 ]

I just tried :

PyObject* sysPath = PySys_GetObject("path");
PyList_Append(sysPath, PyBytes_FromString("./../Debug"));

I'm not sure if it works again, but, it doesn't throw an exception at the BP::object test_module, it does however throw the BP::object test_class exception as stated above.

[ Edit 1 ]

Okay, this is definitely interesting stuff. I just changed my python file to be "test1234" instead of "test", maybe there is a conflict in the python root path for a "test" module. However, once I changed the file name to "test1234" it works as expected. Also, the :

test_object.attr("OnUpdate")()

Does indeed call the function as expected. Very cool. Any further insight on these situations would be greatly appreciated. Another thing I'm still concerned about is if I have multiple BP::import statements in a row for each module / file. I need to make sure those don't cause conflicts amongst themselves.