Jump to content

  • Log In with Google      Sign In   
  • Create Account


Member Since 02 Oct 2013
Online Last Active Today, 11:12 PM

#5316445 Reloading Python script (C API)

Posted by on Today, 06:17 AM

Thanks for the suggestions, this is exactly what I ended up doing. In case some unfortunate soul in the future has a similar problem to mine, here is exactly how I solved reloading scripts.


A quick disclaimer: This unfortunately only works for the functions you explicitly want to support for reloading. If your script imports other scripts, you're pretty much doomed with this approach. You should instead try to reset the interpreter.


I should also note that using a sub-interpreter (Py_NewInterpreter() and PyThreadState_Swap()) for every script you load won't solve this issue. There's a function in the python source code (see import.c) that maintains a global list of imported modules across all instances. There is no way to remove yourself from this list after your module is loaded.


In my situation, I wanted to expose two C++ callback functions to python. This is the class definition:

class ScriptInstance : public common::RefCounted
    ScriptInstance(const QString& fileName);

    void reload();

    void flush();
    void apply(ubox::Frame* frame);

    QString fileName_;
    previewer::PythonObject<> module_;
    previewer::PythonObject<> flushFunction_;
    previewer::PythonObject<> applyFunction_;

Some notes: PythonObject is a wrapper around PyObject*. It steals a reference on construction and calls Py_XDECREF() on destruction. flush() and apply() are the two functions I want to expose to python.


The first thing I do (before creating any ScriptInstance instances) is intialise the python interpreter:

bool PythonInterpreter::initialise()
        return true;

    // Add built-in ubox module before initialisation
    PyImport_AppendInittab("ubox", PyInit_ubox);

    static wchar_t programName[] = L"my app";
        defaultLogger.logError("Failed to initialise python interpreter");
        return false;

    // TODO this assumes the default location of the python interpreter

    // Import modules
    PyObject* tmp;
    tmp = PyImport_ImportModule("ubox"); // XXX This returns NULL for some reason? It loads nevertheless...
    tmp = PyImport_ImportModule("numpy");
    tmp = PyImport_ImportModule("scipy");

    return true;

Then, when I load a script, I create a new ScriptInstance and pass in the file name. In its constructor I create a new python module and register it under the name of the script (without the .py extension).

ScriptInstance::ScriptInstance(const QString& fileName) :
    // Need the global python interpreter to exist
    if(PythonInterpreter::initialise() == false)

     * Create a python module for this script instance, in which functions and
     * other data can be registered.
    module_ = PyModule_New(fileInfo.fileName().replace(".py", "").toStdString().c_str());
    PyModule_AddStringConstant(module_, "__file__", "");



The reload() function is responsible for loading the file, compiling it, and extracting the relevant functions flush() and apply():

void ScriptInstance::reload()
    QFile file(fileName_);
    QByteArray bytes = file.readAll();

    PythonObject<> compiled = Py_CompileString(bytes.data(), fileName_.toLatin1().data(), Py_file_input);
    // error handling omitted

    // these are all borrowed references
    PyObject* main = PyImport_AddModule("__main__");
    PyObject* globals = PyModule_GetDict(main);
    PyObject* locals = PyModule_GetDict(module_);

    PythonObject<> eval = PyEval_EvalCode(compiled, globals, locals);
    // error handling omitted

     * Get the two python functions flush() and apply() and make sure they're
     * correct and can be used.
    flushFunction_ = PyObject_GetAttrString(module_, "flush");
    applyFunction_ = PyObject_GetAttrString(module_, "apply");
    // error handling omitted

PyImport_AddModule("__main__") will return a borrowed reference to the __main__ module, which in turn contains a dictionary that contains all of the imported modules and functions thus far. This dictionary must be passed to PyEval_EvalCode() when you're evaluating your script, otherwise basic functions such as print() won't be recognised.


The locals dictionary is retrieved from the module that was previously created in the constructor and it will hold all of the data that is local to our module (i.e. it will store the flush() and apply() function objects).


After evaluating the code with the proper dictionary objects, you will be able to retrieve the function objects from the module with PyObject_GetAttrString(module_, "func").

#5315574 C++ : Extend class in dll

Posted by on 17 October 2016 - 12:39 PM

Typically this is achieved through some form of dynamic type information, be it using templates + typeid() or some other form of unique type identification


The basic idea consists of two classes. A "Manager" and a "Factory". The manager stores a collection of factories and associated type identification, and the factories know how to instantiate the type you need.


The following is a complete example using std::type_info. Note that you will need C++11 because of hash_code(). If you must use c++03 then you can replace hash_code() with name() and run that through your own hash function (or change the type in the container from size_t to std::string).

#include <typeinfo>
#include <map>
#include <string>
#include <iostream>

class Factory {
    virtual void* Create() = 0;

template <class T>
class FactoryImpl : public Factory
    virtual void* Create() override
        return new T();

class Manager
    template <class T>
    void RegisterObjectFactory()
        factoryMap_[typeid(T).hash_code()] = new FactoryImpl<T>();

    template <class T>
    T* CreateObject() const
        auto factory = factoryMap_.find(typeid(T).hash_code());
        if(factory == factoryMap_.end())
            return NULL;
        return static_cast<T*>(factory->second->Create());

    template <class T>
    void DestroyObject(T* object)
        delete object;

    std::map<size_t, Factory*> factoryMap_;

// ----------------------------------------------------------------------------
// Example usage
// ----------------------------------------------------------------------------
class Test
    void Greet() { std::cout << "it works!" << std::endl; }

int main()
    Manager manager;
    Test* test = manager.CreateObject<Test>();



One pitfall to look out for with factories is to make sure you delete the object in the same DLL you created it. This is why Manager::DestroyObject() is necessary. If you were to convert it from raw pointers to std::shared_ptr then this would no longer be necessary to do.


With this method you would need to pass the manager instance to your plugin DLLs when they start (typically achieved by defining your own start_plugin() function). Then your plugins can call Manager::RegisterObjectFactory<Whatever>() to register the types they require. This will enable the core application to start instantiating types that weren't originally there when compiling.


You might consider extending the example to make it possible to pass the actual class name to the manager, and not just the has code. For example:


instead of


For this you would need to store typeid(T).name() in the corresponding factory class.

#5314617 Coding-Style Poll

Posted by on 10 October 2016 - 11:52 PM

I can live with the prefixes C, I, m_, g_, etc. I can live with different brace styles, I can live with the 80-character-per-line limitation, but the one thing that gets my jimmies rustled is when I encounter the following for literally every function.
// *-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_*-*_
////             SetValue();
////             Sets the value.
////             value
////                       The value to set.
// _--_--_---_--_--_--_--____-__-___-_----_--_--
//// AUTHOR: Autism McTrashWizard
//// \\\\ ==== //// \\\\ ====//// \\\\ ====//// \\\\ ====//// \\\\ ====//// \\\\ ====//// \\
void SetValue(float value)
    g_value = value;

#5308086 trek game

Posted by on 26 August 2016 - 01:44 PM

Just call it Star Shrek and make everyone an ogre

#5304866 Ide For Linux

Posted by on 09 August 2016 - 05:23 AM

Anyway, let's start with proper shell integration. Say you have table in your source file with names that needs sorting. In vim: Mark first and last line, then ":'a, 'b ! sort" (run 'sort' command, using as input the lines between and including marker a and marker b. Replace input with the output of the sort command).



Yeah perhaps this should be a separate topic. But to answer your question on how that would work in KDevelop: Highlight the lines, press F7 and type "sort" (or if you're using Vi input mode, highlight lines, enter ":sort").


KDevelop uses Kate's text editor, which tries to emulate most of vim's features, so most of the common commands you'd use in vim are available in KDevelop:



Qt Creator supports all of those.


Thanks! I didn't know about this.

#5304669 A Lightweight 2D Game Framework Doesn't Seem To Exist

Posted by on 08 August 2016 - 08:37 AM

Urho3D (despite the name) is a decent 2D game engine/framework. It ships with Box2D physics, it supports animation, shading, etc. and it compiles not only to desktop platforms but also to Android, iOS and the web (using emscripten).


As far as "lightweight" goes, it compiles to an approximately ~5MiB library and takes maybe 4-5 minutes to compile on my i5-2440m. So I'd say when compared to something like Ogre3D it is fairly lightweight.

#5304662 Ide For Linux

Posted by on 08 August 2016 - 08:19 AM

TL;DR I recommend KDevelop if your project is CMake based.


To those recommending QtCreator, it's a fairly decent IDE if you're developing Qt applications, but I don't recommend it for anything other than that. One of the things I disliked about it a lot is how it handles file navigation. In KDevelop I can toggle between header/source files with ctrl+shfit+c. I can "alt tab" between the two last files with ctrl+shift. I can press ctrl+alt+o and quick jump to a file (or by using any of those other shortcuts that index function names or class names). These are things QtCreator doesn't offer and it's annoying to have to search for your files in the file tree every time you want to change files.


Why KDevelop?

  1. Integrates with Vi (if you want).
  2. Integrates with CMake. This has proven to work remarkably well, even for highly complex CMakeLists.txt.
  3. Integrates with git. A cool example of this is you can press alt+a and it will use "git blame" to show who edited which line of code.
  4. KDevelop has some of the richest syntax highlighting I've ever seen in any IDE. After programming with it for a while you'll start to see subtle things, like the "++" in "myObject++" changing to purple instead of white because myObject has overloaded operator++. There are *loads* of little things that will help you understand code better.
  5. Lots of helpful navigation shortcuts. As mentioned, you can jump between header/source file pairs, you can jump to declarations/definitions, you can jump to files.
  6. Lots of auto-completion. One of the cooler examples is you can write "it = thing.begin()", then hit alt+1 and the IDE will auto-complete "std::vector<foo>::iterator" for you.

The list goes on and on. Definitely give it a shot!


Personally, I've been developing on Linux for over 20 years and never felt the need to be limited by an IDE, but 

to each their own I guess.



I love this. "Limited" by an IDE. I'd be interested to hear what you consider limiting factors of an IDE?

#5295997 Wannabe

Posted by on 10 June 2016 - 11:29 AM

It's never too late to pursue anything. There are a million ways to get into game programming, you just have to find one that is easy enough for beginners and offers to you the freedom and features you expect to have. Some immediate products that come to mind are: DarkBASIC, Unity, pygame, etc.


I personally started making games with DarkBASIC when I was about 14. It's a very simple language to pick up quickly and the results are immediate. It's also a lot of fun to play around in. I wrote games for about 5 years using only that product.

#5295995 Do you keep a collection of handles in your game for your components

Posted by on 10 June 2016 - 11:21 AM

This is perhaps a shameless plug, but when I first learned about ECS I tried writing my own framework, and I implemented a way for the user to specify execution dependencies:



In my project, the dependency tree is resolved and flattened into a linear ordered list of systems to execute. Thus, when calling world.update(), I only have to iterate through the list and call system->update(). This sounds like the exact thing you need. It might also be a possibility to flatten the dependency tree into multiple lists of systems, where each list contains systems that can be executed in parallel. Just a thought.

#5294975 What is the output?

Posted by on 04 June 2016 - 10:15 AM

Just a tip to ensure the success of your future career: Don't ever write code like that.

#5293976 Animated Cursor: how get frame delay?

Posted by on 28 May 2016 - 04:48 PM

I don't understand this.


First of all, your code won't even compile because "i" is undeclared.


Your while loop will probably never exit, so your program will just get stuck once you call GetAniFrameCount().

Even if this somehow worked, it would not return the frame rate of your app. You'd just be timing how fast DrawIconEx() is, which has nothing to do with frame rate.


And lastly, Windows applications don't have a "frame rate" per se. They are only redrawn when they need to be redrawn (e.g. when you rescale the window), so the concept of "frame rate" doesn't exist.

#5293962 Some people just want to watch the world burn

Posted by on 28 May 2016 - 02:29 PM

x can go to zero even faster in opposite direction

int x = 10;

while( 0 <---- x )
    printf("%d ", x);
8 6 4 2

You can control speed with an arrow!

int x = 100;

while( 0 <-------------------- x )
    printf("%d ", x);
90 80 70 60 50 40 30 20 10

Sad thing is, this actually compiles.  :unsure:

#5265522 Why didn't somebody tell me?

Posted by on 08 December 2015 - 07:52 PM

Cats. They are not dogs.

1) Dogs have four legs
2) Cats have four legs
3) Therefore, cats are dogs.

Checkmate atheists.

#5262674 Question about type, and displaying the bits of a char

Posted by on 18 November 2015 - 07:32 PM

God damn it, I had a super long post typed up and chrome crashed because I hit CTRL+S.
You don't really see bitfields in desktop applications because there's no need to conserve memory. You'll find them in code that makes heavy use if bit masking (crypto?) or memory conservation (networking code?). I program micro controllers for my day job and we make heavy use of bitfields.

[EDIT] Just to be clear and as mentioned in the coming posts: This is not portable.
Here's an example. Imagine you had to pack a "move" instruction of a chess piece in a game of chess into as little space as possible. Reasons being you want to transmit the move over a network and save space. You could encode this by using a "to" and "from" coordinate. Seeing as a chess board is 8x8, a coordinate can be packed into 6 bits. You could write:
u16 move = 0;
move |= (current_x << 0);
move |= (current_y << 3);
move |= (new_x << 6);
move |= (new_y << 9);
/* 4 bits reserved for whatever */
Using bitfields makes this much more readable:
struct chess_move_t {
    union {
        struct {
            unsigned current_x : 3;
            unsigned current_y : 3;
            unsigned new_x : 3;
            unsigned new_y : 3;
            unsigned :4; /* unused */
        u16 data;
The following code does the same thing as the first example.
struct chess_move_t move;
move.current_x = current_x;
move.current_y = current_y;
move.new_x = new_x;
move.new_y = new_y;

Here's a real world example of some micro controller code, just in case you were wondering.
void timer_init(void)
     * Target interrupt frequency is 100Hz
     * Fcy = 7.37 * 65 / 8 = 59.88125 MHz
     * Prescale 1:64 ~ 936 kHz
     * Using 16-bit timer type B: count to 9356 for 100 Hz
     * We'll be using a timer type B, specifically timer 4, so we don't
     * clash with the timer required for ADC conversions.
     * Notes on config:
     *  + Clock source select by default is Fosc / 2
     *  + Default mode is 16-bit mode
    T4CONbits.TON = 0;      /* disable timer during config */
    T4CONbits.TCKPS = 0x02; /* prescale 1:64 */
    PR4 = 9356;             /* period match, divide the 936 kHz by 9356 to 
                             * reach 10ms */
    IFS1bits.T4IF = 0;      /* clear interrupt flag */
    IEC1bits.T4IE = 1;      /* enable timer 4 interrupts */
    T4CONbits.TON = 1;      /* start timer */
The relevant bitfield declarations are the following (this is found in a header file provided by Microchip)
#define T4CON T4CON
extern volatile unsigned int  T4CON __attribute__((__sfr__));
__extension__ typedef struct tagT4CONBITS {
  union {
    struct {
      unsigned :1;
      unsigned TCS:1;
      unsigned :1;
      unsigned T32:1;
      unsigned TCKPS:2;
      unsigned TGATE:1;
      unsigned :6;
      unsigned TSIDL:1;
      unsigned :1;
      unsigned TON:1;
    struct {
      unsigned :4;
      unsigned TCKPS0:1;
      unsigned TCKPS1:1;
extern volatile T4CONBITS T4CONbits __attribute__((__sfr__));

#5260334 Idea about game engine for Linux

Posted by on 03 November 2015 - 09:57 AM

Why not help improve an existing engine rather than trying to create yet another engine? Urho3D comes to mind...