Jump to content
  • Advertisement
Sign in to follow this  
Geoff the Medio

Unity Redirecting Python stderr and stdout to c++ functions

This topic is 3965 days old which is more than the 365 day threshold we allow for new replies. Please post a new topic.

If you intended to correct an error in the post then please contact us.

Recommended Posts

Similar to what was discussed here, I'm trying to redirect python stderr and stdout so that I can get the text of python error messages in the cpp code that's calling the embedded python script. I've been somewhat successful, but am getting some weird results with extra endlines. To simplify the problem, I've made a library using Boost Python that can be imported into Python that does similar redirection of stderr... Some cpp code to expose a simple "logging" library to Python:
void happy_write(const char* text) {
    std::cout << "captured stderr text:"<< text;
}

BOOST_PYTHON_MODULE(Logger)
{
    boost::python::def("write", happy_write);
}
and then, in the python interpreter:
>>> import Logger
>>> import sys
>>> sys.stderr = Logger
>>> blah
captured stderr text:Traceback (most recent call last):
captured stderr text:  File "<stdin>", line 1, in <module>
captured stderr text:NameErrorcaptured stderr text:: captured stderr text:name 'blah' is not definedcaptured stderr text:
>>>
For some reason, I get "captured stderr text:" on stdout twice for each line of text sent from Python to stderr, with the second copy being seemingly randomly inserted into the Python error message text. Any ideas why this might be? Thanks.

Share this post


Link to post
Share on other sites
Advertisement
You're assuming that Python always sends lines of text. In fact, I expect it's often just sending words or characters.

For example, I expect that last line is where it sends the following 4 things, one by one:
"NameError"
": "
"name 'blah' is not defined"
"\n"

Share this post


Link to post
Share on other sites
That seems to have been my problem.

I worked around it by some kludgy-seeming scanning of the input:
void happy_write(const char* text) {
bool flush = false;
for (int i = 0; i < 1000; ++i) {
if (text == '\0') break;
if (text == '\n') flush = true;
buffer += text;
}
if (flush) {
std::cout << "captured stderr text:" << buffer;
buffer = "";
}
}


Which seems to work, but also seems like it's probably a bad idea, and that there's an obvious-but-not-to-me better way.

Anyway, thanks.

Share this post


Link to post
Share on other sites
There's a few problems with your function now. First, it assumes that text will always be less than 1000 characters long. Second, it assumes that write() will only receive a carriage return at the end of the text and that there will only be one carriage return per call. Less of an issue is that it waits until a newline before writing to the stream, which could cause weird effects if python code is waiting for a response based on information that hasn't been written (more of an issue with stdout than stderr).

You might want to try something like this instead:

const char prompt[] = "captured stderr text: ";

void happy_write(const char* text) {
static bool first_call = true;
if (first_call) {
std::cout << prompt;
first_call = false;
}
while (*text) {
std::cout << *text;
if (*text == '\n') std::cout << prompt;
++text;
}
std::cout << std::flush;
}

(Untested code, and it's not the most efficient way to do things as it writes to std::cout a character at a time.)

Share this post


Link to post
Share on other sites
I didn't really assume redirected output wouldn't be longer than 1000 characters, but rather imposed the limit intentionally, in case a bad pointer was passed to a chunk of memory with no null characters for a long while, or other malicious or accidental misuse. I figured 1000 characters was "long enough" for most realisitic use cases.

Not writing output before the next newline isn't a problem, because the code I'm actually writing doesn't just redirect stderr to stdout. The posted code was a simplified example. The real code captures both stderr and stdout and redirects them to a logging system that outputs to a file. There's no way for anyone to see this output interactively while using the exposed library, so I don't see a need to ensure partial-lines are output as soon as possible... and I really don't want to output a partial line anyway, because the logger takes text one line at a time, and wouldn't handle partial lines properly. I forgot to include the declaration of std::string buffer in the previous source block, but it's purpose is to accumulate the null-terminated strings that Python outputs until an endline is reach, at which point the concatenated string is sent to the logger.

The issue about assuming only one newline per log string is valid though... I guess to handle this properly, I should stick the outputting line into the loop, instead of using a flush flag...
static const int MAX_SINGLE_CHUNK_TEXT_SIZE = 1000; 
static std::string log_buffer("");
void LogText(const char* text) {
for (int i = 0; i < MAX_SINGLE_CHUNK_TEXT_SIZE; ++i) {
if (text == '\0') break;
if (text == '\n' || i == MAX_SINGLE_CHUNK_TEXT_SIZE - 1) {
AIInterface::LogOutput(log_buffer);
log_buffer = "";
} else {
log_buffer += text;
}
}
}

Share this post


Link to post
Share on other sites
Sign in to follow this  

  • Advertisement
×

Important Information

By using GameDev.net, you agree to our community Guidelines, Terms of Use, and Privacy Policy.

GameDev.net is your game development community. Create an account for your GameDev Portfolio and participate in the largest developer community in the games industry.

Sign me up!