State of the Snail - Debugging Hell

posted in SnailLife
Published May 20, 2017
Advertisement

It has been a while! SnailLife work has been moving at a snail's pace. Why? Because debugging the snail brain has turned into a highly demotivating, hellish endeavour. The snails make decisions and perform actions based on so much *crap*, that the simple log files I was using all long are just not cutting it anymore. The Laravel log for the original app has turned into 3 Laravel logs because I now have 3 independent apps (the front-end, BrainRunner, and BrainHub to coordinate all the brain runners). That then turned into individual log files per snail plus individual log files per brain task. And still making sense of why snails are choosing to behave in certain ways is nigh impossible - the brain *and* infrastructure issues I have been facing have been seemingly neverending.

Logging issues aside I have added some debug settings to help me narrow down issues. The BrainRunner app now has a snail debug config file which allows me to disable the following actions that would otherwise take place on each brain check:


  • growth
  • life check
  • idle action check
  • organ impact check
  • bodyweight check
  • movement
  • feelings of claustrophobia
  • feelings of loneliness

More settings will be added, such as disabling sensory/short term/long term memories, for example. The "idle action check" is where the bulk of the "magic" happens, that is the biggest black hole.

Adding options to disable certain functionality has helped, but the whole thing has been demotivating to say the least. I got into this to simulate snails, not work on debugging tools. And yes, of course I realize when dealing with a convoluted system like this I should have known what to expect. I did sort of expect this...I just chose to ignore it until the last minute to work on more fun things ;) After putting in a great deal of effort to remain organized and do things right in my projects at work, I've allowed good behaviour to fly out the window at home.

Anyway, I have now moved logging out of local log files and into Loggly. It's not done yet, but the main logs like Laravel logs from the BrainRunner and BrainHub, snail logs, and brain task logs are all in Loggly now (in addition to the local machine).

To send the default Laravel logs to Loggly for each app I added the `configureMonologUsing` method in bootstrap/app.php:
$app->configureMonologUsing(function ($monolog) use ($app){ $today = date("Y-m-d"); $logFileName = "logs/laravel-$today.log"; $monolog->pushHandler(new MonoStreamHandler(storage_path($logFileName), MonoLogger::INFO)); $monolog->pushHandler(new MonoStreamHandler(storage_path($logFileName), MonoLogger::WARNING)); $monolog->pushHandler(new MonoStreamHandler(storage_path($logFileName), MonoLogger::ERROR)); $monolog->pushHandler(new MonoStreamHandler(storage_path($logFileName), MonoLogger::CRITICAL)); if (CoreUtility::InternetOn()) { $envlabel = strtolower(env('APP_ENV')); $maintag = "brainrunner_$envlabel"; $envtag = "env_$envlabel"; $logglyString = CoreUtility::buildLogglyPushHandler(array($maintag, $envtag)); $monolog->pushHandler(new LogglyHandler($logglyString, MonoLogger::INFO)); $monolog->pushHandler(new LogglyHandler($logglyString, MonoLogger::WARNING)); $monolog->pushHandler(new LogglyHandler($logglyString, MonoLogger::ERROR)); $monolog->pushHandler(new LogglyHandler($logglyString, MonoLogger::CRITICAL)); } return $app;});

Aside from having all of my logs in a central place, Loggly lets me query the logs easily to get more relevant information. I use tags to distinguish between individual brain tasks, snails, apps, environments, etc.

But that is not enough. I feel to debug the brain in its current state having a visual representation of the brain would help a great deal. What originally made me think about this was my earlier experimentation with TensorFlow and TensorBoard. TensorBoard provides a visual of your TensorFlow graph and lets you interact with said visual to see how tensors flow between operations. The brain as I have it can also be represented as a set of nodes (each node representing a neuron), and I should be able to visualize the flow of inputs between neurons in a similar way. What if I could have a graph for every brain task that runs on every BrainRunner and see exactly what path each input took and why?

I think I will look into this next. As you can see I have not been doing much work on the actual *snails* here. But I think the return on investment in improved debugging tools will be worth it if I have any hope of getting any further with simulating a snail brain.

On another note it looks like ECAL 2017 ticket prices are out and I'd better start saving: https://project.inria.fr/ecal2017/registration/

2 likes 7 comments

Comments

JTippetts
It HAS been awhile. Glad to see that it's still in progress, and I hope you can get a handle on the debugging.
May 20, 2017 11:39 PM
Awoken

huh, this looks interesting. Is this a game? If so what's it all about?

May 21, 2017 07:14 PM
Liza Shulyayeva

huh, this looks interesting. Is this a game? If so what's it all about?

I wouldn't call it a game; it is intended to be more of an amateur browser based life simulation. You find, raise, race, and breed snails and manage a snail stable :)

May 21, 2017 08:27 PM
Awoken

hahaha, that is awesome! do you have this site up and running?

May 21, 2017 10:59 PM
Liza Shulyayeva

hahaha, that is awesome! do you have this site up and running?

Technically yes, but I have not yet made it available for people to start using (not that anyone would be clamouring to try it anyway, I think it would actually be pretty boring for anybody except me - the focus is very far from "fun"). But because I have a goal to hopefully attend the European Conference on Artificial Life this year, I'm using that as a bit of a vague deadline to have something a bit more interesting and functional to show by that time (whether that be the simulation itself or the brain visualization debugging tool I mentioned above). We'll see how it goes though; I've been working a lot lately so not sure how much time or energy I'll have to really get there in time.

May 21, 2017 11:16 PM
JTippetts
I would definitely want to play it when it's done. I've been interested in it even since it was called Gastropoda.
May 22, 2017 12:08 PM
Liza Shulyayeva

I would definitely want to play it when it's done. I've been interested in it even since it was called Gastropoda.

Thanks! Hoping to not disappoint :)

May 25, 2017 05:18 AM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Advertisement