Now I am back from France and it has been raining for days here in Wales, I have finally had a look at fixing some of the bugs in Godot engine which had stopped me making Android builds.
The biggest, and stop-ship bug has been a bug in the skinning. It turns out if you try and run a Godot game using OpenGLES2 with hardware that doesn't support float texture (a lot of hardware!), it crashes when you have a skinned mesh in the scene. Well not anymore!!
This was very painstaking to debug, which explains why no one had managed to address it for the months it has been a problem. If you want to debug Godot engine on desktop, you compile the engine, and just run it, and can place breakpoints and debug from QT Creator.
If however you want to debug the engine on an external device, it's a lot more complex. I don't know how reduz etc dealt with this but I resorted to the tried and true method of placing print statements throughout the rendering code. Everytime I wanted to debug I had make a change, compile the entire Android export templates, export the project from the IDE to the device, watch it crash, then read the print statements in adb logcat. A very slow and laborious business.
But after around a day of this I finally zeroed in on what was causing the crash. It wasn't as I had initially thought, been a GPU issue. It was in fact a regression issue. The bone data for the skinned meshes was being saved in main memory, and was required for skinning, but this was only happening on desktop, not on devices. What is more this issue only occurred in the backup codepath for when float textures was not supported by the hardware (which is most older hardware).
Once I had identified the crash, I hard coded in some values and was able to avoid the crash (but not render the skin correctly). From there I managed to modify it, practically a one line change that allowed the hardware tablets and phones to also store the bone data in main memory. This has fixed the problem!
I explained the problem and solution on the issues on github, however it was clear I'd have to *gasp* learn how to use git. I'd looked through a couple of tutorials and it looked inpenetrable. Luckily I found a good simple tutorial and slowly worked through bit by bit with help from the Godot documentation, and I had created a 'pull request', which gets reviewed for putting into the master branch. It is still there as now but it is pretty obvious it solves the issue so I'm sure we will get it into the engine soon, with modifications if necessary from the guys who mainly write the rendering code.
I also fixed another bug in the texture importing, and it is now pretty much good to go for running Godot on older Android devices (and also iOS).
I made some test APKs for others to test then I tried converting one of the demo programs successfully, then I have made a first version of my side scrolling shooter game which I made for the last gamedev challenge. This is working great on my Google Nexus 7 2012 tablet, before it would just crash if I tried to run anything. There is still some work to do, I need to get the shadows working if I can, and I need to make the buttons bigger and more polished for the user interface. But it works great, the frame rate is decent without optimization yet.
Very exciting both at the prospect of now being able to get all my previous Godot games working on tablets / phones and also at how useful this is going to be for others with the same problem.