Yes. You can solve the problem numerically by value iteration, if you describe it as the optimization of the sum of future values of a utility function, with some exponential discount as you go further into the future. You will also need to quantize the space of states somehow to make this work.
Is it possible to phrase the problem in a way so that we can find the solution numerically, using only some sort of an error function?
I've thought of trying to write precisely what I just described, but I never found the time.
I don't have a good reference to value iteration. It is described on the second volume of this book, but the book is expensive and not particularly readable, so I won't recommend it. If anyone finds a good online reference, please post it.