I did write another post yesterday but decided to leave it because I was sure someone with more (a little) knowledge of this would pop up, anyway a bit more discussion because I am interested to know the answer myself:
If you watch the movie 'hidden figures' one of the things that popped out is that they figured out using the euler method (which is what we use in game physics) was the key to solving the complex calculations needed for their problem. There is a more in depth discussion here (I don't understand any of it because I don't speak math).
Whether they still use the euler method in the same way now I don't know, but it is a potential way of solving complex scenarios like interacting gravity from different planets (as long as you take into account the downside of euler method being the step sizes and inaccuracy due to this).
Anyway, one possible way of deciding the most efficient trajectory to get towards a desired destination is simply to run a load of simulations. If you consider the closeless to the desired destination as a measure of success (and maybe time taken / fuel used), you can essentially do a shedload of simulations firing off your rocket in all possible directions / velocities and find out which gets you closest to your destination (goodness of fit).
If you could do infinite simulations, you would be guaranteed to find the best answer. This is very inefficient, so for these kinds of problem there is often an optimization strategy to shortcut through the search space. You might for example run a very rough version of the simulation get a rough answer, then run more accurate versions around the rough answer etc.
It is important to note that this type of optimization method doesn't guarantee you the 'best' solution, because often the strategy ends up getting stuck in local peaks of goodness of fit. This is actually analogous to the same thing which happens in evolution, and can lead to evolutionary dead ends and extinctions. Your optimization strategy may for example end up with a direct route to the destination, when the actual best method is to slingshot around another planet.
I like to think this is what happens when han solo is making the calculations to make a jump to lightspeed:
Quote
BEN
How long before you can make the jump to light speed?
HAN
It'll take a few moments to get the coordinates from the navi-computer.
The ship begins to rock violently as lasers hit it.
LUKE
Are you kidding? At the rate they're gaining...
HAN
Traveling through hyperspace isn't like dusting crops, boy! Without precise calculations we could fly right through a star or bounce too close to a supernova and that'd end your trip real quick, wouldn't it?
Anyway of course the potential trouble with this sort of approach of making calculations beforehand is both one of a potential performance blip, and the inability to cope with changing scenarios (perhaps an alien controlled asteroid, like in the Expanse). Whereas the feedback I originally suggested might result in a slightly suboptimal path but has no performance blip, and can deal with dynamic situations.