For finding the shortest path between two points there is Djikstra's two point shortest path algorithm. (Google it). Unfortunately the algorithm takes a long time and a lot of space, so there is a way to guide it to faster results. By adding a heuristic -- such as the physical distance on a map -- it becomes the A-Star or A* algorithm, which is extremely common in games. A* can potentially give a less than optimal path, but it is usually good enough and the speedup is worth the rare exception.
I believe that, given a proper heuristic (distance to destination, for instance), A-star is guaranteed to return an optimal result. (See the "Admissibililty" section of the wikipedia article https://en.wikipedia.org/wiki/A*_search_algorithm) I think this is actually the common case for games, so you'll get an optimal result. Have I missed something?
A* is optimal if you fully close the last node to ensure that there are no neighboring paths that would be shorter. The difference is trivial in some cases, so the step is optional depending on what you're doing. There are several other ways to 'relax' A* in order to fit into performance or memory constraints, such as https://en.wikipedia.org/wiki/SMA*
Regarding OP, if he's searching for paths to multiple destinations then Dijkstra could do it. Just keep expanding until you've touched all the goal nodes and you should end up with a sort of 'all destinations' map. Could take up a lot of memory though depending on the problem space. Without knowing what you're actually doing it's hard to say what the best option would be.