I am always feeling that I should process the movement within the movers (pathfinders) themselves.

But I have a small decision to make,

there should be some compromise by now.

Consideration 1: (one interface preferred)

When 2 different movers, such as astar and the hierarchical astar both produce waypoints of same kind, it should be more convenient to give a consistent interface for both

HPA is not part of AStar, maybe I can do it

Consideration 2: (movers responsible for moving the objects)

It provides a cleaner organization, and I can easily start and stop one mover and switch to another at my own will

I see pathfinders as a general subroutine, a function where you can ask for a path. The returned path is then handled by the object that wants to move.

First the Single Responsibility Principle applies. Finding a path and following a path are different. Give them both separate responsibilities.

Second, finding a path is somewhat expensive, both in memory and compute time.  It can potentially require a large amount of both when paths are complex. That's another argument for keeping them separate.

Not everything moves the same way. If the code returns a path that can be used by anything, then whatever that specific thing is attempts to follow that path, it allows for customizing how things move.  So another argument for keeping them separate.