No. The destructor should not be explicitly called.
Because 'game' is on the stack, exit() when called will free everything on the stack (I think? I don't generally use exit()), and everything freed will call their own destructors.[Edit:] Apparently exit() doesn't unwind the stack!

Another reason to avoid using it in your situation.
Also, 'Game' is not a singleton in the example you gave above. Just because there
is only one, doesn't mean it's a singleton. A singleton is something different (and something you want to avoid).
However, exit() isn't what you probably want to use there (
though it'd work). Instead, why not just call
window.close() when you get sf::Event::Closed, like the SFML documentation suggests? Alternatively, you could have your own bool instead of
window.isOpen(), and set the bool to false when you get
sf::Event::Closed, though the result is the same.
I like my code to exit properly out of the
int main() (or whatever entry point) function by a good
return call.