Presumably the other end already caches the fact that the request was declined, to avoid having to decline a flood of persistent requests.
I think the only thing you can do is not decline the request, or wait until some kind of time-out.
You should make sure to cancel/destroy any previous invitations.
Perhaps turn WiFi off and on again on both devices might reset the decline-cache?

Note that the core WiFi code doesn't know what to do when the invitation is declined; check out the comment on line 1707:
https://android.googlesource.com/platform/frameworks/base/+/b267554/wifi/java/android/net/wifi/p2p/WifiP2pService.java

Overall, I think WiFi Direct is not a great technology. It might help with connectivity when two people want to share pictures and are in the middle of the jungle with no access point or cell phone tower nearby, but otherwise, using regular IP networking over regular WiFi is much better.

Overall, I think WiFi Direct is not a great technology. It might help with connectivity when two people want to share pictures and are in the middle of the jungle with no access point or cell phone tower nearby, but otherwise, using regular IP networking over regular WiFi is much better.

I wish Android would do some equivalent of iOS's multipeer and bundle all their direct device-to-device technologies under one comms layer agnostic API. I don't really want to have to write code to support bluetooth, WiFi Direct and WiFi Aware (new for Android O - https://developer.android.com/preview/features/wifi-aware.html) just for some 1 vs 1 multiplayer to accomodate a minority of users who want to play when there's no router around.