Trying all IP addresses is not feasible, nor is it reasonable.

For finding LAN servers, you can issue a broadcast a specifically formatted UDP packet to a known port. Each server would be listening on this port, and would respond to such packets with information about how to connect to them.

For finding servers on the Internet, most games have a well known central server which is run by the game's owner. Third party game servers will contact this central server on startup, which registers their details with it. Clients can query the central server for a list of game servers. This central server can also help achieve NAT punch-through, which is a common problem on the public Internet.

Simplest thing that works:
Build a service that receives "I'm alive" messages from each game server. When you receive that message, list it as "alive" for the next 10 seconds, using the address you saw as sender of the "I'm alive" message. Have a timer that removes old servers (older than 10 seconds) once a second or so. Have the server re-send the "I'm alive" message every 5 seconds while it still wants to be listed.