Most of these depend heavily on the game.
"Server" is a fairly vague generality for a computer program that accepts a connection and does something with it.
You are correct that a server machine can be just a PC in a closet. It can also be a home PC. It can also be a mobile phone or tablet, or a tiny FPGA board. The server program can be anything from a file sharing system as simple as TFTP or as complex as BitTorrent, it can communicate virtual desktops or game protocols or web pages or a nuclear reactor's interfaces. A server is a very generic term.
Often the publicly visible servers are lobby services that introduce players to each other. They connect players to other players, then stick around to record results and game summaries.
Some games have the game logic run on the company servers, which can expensive to maintain; MMOs typically do this to help reduce cheating. Other games run the simulation on one or more of the player's machines. When a client doubles as the in-game host, which one does the hosting also varies by games.
When the host is dropped, it also depends on the game implementation. Some games have every game client running their own simulation where every player broadcasts their data in an all-to-all fashion, and if the host drops a new host can be selected automatically by the remaining players based on connection properties. Other games will immediately terminate the play session.
Having a 'dedicated server' can mean different things depending on who you are talking to. It could mean the a virtual machine has a dedicated IP address, or that an entire physical machine is sitting in a room specifically for your use, or other things entirely. The benefits depend on what they mean by the term and also on the nature of the game.
