There's no "generic" guide on how to mod a game.

It wildly varies from a game to another and all depends what do you actually mean by "modding".

Something like a texture replacer for a game like Nier implies you "just" have locate the related textures and/or models in the game's files and replace them. Games, even if some try to hide that fact by packing/zipping data files, tend to be using pretty standard formats for data. In most cases the harder part is to determine how the data is packed, and finding/making tools to unpack/repack it once you're done modifying it.

For more complicated stuff, it depends of what's accessible through data files and what the modder can make out of it. It's easier for some games than for others. For instance, you'll have a much easier time replacing textures/models/sounds for a game written with Unity, or any other generic game engine than for a game made from scratch (well, except if it has been designed with modding in mind).

As for the legality of it, it varies. As a general rule, as long as it's for your own enjoyment (aka no redistribution), it's perfectly okay.

And to answer the question "how can a person make a mod from a new game?" on a more abstract level. People who do that tend to already have a lot of experience in modding games. The more you're used to modding, the easier it gets to identify file types, and structure.