Makefiles are basically files that associate rules, dependencies, and actions to effectively build a program (and more - they are often used in compilation but have a wide variety of applications). That's all you need to know if you just want to know how to use one to build a library, but if you want to make a makefile (no pun intended) then you can always find a more detailed explanation on the internet.
If you are under Linux, this is easy, the make utility comes with your distribution. Just cd into the library's folder where the Makefile resides, and type "make" (or perhaps the library tells you to use "make setup", it depends on the makefile). Do that, and it should just work. If there's an error that comes up, try googling it, perhaps you need to run the makefile as superuser or you need to download and install some package first, etc..
If you are under Windows, let's just say your day is going to suck, at least in my experience. Makefiles just don't work all that well under Windows, if the library also comes with an IDE project file (visual studio, or whatever) prefer that to using makefiles IMHO. Otherwise I think you need to install CMake, and a bunch of other stuff, and then I don't know. Sorry