Also, queue push, regardless of the prototype, can fail. It can fail if it fails to allocate memory, and it could potentially have a ring buffer implementation, that would run out of queue space sooner.
Not necessarily. If the queue uses a fixed-size circular buffer then pushing an item into it can never fail. In many situations old queue items can be discarded over new items.
Using operations that cannot fail whenever possible and making use of transactional memory patterns for destructive operations that could fail is a good way to increase your software's reliability. But that may be overkill for many applications such as games; after all, in a game most of the time you can just abort on error, as long as the player's save file is not corrupted and not too out of date, no harm done.
Although I agree that just checking the prototype isn't enough by itself; you want to have a good read of the library's documentation and maybe peruse its source code a bit to see how it handles errors. When you know for sure that functions that return void legitimately cannot fail, then you are good and can just refer to the prototype for quick reference. Of course, if the library is written by baboons that use longjmp to escape to god knows where on error, do you really want to use it in your software that's supposed to be reliable?