It totally depends on what you want to do with your rendering code. The best way to structure your code is to make it flexible so you can choose the right tool for the job at hand. My engine allows for both of the mechanisms that you mentioned, but in my case each object carries around its own material information. The 'state manager' then ensures that there aren't multiple API calls made to set the same data more than once.
The latter of your two ideas is usually called an immediate renderer or something similar. It is generally a higher level object that simplifies the usage of the graphics hardware at the cost of performance.