As this is For Beginners, trying to stick with answering your actual questions before the educational aspects...
Every Java resource I've read goes crazy on the idea of interfaces, and uses them for almost everything. What I don't get however, is how they are better than plain old inheritance.
Because as far as software architecture goes, it is a much better design in general. There are some specific cases where that isn't true, but in the general case you should depend on abstractions rather than concrete types.
The are better than "plain old inheritance" because depending on an abstraction means you can add whatever implementation details you want later and they automatically fit into the system, they just work.
An example I frequently give is with graphics systems.
Programmers rely heavily on the abstract classes in Direct3D yet they don't care one bit about the concrete types.
I request an ID3D11Device* and it just works. I can do all the things I want to with that device. I don't care if the device driver actually implements it as an ID3D11GeForce620, or an ID3D11Radeon6230, or an ID3D11IntelHD4400. The only thing I care about is that it does all the things an ID3D11Device is supposed to do.
I can use my object, no matter what the final concrete type happens to be, and it should work perfectly. Every concrete type is completely replaceable with any other concrete type. It does not matter which one the system provides. If I obtain something that implements an interface it implements it perfectly and could have been interchanged with any other object that implements the type. That doesn't mean they perform identically, different graphics cards from different eras may perform faster or slower, or may perform different operations at different rates, but (barring driver bugs) they will always implement every operation exactly as specified, and you can do all the operations on any card that follows the interface.
I shouldn't ever need to write code that detects what specific device it happens to be and write special-case code for that concrete type; instead I can use the provided interface and decide based on that. Some functionality may differ between cards but I don't need to hard-code every single card that can support the feature, I can rely on the interface to query if the feature exists by getting device capabilities.
That last point also allows future-proofing. If I write my program that works on all of today's cards, and sometime in the future someone introduces a new card with different capabilities, it just needs to implement the right interface and it drops in perfectly. If it provides different capabilities they must be queried exactly the same way the interface allows for querying capabilities.
I don't need to modify today's program to support a new graphics card that ships three years from now, it will still work perfectly because it follows the same interface and is completely interchangeable with any other graphics card that also implements the interface. By following the interface everything works perfectly right out of the box.
For another example, let's say I implement an event bus in my system. I want to be able to broadcast "Here is the event" and have everyone listen. I don't care what systems are listening, they can be logging systems, they can be networking systems, they can be debuggers, they can be game event handlers, they can be anything else. All I care is that they implement the interface called IEventBusListener. If someone six months from now, or six years from now, decides to implement some code that implements the IEventBusListener interface they can add it to the code base and it will work perfectly.
For another example, let's say I'm working on a game and I want to find the nearest thing in the map that is a weapons spawner. The foolish way to do that is to look for a specific list of weapons spawners that the developer happened to know about when the code was written. If sometime later somebody adds another type of weapon they need to search for all the places in the code that relate to the weapons and hope they update all of them correctly. It is impossible for someone to come along and add a new spawner type without modifying all the source code. The smarter way to do it is to provide an interface, perhaps IWeaponSpawner, and search the map for anything nearby that implements that interface. If sometime later someone adds another type of weapon in the world it is automatically hooked up and completely integrated into the system. (This works just as well for component based or ECS systems, some happen to use a slightly different identification method rather than inheriting an interface, but the overall design is identical. Look for something that implements the component, something that implements the interface, something that claims it does the action, rather than looking for a specific list of items that you happened to know about at implementation time.)
When you write code, always write code that depends on the abstract types, on the interfaces. Only use the operations provided in the abstract types, on the interfaces. This helps all development down the road and new code can just drop into place.
Say I have a class GameObject, being the abstract base class for everything in my game. Something in the game might be a visible physics object, so if I want an object that does that using Interfaces, I would write a Renderable interface and a Physics interface and implement both. But say I have an object which doesn't use physics. I would have it implement just the Renderable interface.
You could do that, but most systems don't.
As mentioned above, this is a HAS A interface. This is composition, not an interface.
A game object does not satisfy IS A. Trying to replace them doesn't work. Swapping it out, a game object IS A model doesn't make sense.
A game object HAS A model. Or maybe it doesn't. Or maybe a game object has several models. Maybe there are models for "Full", "2/3 Full", "1/3 Full", and "Empty". Maybe there are models for "Base Started", "Base 10% Complete", "Base 20% Complete", "Base 30% Complete" ... "Base 100% Complete". Maybe there are models based on damage taken, or based on state of repair. You might implement an interface that says your object potentially has a model and request a reference.
Again, your game object does not implement the interface to be rendered. It is not a thing to be rendered. You should not be able to swap it out with any other thing that could be rendered. It may expose an interface to provide a reference to a renderable object, but it is not renderable itself.
Similarly with physics, a game object HAS A physics shape. Or maybe it doesn't. Or maybe the game object -- a character -- has a capsule physics shape when running, a sphere physics shape when squatting, and no physics shape when deceased. Once again, you might implement an interface that says your object potential has a physics object and request a reference to it, but it is a HAS A rather than an IS A relationship.
Why are Interfaces used instead of just Inheritance alone?
There are a set of programming principles under the great acronym SOLID. Through a bunch of trial and error, people have discovered that when you implement these principles in your code designs it goes a long, long way in keeping your code maintainable. It helps reduce bugs. It helps make it easy to swap out systems, to extend systems, to reuse systems.
Interfaces enable several of those principles.
Yes, interfaces have a small cost of a few nanoseconds when they are followed. If there is only one concrete type and there will only ever be one concrete type you should consider just using the concrete types directly rather than using virtual dispatch as interfaces or abstract types use. For Java, that means marking this as final. The O in solid, the Open/Closed principle, means to keep things open for for extension (meaning you can make new behaviors and extend on it but it is also closed that it must implement exactly the behavior of the base or abstract type, and be completely interchangeable with it. If you are not following this rule, when something is closed to both, make sure you make it completely concrete, and in Java that means marking it all over with final so you're not paying the cost since java is virtual by default.
However, any time you want replaceable parts -- and you almost always want replaceable parts -- it is the cheapest method available because every system out there has heavily optimized the pattern.
Use interfaces or abstract base types or whatever they are called in the various other languages when you need it, and you almost always need it. They are a great solution to many problems that every experienced programmer knows and every tool recognizes and makes faster for you.
