As I said in this topic, I currently have an idea for an exremely modular, flexible game. ... my focus would be on the "plugin/modding" system of the game ... Since the game is a platformer, each entity-type, each platform, etcetera would be an add-on; But the question here is, can the performance keep up?
No. No no no no.
First things first. Your game "as a whole" might be the add-on. In the sense you might have no game-specific logic in (C++, or your lang of choice) code. Your game is not an assembly of add-ons. It is a cohesive blend.
Now that we got out the ideological part, let's get to your question. Will perf keep up?
Yes.
Be explorative, don't worry too much! Your priority is getting the job done!
More involved answer is: callbacks by themselves won't kill your perf. What it matters is how often they are called and how much they cause you to deviate from the "optimal" path.
In your example, you want to check if "entity1" collides with "entity2".
As far as this is what goes, you might just have the physics processed as usual and then iterate on all resulting contact points to extract the set involving entity1, then checking if it contains entity2. This is doable for a small amount of checks. As the interesting entities grow, it's probably better to go on a "sensors" approach - they are physics rigid bodies which automatically keep track of their contact points so you can just check their internally collected list. Bullet calls them "ghost objects" if memory serves.
In general, if the check to call is trivial (an n-m test like above is not, albeit it might be viable) the cost of the callback depends on how much often it gets called. As long as it's event-based, I've found this to be very viable.
Keep in mind there could be ways to emulate what you need while still being hi-perf. Here's a real world example on a game I used to work a couple of years ago.
I had those things. We'll call them "rollys". Rollys just roll around. BUT they don't fall in the water, turning back to avoid falling into the void/water. There can be 1-8 rollys each "lair" and there are usually 1-4 lairs.
The initial solution involved a per-frame, per-rolly callback. It involved sweeps and was fairly complicated. Performance was viable albeit far from perfect on the lowest-end devices (Atom).
Solution 2: rollys were updated to include a special "hit descriptor" and the level was annotated with special "rolly only" colliders. Now rollys can just bound off the special colliders - they are the same to them. They still behave the same, but this solution requires way less tests.
Long story short: before going into scripting, ensure you have a decent data-driven design. While you might want to keep game-specific code out of your application (or not) there are features you're going to need and those should be just built-in.