SM2 support is nice if you want to sell your game in India, China, etc, where there actually is a huge market of old PCs.
I've been working on a game for the Indian market that requires SM3+, and there's a huge number of fans who are using SM3-CPU-emulators in order to run the game!!
Are those actual customers who pay for the game, too?
I always struggle to understand when companies target China because it's kind of well-known that they're only willing to pay a fraction of what everybody else is paying, if they don't pirate right away. Development cost and maintenance is the same, however. Insofar I always wonder how this math can work out (but apparently it does).
Now India isn't China, so that might again be different, of course...
EDIT:
About SM2, I would drop it without wasting a thought. I would rather consider whether or not to drop SM3 support. Given SM4, you are deep in the comfort zone, you have everything that you reasonably need, and it's available without weird twists and quirks, and the guaranteed minimums are workable (under SM3 the guaranteed minimums are ridiculous). You really don't need SM5 (nice to have, but who cares otherwise).
Basically, for SM2 you pretty much need to rethink and reimplement everything for a quite low quality result in return, and you already need to rethink and write special paths and workarounds for half of the stuff in SM3. A card that only supports SM2 not only isn't able to do certain shader tricks, it likely can't cope with your triangle count either (so you need to LOD much more aggressively, and likely create special low-poly models in addition so it looks reasonably good).
If nothing else, in SM4 you have guaranteed working MRT support ("working" as in "working on every vendor, and no bogus stuff like multi = 1") and guaranteed vertex texturing (again, no "bogus support"), and guaranteed, working float textures and hardware sRGB. Plus, no obscure limits that you are likely to run into if you do somewhat reasonable stuff.
The fact that you also get geometry shaders and transform feedback is really only sugar on top.
On the other hand, you get entry-level SM5 cards for around 25-30 currency now, so if you expect your customers to pay 25-30 currency for your game, there is really not much of an excuse why they couldn't also have at least a SM4 card (which was already available for the same price years ago).