I don't think it's even feasible to just start a 'porting' company out of the blue without any previously published titles to be completely honest with you.

Beenox started that way. Of course it became one of many Activision studios over time and produces Spiderman Titles now, so its definitely not impossible. They started out by porting games to PC.

You dont need a full blown team of artist, musicians, designers

You'd be surprised. Porting say, a facebook game to a mobile device for example, isn't as straightforward as you'd think. Some technologies don't match together (vectorial art in flash vs iPhone for example, there are workarounds, but still) and visual ratios often mean you need to redo a lot of UI art. You may be under-evaluating the amount of work required.
Also, in a lot of ports, the codebase need to be started from scratch because of native incompatibilities. You can take that chance and refactor the code at the same time, because it may have been developed by iterations, but you know what the end product will be like, so a few optimizations are possible.