The PS4 isn't backwards compatible because the hardware, including the CPU family and instruction set, are entirely different from the PS3. The same was true of the PS3 -> PS2, and PS2->PS1, except that those systems, at least the launch models, actually included a single-chip version of the previous system to provide hardware-based backwards compatibility.
In general software development, you do tend to avoid programming at a level that depends on the precise characteristics of a particular hardware configuration -- for example, if Microsoft Word, or even a high-performance PC game, required say, SSE 4.1 and provided no vanilla-C++ fallback, then it would only run on very modern PCs that supported that Instruction set extension -- The software simply couldn't run on a circa 2005 PC.
But game consoles are different -- There's exactly one hardware configuration that will never differ over the life of the console. Games written to squeeze every last drop of power out of a launch PS4 will run on the exact same hardware when the last $99.99 PS4 rolls off the assembly line in 2024. When you know that the hardware will never change, when you know that the install-base of that configuration will be 100 million+ units, and you have a high performance target, then there are near-zero downsides to tuning your software to perform as well as it can on that platform. Sure, if you have a cross-platform game that also runs on the Xbox One, you might maintain another configuration of the software that has somewhat different low-level choices (even then, the PS4 and Xbox one are remarkably similar), but that's another 100 million+ unit platform that's never going to change either. In short, the return on investment is very, very high for tuning to those platforms.
I do suspect that the PS4 is actually powerful enough to emulate a PS3 in software. The PS3 got most of its computational grunt from the CELL SPEs, whose programming model is quite similar to the programmable GPUs of today that are in the PS4. In fact, the PS4 has 4 compute-only GPU clusters (in addition to 14 rendering clusters) that offer identical GFLOPS to the 8 CELL SPEs in the PS3. It should be quite feasible, I think, for Sony to write a transpiler that could read an SPE binary and output similarly-performing GPU compute code. Those GPU clusters might not work as well for some SPE workloads, so it may not be perfect, but CPU/SSE fallbacks might cover a good percentage of those. In short, it might not be able to play all games perfectly, but probably a large percentage would be possible. But this isn't anything that a hacker can achieve -- only someone with access to normal developer channels, with Sony's blessing, and with intimate knowledge of both the PS3 and PS4.
On top of all of that, building backwards compatibility into each new hardware unit is becoming less attractive to console manufacturers because we're reaching a point where streaming games, not unlike streaming video, is possible. As nice as backwards compatibility is, when do you actually use it? Maybe you or I might use it regularly, but most people will never use it or use it very, very rarely. Sony and Microsoft ask themselves "How much does it increase the cost of each Xbox One/SP3 to build in Xbox360/PS3 hardware, or to write a software emulator?" and then ask, "How many people will use it?" If the answers are $25, and only 5%, then the answer is that it costs them $500 to make just one user happy about having backwards compatibility. That's 2 1/2 Xbox 360s or PS3s -- If that one person really cares to keep playing his old games, is more cost-effective for them to just keep or buy the old system. Heck, buy two and stick one in a closet for 20 years so you still have one when the first finally craps out. Being able to stream games changes the equation though -- Now, Sony can have a server-rack full of cost-optimized PS3s running old games on-demand and streamed to your PS4 (or your PSvita, or your PC, or your phone) -- It may not be a 100% identical experience to having the software running on a box in your living room, but for most people it will be more than good enough, and its much cheaper for Sony to build enough of that to the people who will actually use that feature happy, and it gives them much more freedom in how they build, operate, and upgrade that system. On top of that, it solves the problem of how to make old games available (and to keep making money from them), because now users have no expectation of physical distribution.
