Anyhow, I get talking about cool hardware and I start to ramble -- Long story short, yes you can do what you want to do today, but the tricky part is that GPUs just aren't organized like a CPU or even a bunch of CPUs (Midgard is the exception, Knights Landing to a lesser extent) and so you can't just expect it to run CPU-style code well. A big part of making code go fast on a GPU is partitioning the problem into managable, cache-and-divergence-coherent chunks, which tends to either be super-straight forward (easy) or requires you to pull the solution entirely apart and put it back together in a different configuration (hard).

Is there something that could be mentioned as a cause for this?

(that each of those cores cannot execute in 'independant direction'?)

I'm not sure if you're looking for search terms or what, but I can summarize the driving force between why CPUs and GPUs are so different at a hardware level.

CPU cores are designed to do only one thing at a time. To get more stuff done per wall-clock-time-unit while working on only one thing at a time, CPUs have evolved to reduce the amount of time from reading an instruction to having the results. When you do only one thing at a time, the only way to do more is to do that one thing more quickly.

In a modern CPU, only between 1-2 percent of the transistors comprise the ALU, registers, and basic control decoding and execution logic that your CS100 class tells you that a CPU is, and that you think of as doing the work of the processor -- the other 98 percent of transistors implement and manage structures that exist only to make that 2% "CS100 model" of the CPU go fast. Caches reduce memory latency, out-of-order execution finds instructions that are independent and send them to available execution units at the same time -- SMT (hyperthreading) does the same but kicks in when there's not enough independent instructions in a single program thread, branch predictors track which way your branches (ifs and loops) go most often so that the rest of the fancy footwork isn't mis-spent on doing the wrong thing. As a consequence of only doing one thing, that one thing can walk all over memory doing what it likes without dragging anyone else along -- it can branch here and there without much care.

GPU cores, on the other hand, are designed to do many versions of the same thing all at once. To get more stuff done per wall-clock-time-unit, and being able to do many things at once, GPUs have evolved not to do the same number of versions in less time, but to do many more versions in the same time. When you can do many things at a time, the best way to do more is to just multiply the number of workers, rather than making fewer workers go faster -- the trick is to manage complexity that results.

In a GPU, much, much larger percentage of the transistors actually do the work that you think of it as doing. They achieve such a high percentage by simplifying or simply not having the kinds of structures that CPUs use to go fast, then make up for it by putting lots of copies of what's left on the chip. There are still caches, of course, those are always a win. Something like SMT (hyperthreading) is there, but in the simplified form Phantom described. There is no out-of-order execution and no branch prediction. The most important and fundamental thing that GPUs do to manage complexity, though, is that while each "thread" can have a certain amount of data input and state that's unique to itself, it *has* to share its code with ~64 (typical) siblings, and that program is executed in lock-step. The benefit of this to silicon complexity is that you spread that free-loading control logic that's not doing any "work" among 64 "workers" that are, the downside is that none of the workers can go off and do their own thing -- if the 64 workers reach an 'if' statement, and even 1 of the workers goes in a different direction than all the others, then all the workers have to do all the work of both the 'true' and the 'false' code-paths, then decide which version of the work to keep and which to throw away. On the hardware level, the 64 workers are inseparable, and so the programmer who want's his code to go fast on a GPU takes the burden on himself to try and make sure that his solution is structured in a way where all the workers can all go down the 'true' or 'false' code-path together -- then all the throw-away work can be avoided and all the workers are doing useful work all of the time.

However, this GPU model of brigades of workers marching in lock-step is probably starting to sound rather restrictive compared to the free-wheeling CPU model, and you're absolutely right. CPUs, for all their single-mindedness, tackle problems in any shape with equal capability. But the hive-mind of GPUs is not so flexible, every problem must be adapted to its collective way of thinking, and the problems that easily lend themselves to this way of thinking are the ones that will shine most brightly on a GPU (this is also part of the reason GPUs have been able to change the underlying architecture so drastically, because the programmer speaks to the hive-mind, rather than the individual workers). Some kinds of problems don't map well at all, for a variety of reasons -- perhaps the code is too branchy, the dataset too small, the memory accesses too sparse, or the latency to travel the PCIe bus to the GPU and back again overwhelms any benefit of parallel execution. Luckily for GPUs, however, a great many interesting problems map naturally, or well enough with clever coding, that GPUs are extremely interesting for some problems -- aside from graphics, GPUs can be used in artificial vision, physics simulations, weather simulation, financial and risk analysis, gas and oil exploration, and many others.

All of the reasons above are why CPUs and GPUs are so different at the transistor level, and why you have to approach them so differently. If you approach one as the other, you trade away all of its strengths for its weaknesses -- and you will very likely find that the resulting code will run no better, if not worse, than where it ran to begin with.