Jump to content

  • Log In with Google      Sign In   
  • Create Account


Moores Law


Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.

  • You cannot reply to this topic
18 replies to this topic

#1 DevLiquidKnight   Members   -  Reputation: 834

Posted 02 February 2013 - 01:59 AM

How do you suspect Moore's law will be mitigated, in the event that it begins to slow? Some information I read online suggests that I may already be slowing and quantum tunneling offers a large hurdle.



Sponsor:

#2 Bacterius   Crossbones+   -  Reputation: 8523

Posted 02 February 2013 - 03:44 AM

Yes, we have generally reached a limit which means we won't be able to clock our processors any faster until they are made of something other than silicon. In the absolute worst case that we are no longer able to develop better technology, Moore's Law will degenerate into a "parallel processing" argument which states than we can double computational power by doubling the number of cores, which we won't be able to keep doing very long because of spatial constraints (since we can't make them any smaller either). And many applications do not scale linearly with computational execution units.

 

Fortunately, "moar power" isn't the only way to make stuff work faster. In fact, there are much more important factors, including memory access patterns, I/O dependencies, SIMD/MIMD and of course the odd algorithmic improvement. These make a huge difference in the running time of a computationally heavy workload, perhaps more so than just throwing more, faster cores at it. There are also a bunch of specialized tricks hardware manufacturers keep introducing to make common operations work that little bit faster (think horizontal add, fused multiply-add, etc..). I'm sure they have a few tricks up their sleeves for the next few years to compensate.

 

Though I think they are banking on mostly adding more cores to everything while keeping power usage steady, at least GPU's can afford to because most graphics rendering scales almost perfectly, CPU's don't have that luxury and can only do so much to speed up badly designed single-threaded programs. There will be a gradual paradigm change to parallel computing, we may even integrate GPU technology in our current CPU's to create a unified general-purpose parallel processor, and our development patterns will adapt accordingly. Perhaps "cloud computing" will really take off, perhaps it won't. This will take a long time, don't expect to see any of this for at least eight years and perhaps much longer.

 

But, in the limit, yes, from our current understanding of physics, Moore's Law will simply cease to apply until our technology is built from something other than matter. You can only double anything so many times before it becomes too large to handle (ever heard of the rice and chessboard anecdote?)

 

In any case, consumer CPU's do not need more power right at the moment. They are more than adequate. What really needs to be addressed is not at hardware but at software level.


The slowsort algorithm is a perfect illustration of the multiply and surrender paradigm, which is perhaps the single most important paradigm in the development of reluctant algorithms. The basic multiply and surrender strategy consists in replacing the problem at hand by two or more subproblems, each slightly simpler than the original, and continue multiplying subproblems and subsubproblems recursively in this fashion as long as possible. At some point the subproblems will all become so simple that their solution can no longer be postponed, and we will have to surrender. Experience shows that, in most cases, by the time this point is reached the total work will be substantially higher than what could have been wasted by a more direct approach.

 

- Pessimal Algorithms and Simplexity Analysis


#3 phantom   Moderators   -  Reputation: 7092

Posted 02 February 2013 - 04:49 AM

"Moores' Law" annoys me, mostly because when it's used by anyone (and, tbh, gamers are the worst for this) they seem to give it the same weight as the law of gravity in a 'this must happen!' sense... Should have been called 'Moore's observation of the past and projection for the future'.... *grumbles*



#4 JTippetts   Moderators   -  Reputation: 8350

Posted 02 February 2013 - 08:40 AM

"Moores' Law" annoys me, mostly because when it's used by anyone (and, tbh, gamers are the worst for this) they seem to give it the same weight as the law of gravity in a 'this must happen!' sense... Should have been called 'Moore's observation of the past and projection for the future'.... *grumbles*

 

This. Years ago, when people first started talking about it, my first thought was "bullshit." To the best of my remembrance, I've never used the term in any conversation, simply because I never believed it. I'm not any kind of genius prognosticator, but it doesn't really require one to be a genius to intuit that there will be hard limits on scaling and speed that Moore's so-called Law can't circumvent.



#5 SiCrane   Moderators   -  Reputation: 9559

Posted 02 February 2013 - 08:48 AM

It's not like Moore himself said that it would continue indefinitely. He wrote a paper saying transistor densities were doubling every two years in the 1960s and said it would continue for at least ten years. We hit the expiration date on his prediction around thirty years ago.

#6 Servant of the Lord   Crossbones+   -  Reputation: 18498

Posted 02 February 2013 - 10:06 AM

I found this article interesting - the author argues that Moore's Law has already started slowing down, but that we can (and are, and inevitably will) take advantage of similar trends to keep pace in other ways. I don't have the requisite knowledge to critically examine the article - hardware ain't my field - so I'd be interested to hear what others think of it.


It's perfectly fine to abbreviate my username to 'Servant' rather than copy+pasting it all the time.
All glory be to the Man at the right hand... On David's throne the King will reign, and the Government will rest upon His shoulders. All the earth will see the salvation of God.
Of Stranger Flames - [indie turn-based rpg set in a para-historical French colony] | Indie RPG development journal

[Fly with me on Twitter] [Google+] [My broken website]

[Need web hosting? I personally like A Small Orange]


#7 D_Tr   Members   -  Reputation: 362

Posted 02 February 2013 - 04:59 PM

It seems that everything will continue normally for at least 8 years, since 5 nm is on Intel's roadmap. It's hard to imagine anything much smaller than 5 nm being possible with technologies whose core component is photolithography...

IBM has been doing research on stacking multiple die and interconnecting them vertically by thousands of interconnects per square millimeter, with special cooling micro-channels taking care of the generated heat. With such an approach it might be possible to keep increasing the computational capability and probably also drive costs down by not replacing the manufacturing equipment as often.

Meanwhile, there is research in nanoelectronics, namely in methods to include trillions of transistors in a single chip. This paper (http://www.ee.washington.edu/faculty/hauck/publications/NanoSurvey.pdf) suggests that these trillions of devices will probably have to have a regular layout on the chip and that a large number of defective devices per chip will be the norm, so a method will be necessary to avoid using these defective devices.

Bacterius mentioned architectural and algorithmic improvements. True, a good choice of algorithm may make a program execute thousands of times faster, and the choice of the correct architecture might speed a programm up x times, but currently we have the luxury of BOTH architectural/algorithmic improvements and a nice 2x speedup every 20-24 months. If manufacturing technology had stopped evolving in 2007, we still wouldn't be able to play Crysis 1 on a cheap gaming PC!

Edited by D_Tr, 02 February 2013 - 05:05 PM.


#8 mdwh   Members   -  Reputation: 854

Posted 04 February 2013 - 09:15 AM

"Moores' Law" annoys me, mostly because when it's used by anyone (and, tbh, gamers are the worst for this) they seem to give it the same weight as the law of gravity in a 'this must happen!' sense... Should have been called 'Moore's observation of the past and projection for the future'.... *grumbles*

A "law" is an observation though - a simple relation of observed behaviour. Not all scientific laws have the same weight as that of gravitation either, e.g., gas laws which are just an approximation. Perhaps it's unfortunate that people misunderstand what "law" is (same with "theory"), but that applies to all usages of the term.
http://erebusrpg.sourceforge.net/ - Erebus, Open Source RPG for Windows/Linux/Android
http://homepage.ntlworld.com/mark.harman/conquests.html - Conquests, Open Source Civ-like Game for Windows/Linux

#9 samoth   Crossbones+   -  Reputation: 4684

Posted 04 February 2013 - 12:14 PM

Since everything must be portable and wireless and green nowadays (bleh), it's likely that Moore's Law (which is purely a marketing gag) will be replaced by some other bullshit. Such as CPUs will consume 30% less power every year. CPUs with integrated GPUs are another such marketing gag.

Some nonsense will be thought of to give you an incentive to buy a new processor, rest assured. Maybe the new SSE6.3 instructions, which are only supported on 0.5% of all CPUs and useful for 0.01% of all programs, and have different opcodes between Intel and AMD. And of course there will be support for SHA-3 instructions.

GPU manufacturers are going the "low power" route already, though in this case I consent somehow (having to buy a bigger power supply unit because the GPU alone draws 300-400W is simply not acceptable). With Kepler, nVidia who probably has the longest history of ultra power-hungry hardware on the planet, for the first time brought out a new chip with a lot more silicon and only marginally better performance -- but with half the power consumption.

When CPU speeds are maxed out, and more speed is desired, there's still memory bandwidth to be dealt with. I'm sure it is possible to make memory faster, there just hasn't been such a great interest in the past (because L2 cache works reasonably well). However, faster RAM obviously makes a faster computer, too.

#10 Hodgman   Moderators   -  Reputation: 29477

Posted 04 February 2013 - 03:31 PM

If moore's law slowed to a halt right now, but we still wanted faster, smaller PC's, then we'd just have to throw out all our software and start again -- this time teaching everyone multi-core practices from the get-go, so that we can make 100 90's era CPU cores onto a chip and actually have them be utilized (which was the idea behind larabe)

Hardware manufacturing is pretty foreign to me, but AFAIK, chip design is still mostly a 2D affair -- moore's law is about surface area, not volume. So, once we hit an area limit, it's time to start stacking vertically, and then racing to reduce the thickness of these layers too... ;|

Power usage, as mentioned above, is another factor that needs to be addressed. Moore's law has been both driving this up and down -- down when things are shrunk, then back up when more shrunken things are crammed into each chip. Biological computers (us) absolutely put computers to shame when it comes to power usage, so there's a lot of work that needs to continue here after moore's law fails.

#11 Ravyne   Crossbones+   -  Reputation: 7116

Posted 04 February 2013 - 03:57 PM

Moore only said that transistor counts double every 2-3 years, over the years its become synonymous with "processing power doubles every three years", which is really what the consumer cares about anyhow. There's a relationship between performance and number of transistors, sure, but no one cares about how many transistors their CPU sports.

 

To continue the single-threaded performance curve, I suspect that chip manufacturers will find ways to get around the slowing of moore's law. As we approach the limit of transitor shrinks on current technology, we'll just switch to new materials and processes, make bigger and bigger dies, bigger and bigger wafers, and move into '3-D' chips--stacking 2-D silicon dies one on top of the other. There's still hurdles there today, but they're far from insurmountable.

 

I also think that the transition to predominantly-multi-threaded, asynchronous programming can happen quite gradually. As those practices become more mainstream, a chip manufacturer can build future CPUs out of cores that are, for example, only half as fast, but consume only 25% of the area of previous cores -- thus, aggregate performance doubles without affecting area. As apps take advantage of this new way of doing things, they'll still seem twice as fast. That said, some problems are necessarily sequential, and so there will probably always be a place for high single-threaded performance. In the future, I can foresee a CPU that has 1-2 'big' cores, and 8-16 (or more) smaller cores -- all functionally identical, but the big ones chasing highest-possible IPC, and the little ones chasing smallest reasonable surface area. It would be best for OSes to know about these different kinds of CPUs, but you can even handle scheduling to the appropriate kind of CPU at a silicon level, and dynamically move between kinds of cores.

 

Another angle is stream-processing (GPGPU, et all) which accounts for a significant portion of our heaviest workloads. There, they've already figured out how to spread work across thousands of working units. If your problem can be solved in that computing model, we're already at a point where we can just throw more area at the problem, or spread it across multiple chips trivially.

 

The single-threaded performance wall is a hardware issue. The multi-threaded performance wall is a wetware (that's us) issue.



#12 phantom   Moderators   -  Reputation: 7092

Posted 05 February 2013 - 05:26 AM

we'd just have to throw out all our software and start again -- this time teaching everyone multi-core practices from the get-go, so that we can make 100 90's era CPU cores onto a chip and actually have them be utilized (which was the idea behind larabe)

The thing is, while the idea of throwing all the software away isn't workable the fact is that really "we" should be rethinking the way software engineering is taught.

There needs to be a push away from the 'OOP for everything!' mindset and towards one which highlights OOPs time to shine while also exposing people to functional programming styles and teaching them to think about data too so that people have a better understanding of the various ways problems can be solved and we don't get people trying to fill GPUs with C++ virtual monsters which burn resources doing vtable look ups when they really don't need it.

I admit I've been out of the educational circles for a while but if they are still turning out 'OOP are the bestestest!' "programmers" every year we have no chance of over coming the problem.

#13 DevLiquidKnight   Members   -  Reputation: 834

Posted 05 February 2013 - 07:00 AM

So should more computer science majors be focused on multi-core/multi-processor programming is that is the primary way stuff will be heading in the next 20-40 years? Stuff such as distributed systems and concurrency control/parallel computation.



#14 way2lazy2care   Members   -  Reputation: 782

Posted 05 February 2013 - 07:12 AM

The thing is, while the idea of throwing all the software away isn't workable the fact is that really "we" should be rethinking the way software engineering is taught.

There needs to be a push away from the 'OOP for everything!' mindset and towards one which highlights OOPs time to shine while also exposing people to functional programming styles and teaching them to think about data too so that people have a better understanding of the various ways problems can be solved and we don't get people trying to fill GPUs with C++ virtual monsters which burn resources doing vtable look ups when they really don't need it.

I admit I've been out of the educational circles for a while but if they are still turning out 'OOP are the bestestest!' "programmers" every year we have no chance of over coming the problem.

I can back this. More and more frequently OOP isn't the most optimal way to go about things. Being stuck in an OOP mindset can be very damaging in that regard.



#15 Waterlimon   Crossbones+   -  Reputation: 2459

Posted 05 February 2013 - 08:09 AM

I think we should just throw some more information at the compilers so they can do the threading...

 

That means telling it which calls to 3rd party stuff are dependent/thread safe and define the different tools needed for threading (similiar to how you can override new)

 

Of course the programmer will always be able to have more information than the compiler so doing the threading manually might get performance benefits. But if the compiler does it, it will thread everything without mistakes if done correctly.

 

I believe there was an intel compiler (experiment?) for c++ that does just that. Not sure how extensively it could do it.

 

But this probably works well only for our general normal CPUs with a bunch of cores, because if we go to more fine grained parallization i would imagine that more changes would need to be made at the algorithmic level which likely isnt very easy of a job for compilers.


o3o


#16 Sik_the_hedgehog   Crossbones+   -  Reputation: 1605

Posted 06 February 2013 - 04:36 AM

I think we should just throw some more information at the compilers so they can do the threading...

In theory, this is one of the benefits of functional programming. Since the order in which things happen doesn't matter the compiler is free to rearrange the operations as it sees fit, including doing multithreading on its own if it feels like it. No idea how well do functional languages cope with this, though (at least current ones).

 

Of course the programmer will always be able to have more information than the compiler so doing the threading manually might get performance benefits. But if the compiler does it, it will thread everything without mistakes if done correctly.

This used to be the case with code optimization, where a human could generate better assembly than the compiler could. Then over time processors became a lot more complex and compilers became much more smarter, and most of the time a compiler is going to beat a human when it comes to optimization since it can see a lot more. I wouldn't be surprised if the same could be applied to making the compilers multithread the code.


Don't pay much attention to "the hedgehog" in my nick, it's just because "Sik" was already taken =/ By the way, Sik is pronounced like seek, not like sick.

#17 way2lazy2care   Members   -  Reputation: 782

Posted 06 February 2013 - 07:11 AM

I think we should just throw some more information at the compilers so they can do the threading...

In theory, this is one of the benefits of functional programming. Since the order in which things happen doesn't matter the compiler is free to rearrange the operations as it sees fit, including doing multithreading on its own if it feels like it. No idea how well do functional languages cope with this, though (at least current ones).

 

 

>Of course the programmer will always be able to have more information than the compiler so doing the threading manually might get performance benefits. But if the compiler does it, it will thread everything without mistakes if done correctly.

This used to be the case with code optimization, where a human could generate better assembly than the compiler could. Then over time processors became a lot more complex and compilers became much more smarter, and most of the time a compiler is going to beat a human when it comes to optimization since it can see a lot more. I wouldn't be surprised if the same could be applied to making the compilers multithread the code.

 

I feel like unless it was JITed you would get situations where it might give multiple actions the same weight despite them running much more/less frequently than the compiler expects. The compiler would need more data about the expected use of the program along with the code you are giving it.



#18 Hodgman   Moderators   -  Reputation: 29477

Posted 06 February 2013 - 07:16 AM

I think we should just throw some more information at the compilers so they can do the threading...

Of course the programmer will always be able to have more information than the compiler so doing the threading manually might get performance benefits. But if the compiler does it, it will thread everything without mistakes if done correctly.
 
I believe there was an intel compiler (experiment?) for c++ that does just that. Not sure how extensively it could do it.

Unfortunately, the two bold bits are at odds with each other -- if we're supplying the extra info, then we can still make mistakes wink.png
 
Also, generic C++ isn't conducive to automatic concurrency -- not even on a single thread!
For example:

struct ArrayCopyCommand : public Command
{
  int* from;
  int* to;
  int count;
  virtual void Execute()
  {
    for( int i=0; i!=count; ++i )
      to[i] = from[i];
  }
};

The compiler will already try to pull apart this code into a graph of input->process->output chunks (the same graph that's required to generate fine-grained parallel code), in order to generate optimal single-threaded code. Often, the compiler will re-order your code, because a single-threaded CPU may be concurrent internally -- e.g. one instruction make take several cycles before it's result is ready, the compiler wants to move that instruction to be several cycles ahead of the instructions that depend on those results.

However, the C++ language makes this job very tough.
Take the above code, and use it like this:

ArrayCopyCommand test1, test2;
int data[] = { 1, 2, 3 };
test1.from = data;
test1.to = data+1;
test1.count = 2;
test1.Execute();
 
test2.from = data;
test2.to = &test2.count;
test2.count = 42;
test2.Execute();

In the case of test1, every iteration of the loop may in fact be dependent on the iteration that came before. This means that the compiler can't run that loop in parallel. Even if you had a fantastic compiler that can produce multi-core code, it has to run that loop in sequence in order to be compliant to the language.

In the case of test2, we can see that the loop body may actually change the value of count! This means that the compiler has to assume that the value of count is dependent on the loop body, and might change after each iteration, meaning it can't cache that value and has to re-load it from memory every iteration, again forcing the code to be sequential.

As is, that ArrayCopyCommand class cannot be made parallel, no matter how smart your compiler is, and any large C++ OOP project is going to be absolutely full of these kinds of road-blocks that stop it from being able to fully take advantage of current/future hardware.

 

To address these issues, it's again up to us programmers to be extremely good at our jobs, and write good code without making simple mistakes...

Or, if we don't feel like being hardcore C++ experts, we can instead use a language that is conducive to parallelism, like a functional language or a stream-processing language. For example, HLSL shaders look very much like C/C++, but they're designed in such a way that they can be run on thousands of cores, with very little room for programmers to create threading errors like race conditions...


Edited by Hodgman, 06 February 2013 - 07:32 AM.


#19 Sirisian   Crossbones+   -  Reputation: 1727

Posted 08 February 2013 - 03:31 AM

Moore's law will probably continue for a while longer. There have been demonstrated 7 atom transistors and single atom transistor. Quantum tunneling did offer a hurdle, but it's seeming like less of a hurdle nowadays from what I've read. I remember when Wikipedia stopped their fabrication pages at 11 nm since that was believed to be the smallest they could go. Then when new research was made they changed the page to 10 nm. Then when more research was done they added 7 nm and then 5 nm. Humans are notoriously bad at predicting the future.

 

As Hodgman said layers is the newer idea. Stacking transistors and building 3D networks. I think a core idea of this will be fast interconnects at that scale. Processors are already extremely tiny. Xeon Phi for instance is increasing the transistor counts by just moving away from the socket methods. We can easily double the number of transistors in a chip by adding two of them on a board. Keeping doing that and you end up with Xeon Phi's 62 cores which has 5 billion transistors per card at 22 nm. Intel's already designing for 10 nm. Once there it'll be two more steps then they'll be working with a handful of atoms.

 

One thing I think that is going to happen though in relation to processing speed is a huge drop in the number of simple instructions. A switch to a purely SIMD architecture with highly CISC instructions for specialized operations. (Stuff like multiply 1 number is the same cost as multiplying 16 at the same time). We're already seeing that with encryption and compression algorithms which have special instructions so that they run way faster than with the basic RISC set of instructions. I think at one point we'll start to see stuff like hundreds of matrix/quaternion operation instructions and we'll continue to see speed improvements possibly doubling for a while as pipelines are redesigned to speed things up. That or specialized FPGA instructions as transistor count would allow one to program their own cores on a processor for running calculations or whole algorithms. I digress. I don't think the limit if/when we reach it in our life-time will be a big deal. There are so many avenues to research that we probably won't see it in our life-time.
 


Edited by Sirisian, 08 February 2013 - 03:32 AM.





Old topic!
Guest, the last post of this topic is over 60 days old and at this point you may not reply in this topic. If you wish to continue this conversation start a new topic.



PARTNERS