I’m conflicted. Generally speaking, if you have to go out of your way to proclaim something is not dead, it’s usually dead or dying. But in this case it’s Jim Keller that’s proclaiming it and he is no ordinary engineer. He is the closest thing to a rock star that us computer architecture and ECE nerds have had since the passing of Bob Widlar.
Keller’s talk flies in the face of Hennesy and Patterson’s Turing lecture who advocate hardware and software co-design to address the diminishing returns of Moore’s Law and Dennard Scaling. When heavyweights like these are at odds, small fry like myself sit back and try to soak it all up. Intuition tells me that neither of them are wrong and both of their approaches will contribute to the overall acceleration of computing technology. I’d suggest watching both lectures and deciding for yourself what you think the future of computing will be like.
The Law of Accelerating Returns: The most poignant part of Jim Keller’s lecture outside of computing technology was his introduction of the law of accelerating returns.
First, we start with the familiar law of diminishing returns which says there is a point where the level of benefit gained is less than the amount of work invested. With regard to computing, many of the sub-technologies are already at the top of the s-curve where returns are flattening out. This is a common occurrence in the technology field where new and superior technology or methods replace or vastly improve upon their outdated predecessors. However, as the technology or method matures, getting more return is like extracting blood from a stone. Think of the jump from planar transistors to finFET and how difficult it is currently getting to improve upon them (i.e. Intel’s widely publicized 10nm gaffe).
The law of accelerating returns as it relates to computing says that individual technologies are on diminishing return curves but total computing power is on an accelerating return curve. This is driven by the sum of sequentially faster S-curves. Going back to the transistor example, the finFET is staged to be replaced by a newer technology such as nano-wires and will accelerate our ability to scale transistor density. Similarly, advances in hardware and software co-design will be another fast s-curve that will accelerate the overall compute capability curve.
The law of accelerating returns makes perfect sense and it applies as much to every day things such as compounding investing returns as it does to technology. Somehow up until this point, I’ve failed to recognize it and apply it to other aspects of life.
In any case, I’m satisfied to have picked up a new framework to view complex multi-part problems and will look forward to utilizing this new knowledge.
Pado Pado 
One comment