Electrical and Computer Engineering Professor Neal Anderson’s doctoral student Natesh Ganesh has won the Best Student Paper award at the Institute of Electrical and Electronics Engineers (IEEE) 2017 International Conference on Rebooting Computing (ICRC). The title of his paper and presentation was "A Thermodynamic Treatment of Intelligent Systems." The conference was held on November 8 and 9 in Washington, DC. The goal of the IEEE ICRC was to discover and foster novel methodologies to reinvent computing technology, including new materials and physics, devices and circuits, system and network architectures, and algorithms and software.
“In this paper, based on a prediction-focused definition of intelligence, results characterizing the thermodynamic constraints on finite state automata models of intelligent physical systems were presented,” Ganesh says about his work. “I showed that under the right thermodynamic conditions, the energy efficient dynamics of a system is synonymous with learning.”
As Ganesh observes now about his paper, “In retrospect, I could have been a little clearer (and bolder) in stating that the results in the paper could indicate a way forward towards “thermodynamic computing,” a radically different design philosophy for building intelligent machines.”
Ganesh notes that thermodynamic computing is a new computing paradigm that looks to leverage thermodynamics and information theory.
“The goal,” he says, “is to obtain self-organized systems capable of learning and intelligence at high energy efficiencies by fabricating them to satisfy the necessary thermodynamic constraints. In the process, we might well have to abandon well-defined architectures and learning algorithms for complex messy systems just like the brain.”
In his winning paper, Ganesh explains that the focus of the computing industry continues to shift towards designing and building intelligent systems that can handle and learn from large amounts of data. He notes that the tremendous progress made in the field of machine learning to perform numerous learning tasks has been due to two major driving factors. The first is the emergence of extremely sophisticated learning algorithms for supervised learning and reinforcement learning techniques. The second is the availability of powerful processing hardware – such as central processing units (CPUs) and tensor processing units (TPUs) – which has powered the tremendous success of many sophisticated resource-intensive, machine-learning algorithms.
However, explains Ganesh, “As we approach the physical limits to scaling and dissipation, we have started to look away from these conventional computing devices and architectures as solutions to our new computing tasks.”
As a result, now we are moving towards neuromorphic and other bio-inspired computing systems using novel devices such as memristors and photonics.
In this context, Ganesh explains that biological systems such as the human brain are useful models, being the product of “millions of years of bottom-up, self-organizing, evolutionary processes and often operate near the limits of energy efficiency. Therefore, improved understanding of these systems, their capabilities, and the self-organization processes that produce them from a computational perspective will help us build better learning machines.”
Ganesh also observes that the use of non-equilibrium thermodynamics and information theory to understand such biological processes has gained significant momentum in recent years. Thus, there has been increased research into developing fundamental relationships between thermodynamics, information theory, and neurobiological systems.
“Since intelligent processes are physical as well,” writes Ganesh, “extending such work to establish the thermodynamic conditions under which a physical system exhibits learning capabilities would be extremely beneficial in the design and fabrication of intelligent systems and usher in the new paradigm of thermodynamic computing.”
Ganesh concludes that “This paper seeks to bridge the gap between thermodynamics, information theory, and the learning capabilities of complex systems.”
ICRC 2017 was part of the Rebooting Computing week, which also included a meeting of the International Roadmap for Devices and Systems (IRDS) on November 6 and 7 and the first Industry Summit on the Future of Computing on November 10. (November 2017)