Governing Eq. for Thermodynamic Computing

Following my previous post on thermodynamic computing, I'm going to just write down my summary notes from section II A.1 in ref. [1]. For the most part it is just a rephrasing of the paper, so you could easily just read it, but for me I needed to write it down somewhere. There also is a more through derivation and review of Langevin dynamics in ref. [2] as well as the Fokker-Planck equations.

Langevin Equations

From my perspective the primary focus of thermodynamic computing is the use of continuous variables, that is a signal is given by $x \in \mathbb{R}$. The physics of such signal is given by stochastic dynamics¹ which contains terms related to dissipation and fluctuations. The equations which govern the dynamics are:

$$ \begin{align} \mathrm{d}x &= \frac{p}{M} \mathrm{d}t \\ \mathrm{d}p &= -\nabla U(x) - \frac{\gamma}{M} \, p \,\mathrm{d}t + \mathcal{N}\left(0, \frac{2\gamma}{\beta} \, \mathbb{I} \, \mathrm{d}t\right) \label{eq:langevin} \end{align} $$

The second equation is where the stochastic dynamics comes into play and as a result we call these types of equations stochastic differential equations (SDEs). The first equation is just the kinematic equation which links the change in generalized coordinates.

In the second equation, the first term is the conserved force due to a potential, while the second and third terms correspond to a dissipative force due to dampening-frictional term and a random force due to real-world noise, i.e., white noise that occurs at various time-scales.

The constants $\gamma$ and $\beta$ are positive and real valued, and correspond to the strength of the dissipative force while $\beta$ is usually the temperature-scale which modulates the intensity of the thermal/noise fluctuations. A smaller $\beta$ would mean more intense noise fluctuations. The $M$ corresponds to the mass which is a positive real scalar (or positive definite matrix). Depending on the relative scale of $\gamma$ and $M$ the Langevin equations are referred to as overdamped ($\gamma \gg M$) or underdamped ($\gamma \ll M$).

The eqs. $\ref{eq:langevin}$ correspond to a single trajectory in phase space. However, what we are typically most interested in is the time-evolution of the system's probability density function (PDF). This evolution is described by the Fokker-Planck equation, which is derived from the Langevin equations. The Fokker-Planck equation accounts for both the deterministic dynamics and the stochastic fluctuations captured by the Langevin equations, providing a comprehensive description of the system's statistical behavior over time. For more details on Fokker-Planck equations see ref. [2].

Fokker-Planck solution

The stationary solution of the Fokker-Planck equation, which describes the system's state at thermal equilibrium, is given by the Gibbs distribution:

$$ p(x) = \frac{1}{Z} e^{-\beta U(x)} $$

with $Z$, the partition function, ensuring normalization. This distribution emerges from the interplay between the deterministic forces that drive the system towards lower energy states and the stochastic forces that induce thermal fluctuations, highlighting the fundamental principle of equilibrium statistical mechanics.

The primary interest is now to determine what the exact PDF might be given a specific energy potential $U(x)$. In the case that $U(x)$ is quadratic, i.e., $U(x) = x^T A x + b^T x$ the Gibbs distribution becomes Gaussian²:

$$ \begin{equation} \label{eq:solution} x \sim \mathcal{N}(A^{-1}b, (\beta\,A)^{-1}) \end{equation} $$

It is important to note that $x$, along with corresponding³ momenta $p$, in this context is a vector of generalized coordinates that can represent various physical or analogous quantities, depending on the dimensionality and nature of the system being studied.

If we pay particular attention to eq. $\ref{eq:solution}$ we see that if we can draw enough samples of $x$ then we can determine $A^{-1}$, which is the inverse matrix from $U(x)$. Therefore we have a way using stochastic dynamics to invert a matrix assuming we can prepare and equilibrate the Gibbs distribution given $U(x)$. The excitement is obviously that many problems in ML/AI and science are all about inverting matrices.

Footnotes

---

1. The nuanced point to note about term stochastic dynamics is it means that the dynamics are represented by a probability density function. The distinction is made here because we don't want to just say its a random variable but rather a time-evolving pdf. ↩

2. You can show that this potential indeed results in a Gaussian distribution if you use the technique to complete the square, which is just basic algebra manipulation. You'll end up with something like $p(x) = \frac{1}{(2\pi)^{n/2} (\beta^{-1} |A^{-1}|)^{1/2}} e^{-\frac{1}{2} \beta (x - \mu)^T A (x - \mu)}$, with $\mu = A^{-1}b$ and $\Sigma = (\beta\,A)^{-1}$. The assumption is $A$ is symmetric and invertible. ↩

3. This is usually referred to as the canonical conjugate momenta (see Hamiltonian mechanics). ↩

References

[1] D. Melanson, M.A. Khater, M. Aifer, K. Donatella, M.H. Gordon, T. Ahle, G. Crooks, A.J. Martinez, F. Sbahi, P.J. Coles, Thermodynamic Computing System for AI Applications, arXiv Preprint arXiv:2312.04836 (2023).

[2] L. Peliti, S. Pigolotti, Stochastic Thermodynamics: An Introduction, Princeton University Press, 2021. URL.

Reuse and Attribution

Stochastic Bits

We are all familiar with a bit, a 0 or 1, a false or true, that describes a discrete binary outcome. We can also think of a probabilistic bit, representing a continuous probability between 0 and 1. We then have quantum bits, which represents the superposition state $\vert\Psi\rangle = a\vert 0 \rangle + b\vert1\rangle$, with the probability of measuring a $\vert 0 \rangle$ or $\vert 1 \rangle$ given by the probability amplitude squared. But what if we have the scenario where randomness is introduced to the information state?

s-bit

A stochastic bit, s-bit, which is a bit which experiences random dynamics (i.e., stochastic) between two states:

$$ \begin{equation} s(t) = \begin{cases} 0 & p(t) \\ 1 & 1-p(t) \end{cases} \label{eq:sbit_discrete} \end{equation} $$

This says that the state at any given time of an s-bit, $s(t)$, is 0 or 1 given the probability $p(t)$. The signal, $s(t)$, could look something like:

Stochastic bit.

If we had a continuous scenario where we deal with nats¹, we can write this as:

$$ \begin{equation} s(t) = \int_{-\infty}^{\infty} x \cdot p(x,t) \, dx \label{eq:sbit_cont} \end{equation} $$

with $x$ being a random variable with probability density $p(x,t)$ corresponding to the signal state and $t$ the stochastic dynamic time variable. We can get something like:

Stochastic unit.

This is emblematic of random-walk or Brownian motion, that is, a drift in the signal due to random motion (i.e. noise). Typically when it includes time domain processes, it is usually called the continuous-time stochastic process or Wiener process.

Why use s-bit

As Patrick J. Coles puts it in his good overview video [1], some ideas are to think about how stochasticity can be used as a resource in ML and AI. He gives some examples on how random neuron dropout is used to prevent overfitting and sampling for generative AI uses random noise. Moreover, the interesting thing is the shift from discrete to continuous variables, since in ML/AI the weights, features, and probabilities are over continuous values. The idea is to use the stochastic behavior of physical systems to process information. In the video Patrick calls these s-units which seems fitting. All this obviously leads to thermodynamic computing which I'm diving into².

Footnotes

---

1. When dealing with continuous variables we use nats, a natural unit of information, which can be calculated from the Shannon entropy as $H(s) = -\int_{-\infty}^{\infty} s(t) \ln(s(t)) \, dt$ ↩

2. See my post on my recent focus on thermodynamic computing. ↩

References

[1] Thermodynamic AI and the Fluctuation Frontier | Qiskit Seminar Series with Patrick Coles, Qiskit Quantum Seminar, 2023. URL (accessed March 19, 2024).

[2] P.J. Coles, C. Szczepanski, D. Melanson, K. Donatella, A.J. Martinez, F. Sbahi, Thermodynamic AI and the fluctuation frontier, (2023). arXiv.

Reuse and Attribution

Shifting focus for a little

Recently my intellectual curiosity has driven me towards the topics of stochastic thermodynamics, analog computing, and probabilistic ML all connected to the the recent pop in thermodynamic computing by start-ups Extropic and Normal Computing. I can't say I know enough to really explain what the value is, other than knowing that these types of computing paradigms are extremely energy efficient and are one of the best prospects for scalable AI. Therefore in light of this, I'm going to pull a little back from my self-learning activities on graph NN (😢) and focus on these topics; I have no idea for how long or how much steam I'll maintain. My future post will probably be on these topics, or I might just keep with posting on new "tools" I'm exploring.

How am I going to achieve some baseline knowledge? Well I already know thermodynamics pretty well, but it's all from an equilibrium perspective with thermodynamic potentials and state variables. What I need is to study a bit more non-equilibrium thermodynamics, which is the focus on the time evolution of the state variables. But in particular to thermodynamic computing is stochastic¹ thermodynamics, which is the non-equilibrium dynamics due to random fluctuations that occur on small collection/subset. In other words, the study of random dynamics in microscopic/mesoscopic collections of particles. The reason this is different that just non-equilibrium thermo, at least I think, is that when you deal with small collections of micro/meso-scopic systems, the statistical law of large numbers does not guarantee the same convergence of ensemble averages to thermodynamic values as in macroscopic systems. Meaning that expectation values remain meaningful but must be interpreted with consideration of significant fluctuations around these values. For instance, temperature can still be considered as an average kinetic energy of the ensemble, but with an increased importance of understanding the distribution and fluctuations of these energies.

Futhermore, stochastic thermodynamics distinguishes itself by explicitly accounting for the random, fluctuating dynamics of individual particles or small assemblies of particles. One of the key concepts in stochastic thermodynamics is that the fluctuation theorem, which quantifies the probability of observing fluctuations away from the average behavior in non-equilibrium systems, is dominant. These theorems provide a deep understanding of the irreversibility and the second law of thermodynamics in a statistical sense, showing that while the second law holds on average, there are observable, albeit rare, violations on the microscopic scale due to thermal fluctuations.

In essence, stochastic thermodynamics provides a comprehensive framework to study the energetics and dynamics of systems where fluctuations cannot be ignored, making it crucial for understanding processes at the molecular and nanoscale, especially in biological systems and nanotechnology.

Warning

This is what I think it all means! I'll find out if I am wrong or right as I learn more.

So what do I need to do? Well I have the following set of books that I'm going to read through:

1. L. Peliti, S. Pigolotti, Stochastic Thermodynamics: An Introduction, Princeton University Press, 2021. URL.
2. Ulmann, Bernd. Analog Computing, Berlin, Boston: De Gruyter Oldenbourg, 2022. DOI
3. K.P. Murphy, Probabilistic Machine Learning: An introduction, MIT Press, 2022. URL.

I'm going to read 1 and 2 in tandem, and will read 3 over the next 6 months. The goal is to understand stochastic thermodynamics and analog computing thoroughly. Then I'll read through the papers [1-3] on thermodynamic computing. Finally I'll try to connect 3 to 1 and 2.

In the process of this learning, I'm going to write my notes up in a small handbook using the latex format graciously provide by Francois Fleuret[4]. No promises on the quality of the book, only that it will contain my notes on 1 and 2. I'll also write blog post, but these will most likely be renditions of my notes in to book. I'll probably make my notes available on my website if I'm happy with the final result.

Footnotes

---

1. The use of the term "stochastic" refers to random fluctuations in non-equilibrium thermodynamic processes at microscale. ↩

References

[1] T. Conte, et al., Thermodynamic Computing, (2019). https://doi.org/10.48550/arXiv.1911.01968.

[2] P.J. Coles, et al. Thermodynamic AI and Thermodynamic Linear Algebra, in: Machine Learning with New Compute Paradigms, 2023. https://openreview.net/forum?id=6flkWTzK2H.

[3] A.B. Boyd, J.P. Crutchfield, M. Gu, F.C. Binder, Thermodynamic Overfitting and Generalization: Energetic Limits on Predictive Complexity, (2024). https://doi.org/10.48550/arXiv.2402.16995.

[4] François Fleuret’s git - littlebook.git/tree. https://fleuret.org/git/littlebook (accessed March 13, 2024).

Reuse and Attribution