Bamboozeled by Gaussian Processes

For the past 2 years or so I've been trying to get comfortable with Gaussian processes (GPs). I thought I understood GPs, especially after working through a basic example. The problem is in that example I used an external Julia package called GaussianProcesses.jl and recently I've been trying to implement GPs from scratch for multivariate cases¹.

Usually, the application of GPs is straightforward because it's just the construction of the covariance matrix via the chosen kernel (which does have some hyperparameters to tune). What's nice is that the kernel trick handles the scenario for any dimension feature input, $k: \mathbb{R}^d \times \mathbb{R}^d → \mathbb{R}$. But for some reason, this confuses me in the programming of it, I just need to stop second-guessing myself. Also, the tuning of the kernel parameters seems not to be that trivial.

To alleviate my frustrations in feeling like I don't understand GPs as well as I would like to, I'm going to be spending the weekend reading through:

• R. Garnett, Bayesian Optimization, Cambridge University Press, 2023.
• D. Packwood, Bayesian Optimization for Materials Science, Springer, Singapore, 2017. https://doi.org/10.1007/978-981-10-6781-5.

Both are excellent texts. The one by R. Garnett is particularly complete and ideal for self-learning. I do wish it had some code (from scratch) to go along with the book. I would say the book is as good as Gaussian Processes for Machine Learning, but as the title suggest, it is focused on Bayesian optimization.

The book by D. Packwood addresses the lack of hands-on examples in Garnett's book because Ch. 2 goes through every detail to implement the GPs prior used in the Bayesian optimization and along with the companion R code. A while back I reworked the R code in Julia; I'll revisit this for sure.

I actually really love the book by D. Packwood because it's concise and directly applicable to my domain area. It's a bit expensive for how brief it is (42 pages), but worth it. The best part is you can read it over and over again because it is so short, making a great reference. It does assume a good foundation probability and at times the symbol choice and nomenclature are confusing or unclear.

Credit Kat Bailey

There are probably also tons of good talks on GPs. I found this one by Neil D. Lawrence which seems very detailed and in-depth. Not to mention very clean slide deck. There is also this quick and simple implementation from scratch in Python by Katherine Bailey. I particularly like her meme

Footnotes

---

1. I always get this confused, but when I say multivariate I think of multiple inputs single output, where multivariable, is multiple inputs and outputs. ↩

2. You can actually access Ch. 2 for free from the publisher in the provided product sample file here. ↩

Reuse and Attribution

Stranded on a Quantum Branch

I'm going to try to cover an important concept that was put forth in quantum mechanics during the 1970's, which is quantum decoherence. In short decoherence is the process that provides a way to discuss the transition of a system from one described using quantum theory to one which can be explained using classical theories. It basically allows us (i.e. the scientist) to say something about why electrons in an atom are best described quantum mechanically yet a ping-pong ball is not¹. One thing that has piqued my interest in this topic again is a the whiteboard session Sean Carroll and Timothy Nguyen had on Timothy's podcast The Cartesian Cafe. The focus was on many-worlds interpretation, but what stood out to me was how it seemed many-worlds viewpoint intrinsically includes the idea of decoherence. After some reading about the history it seems this is indeed the origin.

Comment

It may be the case that I conceptual get things wrong about many-worlds and decoherence. My goal is to try and use this post as a recitation of my understanding from the video.

What is Decoherence?

Starting with a system in a quantum superposition state, which can be written as a linear combination:

$$|\Psi \rangle = a|\psi_1 \rangle + b|\psi_2 \rangle + c|\psi_3 \rangle + ...$$

Here, $|\psi_1 \rangle$, $|\psi_2 \rangle$, $|\psi_3 \rangle$, etc., represent the different states in the superposition, and $a$, $b$, $c$, etc., are the complex coefficients determining the probability amplitude of each state. In many-worlds its the probability of being on a "branch" and in traditional views its the probability of the collapsed² wavefunction in that state.

Now, suppose this system interacts with the environment, represented by the state $|E\rangle$. Before the interaction, the total system plus environment can be described as the product state $|\Psi ⟩ \otimes |E \rangle$. However, due to the interaction, the system and environment become entangled:

$$|\Psi ⟩ \otimes |E \rangle \rightarrow \alpha|\psi_1 \rangle |E_1 \rangle + \beta|\psi_2 \rangle |E_1 \rangle + \gamma|\psi_3 \rangle |E_1 \rangle + ... + \zeta_{ij}|\psi_i\rangle|E_j\rangle $$

The coefficients $\alpha$, $\beta$, $\gamma$, and are now also associated with specific environmental states, e.g., $|E_1 \rangle$, and are not separable. The system and environment have become correlated, i.e., entangled, through the Schrödinger equation.

Decoherence in the Many-Worlds Interpretation

The Many-Worlds Interpretation of quantum mechanics interprets decoherence in a unique way. From my understanding, each term in the superposition after decoherence can be viewed as a separate "world". Moreover, the observer also becomes entangled with the system and environment:

$$\begin{align*}|\Psi \rangle \otimes |E \rangle \otimes |\text{O} \rangle \rightarrow& \aleph\,|\psi_1 \rangle |E_1 \rangle |\text{O}_1 \rangle + \beth\,|\psi_2 \rangle |E_1 \rangle |\text{O}_1 \rangle +\\ &\daleth\,|\psi_3\rangle |E_1\rangle |\text{O}_1⟩ + ... + \Omega_{ijk} |\psi_i\rangle |E_j\rangle |\text{O}_k\rangle\end{align*}$$

Here, $|\text{O}_1 \rangle$, etc., represent the state of the observer who has become entangled with the system to be in the state $|\psi_1 \rangle$, $|\psi_2 \rangle$, etc. Keep in mind the Schrodinger equation is what evolves $|\Psi⟩ \otimes |E\rangle \otimes |\text{O} \rangle$. This leads to a superposition of all possible outcomes, each corresponding to a different branch in the many-worlds interpretation. It's not entirely clear to me if I'm getting the representation of the environment and observer right.

So what does this represent? Each component would be a branch where the observer is in their respective world, it appears as though the system has collapsed into a definite state. However, from an "God's eye" perspective, all outcomes have occurred, and all observer states exist, each in their respective branch, 🤯.

Question

There is one thing that I'm still very confused about. It could be that I'm getting the details above wrong, however, what is the size of the Hilbert space for $|E\rangle$ and $|O\rangle$? Because if it is large then most certainly there will be branches where things are very strange. Phrasing it another way, if the probability amplitudes for the branches are heavily distributed to say, states $|\psi_i\rangle |E_1\rangle |\text{O}_1\rangle$, then we may confidently say all outcomes/branches will appear to have identical states for the observer and environment. But what if this is not the case? What if there is an observer state where all electrons in the observer's body corresponds to un-bonded atoms, would there be a branch where the observer is a dissociated mess of nothing consistent with a conscious human? Admittedly, even if this is indeed true, the chance that observer is on this branch can be taken to have such a miniscule probability amplitude as to be irrelevant, however, its not zero! To me this is somewhat reminiscent to Schrodinger's cat.

The Role of the Hamiltonian in Decoherence

How does entanglement and decoherence come about? In this context, the Hamiltonian is an essential element. It is the operator that corresponds to the total energy of the system, and it dictates the time-evolution of the system via the Schrödinger equation, that is, how the state of the system changes over time. When a system interacts with its environment, the Hamiltonian leads to an entangled state, which causes the state, $|\Psi\rangle$, of the system to become decoherent.

Trying to Summarize Many-Worlds

In many-worlds, each potential quantum state within the superposition corresponds to a separate "world". What the traditionalist call a quantum measurement is viewed as a branching where all possible outcomes materialize in some Hilbert space³. Decoherence, in this perspective, is the mechanism (via the Schrödinger equation). Once decoherence has occurred, the various outcomes can no longer interfere with each other. It's this process that effectively 'splits' these worlds.

The entanglement caused by the interactions between a quantum system, observer, and surrounding environment, as governed by the Schrödinger equation, leads to decoherence. This process is at the core of the transition from quantum to classical behavior, as well as the basis of the many-worlds interpretation of quantum mechanics. It's an ongoing area of study and a fascinating frontier in our quest to explain reality.

References

[1] Timothy Nguyen, Sean Carroll, The Many Worlds Interpretation & Emergent Spacetime, The Cartesian Cafe podcast. Accessed July 12, 2023. https://www.youtube.com/watch?v=LGtimjuA5gA.

Footnotes

---

1. My understanding is that a ping-pong ball has all the degrees of freedom entangled with it's environment, while an atom does not, although it can once the state of an observer comes into play. An important distinction is that things become entangled with the environment, not only interact, because in principal it is possible to have interacting systems who's states are just products of the individual states, i.e., product states. This latter scenario would mean that it would be possible reverse things to get back the individual quantum states. This is like doing the double slit experiment in reverse such that you reconstruct the wavefunctions of the electron and the observer before they interact, which never happens especially if you taken the viewpoint of wavefunction collapse. ↩

2. This is whats referred to as the measurement problem, because if we take the expectation value for some observable/operator like position operator, $|\psi(x)|^2 = \langle \psi |\hat{x}| \psi \rangle$, this only tells us the probability of measuring something at $x$. The problem is that this is not considered a physical process (i.e., the Schrodinger equation), so we need mechanism to localize to a specific state in the superposition of $|\psi\rangle$. Mathematically we can do this with projection-valued measures which introduces a way to describe measurement. The important point to keep in mind this is not part of the postulates of quantum mechanics and therefore is more like a addendum. Many-worlds has this baked via the perspective of branching. ↩

3. The more fundamental question that arises for me is, is Hilbert space "The Universe"? Is it what is underlying reality? Basically is Hilbert space the fabric of the universe? ↩

Reuse and Attribution