Entropy: Thinking Beyond Disorder

Every so often, I casually refer to terms like entropy or free energy and forget to really think about what the fundamental meaning of those words are. So, with this post, I aim to focus on the meanings of these terms (i.e., without involving math), particularly entropy.

Entropy: More Than Just Disorder

Entropy is often described in terms of 'disorder' or 'randomness', but these terms, in my opinion, can be misleading and confusing. When we consider epistemology of a system, that is the study of what we can know about a system, entropy is meaningful only from a statistical perspective. This is because, scientifically, we understand that everyday matter¹ is made up of microscopic constituents (i.e., atoms or molecules), yet it's impossible² to know all details about these constituents. Therefore, we rely on a statistical representation of their collective behavior. Entropy is fundamentally about the number of ways a system composed of matter (or information) can be arranged at the microscopic level while still appearing the same at the macroscopic level. It measures the diversity of microstates corresponding to a given macrostate.

As a result, I think it's best not to conceive 'disorder' in the traditional sense, where objects are randomly spread out in a chaotic fashion, but rather as the numerous microstates a system can adopt while maintaining familiar macroscopic properties, such as temperature or pressure. Thus, entropy reflects a lack of specific information³ about individual microstates of a system, while still enabling knowledge of macroscopic properties.

The 'randomness' in entropy relates to the probabilistic nature of microstates. At the microscopic level, the exact configuration of particles in a system follows the laws of probability. This aspect of randomness is key to understanding why macroscopic properties of systems emerge as averages over many microstates. Simply put, we can never truly know what microstate the system is in, only the expectation value of a given microstate while considering all other microstates.

Nature's Tendency To Maximize

What we empirically observe in nature, is that matter at the microscopic and mesoscopic scales, tends towards the largest possible configurational space of microscopic states. This tendency to maximize the number of accessible microstates drives the natural progression towards states of higher entropy and thermodynamic equilibrium. Recall that entropy relates the microstates to the macrostate, and thus nature aims to maximize the number of microstates corresponding to an equilibrium macrostate.

To recap, entropy, often encapsulated in terms like 'disorder' and 'randomness', is more easily thought of as a nuanced measure of the probabilistic distribution of microstates in a system. Understanding entropy in this way helps better appreciate the intricate relation between microscopic thermodynamics (i.e., statistical mechanics) and macroscopic thermodynamics.

Bonus: Free Energy

Free Energy is another term worth digesting. It's best understood as the energy freely available in a system to do work, hence the name free energy. This concept is intertwined with the energy content of a system and its capacity to perform work. Under the laws of thermodynamics, free energy represents the principle that, while energy cannot be created or destroyed, it can be transferred or transformed. Thus, free energy is a conceptual tool that helps us understand how energy moves within a system and its surroundings.

Footnotes

---

1. A famous thought experiment that attempts to circumvent the idea of not being able to determine what every constituent particle is doing is Maxwell's Demon. It was used as a way to violate the second law of thermodynamics for an adiabatic system, $\Delta S \ge 0$. ↩

2. Our best scientific theories, confirmed by experiment, indicate the fundamental constituents of matter are more elementary particles than atoms and molecules. However, for the most part, our interactions with the physical world at the smallest scales can be well described by thinking in terms of electrons, atoms, molecules, and the like. ↩

3. In information theory, entropy is used to quantify details about information. So that rather than the microscopic states of matter, one thinks about how information can be represented. This was one of the seminal results put forth by Claude Shannon and I believe inspired by John von Neumann. ↩

Reuse and Attribution

Unofficial way to use Phonopy with ASE

If you've ever done any phonon calculations with VASP, then you are most certainly familiar with phonopy. In many scenarios you may want to use interatomic potentials for the forces instead of ab-initio¹ calculated forces. This allows you to more quickly converge and calculate ground-state bandstructures and thermal quantities. You can even look at finite temperature effects with lammps using fix phonon. Phonopy does have a good amount of support for several DFT² and LAMMPS.

I recently have been using atomic simulaton environment (ASE) to leverage some of its calculator interfaces. While ASE does have built in finite-difference method support for phonon calculations, phonopy is more comprehensive. So I wanted to explore how much effort it would take to get the two to "talk", turns out its extremly straightfoward.

The Workflow

First things first, you'll need to run pip install ase and pip install phonopy, its also ideal to install the Brillouin zone path library via pip install seekpath. Now, let me create a structure with ASE and define a calculator, for simplicity I'm going to use the effective medium potential (EMT) calculator thats built into ASE.

from ase.build import bulk
from ase.calculators.emt import EMT
calculator = EMT()
structure = bulk('Cu', 'fcc', a=3.6)

Now to import the phonopy python API tools:

from phonopy import Phonopy
from phonopy.interface.calculator import get_force_sets
from phonopy.structure.atoms import PhonopyAtoms

The import thing to notice is we need to use PhonopyAtoms to create the supercells and displacements. At first this may seem like a big hurdle, in that you'll need a lot of wrapper code, but this is not the case. Simply do:

phnpy_struct = PhonopyAtoms(
    symbols=structure.get_chemical_symbols(),
    positions=structure.get_positions(),
    cell=structure.get_cell(),
    )

Now we construct our phonopy object:

phonons = Phonopy(
    phnpy_struct,
    supercell_matrix=[[5, 0, 0],
                      [0, 5, 0],
                      [0, 0, 5]],
    )

Then we need to specify the displacement magnitude that is used when using the finite-difference method.

phonon.generate_displacements(distance=0.5)

Note

One thing I've noticed is that the distance values to converge the phonon calculations are significantly larger than I've used in the past. Usually displacements of 0.01-0.05 angstroms, but I'm noticing 0.5-0.8 angstroms are need to converge! Seems concerning since we need to remain within the harmonic regime.

Okay straightforward, now what about the forces? How do we get them into our phonopy object phonons. Turns out this is also relatively simple:

import numpy as np
sets_of_forces = []
supercells = phonons.get_supercells_with_displacements()
for i,d in enumerate(supercells):
    # Convert back to ASE
    d_ase = Atoms(symbols=d.get_chemical_symbols(),
        positions=d.get_positions(),
        cell=d.get_cell()
        )
    d_ase.set_calculator(calculator)
    forces = d_ase.get_forces()
    sets_of_forces.append(forces)

Now that we have the forces, fortunately, all we need to do to get the force constants to calculate the phonon properties is:

sets_of_forces = np.array(sets_of_forces)
phonons.forces = sets_of_forces
phonons.produce_force_constants()
phonons.save()

The last command saves the details for phonopy which I believe you can use the outputs to run the standalone phonopy command-line tool. Now we can view the phonon bandstructure:

phonons.phonons.auto_band_structure(plot=True)

which gives:

Phonon bandstructure of Cu FCC using ASE and Phonopy together.

Using the phonopy python api you should also be able to calculate the DOS and thermal properties.

Footnotes

---

1. If the forces on atoms are calculated using the Hellmann-Feynman theorem, then they are referred to as first-principles or ab-initio because they arise from the expectation value of the Hamiltonian about some continous parameter, ex., $\mathbf{F}_i = -\frac{dE_i}{d\mathbf{r}_i} = \langle \psi_{\mathbf{r_i}} \lvert \frac{d\hat{H}_{\mathbf{r}_i}}{{d\mathbf{r}_i}} \rvert \psi_{\mathbf{r_i}} \rangle$. ↩

2. Density functional theory is the primary compute for most high-quality phonon calculations. ↩

Reuse and Attribution

Calculating Phase Diagrams

Why do I love phase diagrams so much? I've always been fascinated by these relatively cursory-looking plots that show where the phases of matter at different thermodynamic conditions are stable. I remember my excitement in the intro lecture and lab, MATE 25 at SJSU, where in the lab we constructed points on the phase diagram of a lead alloy system. I thought this was the coolest thing that we could build these maps and then use them later to determine what phase a material would be in a given temperature and composition. At the time I had no thermodynamics course work so I didn't realize the underlying driving force of this phenomenon nor did I realize you can calculate these phase diagrams using the CALPHAD method. When I got to grad school I took the required thermodynamics course and then I was even more blown away at how powerful this framework was. I was particularly lucky because the course was taught from a "grassroots" approach where everything was built from the ground up given a set of postulates (you can see the book by H. Callen to get the gist). 

So what does a phase diagram look like and how does one use it? Here I'm going to leverage the excellent Python library pycalphad [1]. Which lets you construct phase diagrams using thermochemical databases, if available. Let's take an example of Cu-Ni system, you can work through the CALPHAD calculation with pycalphad in this google colab notebook. Here is the  binary phase diagram predicted for Cu-Ni:

Phase diagram predicted using cost507.tdb and pycalphad.

How do you read this? Well the blue and yellow points indicate the equilibrium phase boundary. The regions between phase boundaries indicates what phases are in eqiulibrium and how much (see my old notes on tie-lines). So how does the prediction look? Not very good if you consider what the textbook phase diagram looks like:

Textbook phase diagram for Cu-Ni from adapted by ref. [1] 

As we see the predicted phase diagram isn't even close to that shown in the textbook version. This is a direct consequence of the thermodynamic database. However, if we use the same thermodynamic database and look at another system like Al-Zn its much better:

Phase diagram prediction using cost507.tdb. Not bad!

This is much better when you compare it to the phase diagram reported in ref [2]. I just think the CALPHAD approach is so cool in that you just need thermodynamic descriptions of various phases of a material system to make predictions about stability regions. To make a CALPHAD calculation work you would need the following:

1. Thermodynamic data/descriptions of individual phases such as enthalpy, entropy, and Gibbs free energy.
2. The phases that could exist and their structure.
3. Well-defined reference states (e.g.pure metal) allow for consistent and accurate calculations.
4. Interaction model/parameters to describe mixing behavior of components/species.

The CALPHAD framework then enables the building of a model with these inputs to predict the phase equilibria and diagrams. You can also calculate other thermodynamic properties like heat capacity or even more useful is that the free energy models can be used within the context of phase-field simulations to evolve microstructures.

References

[1] R. Otis, Z.-K. Liu, pycalphad: CALPHAD-based Computational Thermodynamics in Python, JORS. 5 (2017) 1. https://doi.org/10.5334/jors.140.

[2] https://sv.rkriz.net/classes/MSE2094_NoteBook/96ClassProj/examples/cu-ni.html, reproduced from Callister, William D., Materials Science and Engineering: An Introduction. United States, Wiley.

[3] A. Pola, M. Tocci, F.E. Goodwin, Review of Microstructures and Properties of Zinc Alloys, Metals. 10 (2020) 253. https://doi.org/10.3390/met10020253.

https://doi.org/10.5281/zenodo.7901067

Reuse and Attribution

Atomic View of Thermal Expansion

To discuss how thermal expansion emerges in materials one can start with a simple view of the potential energy between atoms. Let us first look at the potential curve as function of separation, $R=||\bf{r}_1 - \bf{r}_2||$, between two atoms as shown in the figure below:

In the figure the equilibrium bond distance between two atoms is indicated by the minimum in the potential energy. In a system where the thermal energy is not dominant (i.e., low temperatures) the potential energy can be approximated harmonically, and therefore the displacements, $\Delta r = (r -r_{eq})$, due to the forces will be symmetric in an averaged sense. This is highlighted in the figure inset showing that the harmonic (blue) matches well with the potential curve for small $\Delta r$.

When the thermal energy in the system begins to become significant, the harmonic approximation is no longer appropriate and the displacements are non-symmetric about the equilibrium bond distance. This asymmetry gives rise to thermal expansion of a material. This is further shown in the inset with the anharmonic curve (red) showing better agreement with the potential curve than the harmonic at larger $\Delta r$.

The atomic view of thermal expansion manifest in bulk through the isotropic (i.e. volumetric) thermal expansion of a material which is given by the coefficient of thermal expansion,

$$\alpha = \frac{1}{V}\frac{\partial V}{\partial T}_P $$

where $V$ is the volume, $T$ the temperature, and $P$ indicates the derivative at constant pressure. It is also common to refer to this as the coefficient of thermal expansion (CTE).  

The quote for this post is:

Science knows no country, because knowledge has no identity, and therefore exist to illuminate the world.
-Louis Pasteur (modified)

References

[1] The Materials Physics Companion, IoP, 1st ed., A.C. Fischer-Cripps.

Reuse and Attribution

Ideal Solution Mixing: A-B Lattice

Here we will review the ideal solution mixing model for simple A-B lattice random mixing alloy system. The first step is to recall that for any system (e.g. state or phase) we can write the Gibbs free energy as:

$$ G = H-TS $$

where $H$ is the enthalpy, $T$ the temperature, and $S$ the entropy. We now propose that we have two isolated systems, lattice A and lattice B, and we want to find the change in Gibbs free energy when the two are combined to form a lattice with both A and B sites (randomly). An illustrative example would look something like below.

The next step is to write the change in Gibbs free energy as:

\begin{align}
\Delta G^{mix} & = G_{initial} - G_{final} \\
& = \Delta H^{mix} - T \Delta S^{mix} \\
\end{align}
Notice that we are using the label $mix$ to indicate that the change in Gibbs free energy is due to the mixing of the two lattices into one (i.e. Gibbs free energy of mixing).

In the ideal solution mixing model, we first approximate that $\Delta H^{mix}$  is negligible and taken to be zero. We can think of this as meaning that we assume no change in internal energy due to the chemical interactions between A and B. The next assumption is that the change in entropy is strictly due to configurational arrangement of A and B points on the combined lattice. This means that entropic effects due to lattice vibrations or magnetic ordering are not accounted for. Thus the Gibbs free energy has a simple relation to entropy:

$$ \Delta G^{mix} = - T \Delta S^{c} $$

The next step is to define the representation for the configurational entropy. To do this we will use to facts that each microstate is probabilistic and given by the combinatorics (i.e. possible configurations ). This is compactly represented by the famous equation:

$$ S^{c} = k_b \ln \omega^{c} $$

with $k_b$ being the Boltzmann constant and $\omega^{c}$ the configuration combinatorics. For the lattice AB this is going to be given by:

$$ \omega^{c} = \frac{N!}{N_{A}!N_{B}!}$$

where $N=N_{A}+N_{B}$ and $N_{A}$ and $N_{B}$ are the number of sites of a given type. At first glance calculating the configurational entropy may not seem daunting, however, logarithms of factorials can become demanding to calculate very quickly. Fortunately enough there is an approximation provided by mathematician James Stirling that allows one to approximate logarithms of factorials and is given by:

$$ \ln N! \approx N \ln N - N $$
Using this approximation we can determine $\omega^{c}$ and distill the expression of $S^{c}$ into something that is relative compact and meaningful. Apply the approximation we get:
\begin{align}
 \ln \omega^{c} &= N \ln N - N - \left[\ln\left(N_{A}!N_{B}!\right)\right] \\
&= N \ln N - N - \left[ N_{A} \ln N_{A} - N_{A} + N_{B} \ln N_{B} - N_{B}\right] \\
&= N \ln N - N_{A} \ln N_{A} - N_{B} \ln N_{B} - N + N_A + N_B \\
\end{align}
The last three terms cancel out, e.g., $N_A + N_B = N$ and we then rewrite the first term as:
\begin{align}
\ln \omega^{c} &= \left(N_A + N_B\right) \ln N  - N_{A}\ln N_A - N_{B}\ln N_B \\
&=-\left[N_A \ln \left(\frac{N_A}{N}\right) + N_{B}\ln\left( \frac{N_B}{N}\right) \right]
\end{align}
the ratio of $X_A = \frac{N_A}{N}$ or $X_B = \frac{N_B}{N}$  are the fraction of sites on the mixed lattice with A and B sites, respectively. Let us take one further step by multiplying the equation above by $\frac{N}{N}$ to get
$$ \ln \omega^{c} = -N \left[ X_A \ln X_A + X_B \ln X_B \right] $$
Now we can write $\Delta S^{mix}$ as,
$$ \Delta S^{mix} = -k_{b} N \left[ X_A \ln X_A + X_B \ln X_B \right] $$
if we assume that the total number of N sites on the alloy lattice is comparable to the number of particles in 1 mole, i.e., Avogadro's number $N_a = \text{6.022}\times \text{10}^{\text{23}}$, then we can write the Gibbs free energy of mixing in most familiar form as:
\begin{align}
 \Delta G^{mix} &= -T \Delta S^{mix} \\
&= -T \cdot -k_{b} N_{a} \left[ X_A \ln X_A + X_B \ln X_B \right]  \\
&= \boxed{RT \left[ X_A \ln X_A + X_B \ln X_B \right]}
\end{align}
where $R$ is the gas constant given by  $k_b N_a$. We can get a sense for how the Gibbs free energy of mixing changes with temperature as shown in the graph below,

From the graph we observe two features, 1.) the Gibbs free energy of mixing for an ideal solution is a symmetric function, 2.) as the temperature is increased $\Delta G^{mix} is decreases. Not that in the graph the line(s) do not extend to zero and one, this is because these would be given by the Gibbs free energy of the reference states of lattice A and B.

Ideal solution mixing is typically not suitable for real material alloy systems and thus other approximations such as the regular solution model are used. In the regular solution model we use the same $\Delta S^{mix}$ and include a non-zero expression for $\Delta H^{mix}$. The most accurate approach for calculating Gibbs free energy of mixing for real materials is to use CALPHAD methodologies.

For this blog postings quote we will get two quotes:

"Nothing in life is certain except death, taxes and the second law of thermodynamics."
-Seth Lloyd, MIT Professor 

"In this house, we obey the laws of thermodynamics!"
-Homer Simpson, response to Lisa's perpetual motion machine

References & Additional Reading

[1] Thermodynamics in Materials Science, Robert DeHoff, 2nd ed.
[2] Introduction to the Thermodynamics of Materials, D.R. Gaskell, 5th ed.

Reuse and Attribution