I was thinking about how universal these foundation MLIPs are, or more specifically, what universality actually means. Some use "foundation" or "pre-trained," which is probably more accurate, i.e. a training recipe of broad data and then deploy, whereas universal would be a potential energy surface (PES) that still behaves for any chemistry regardless if it was captured in the training data. The word fight is mostly vocabulary. Still, the blunt question is, what does "universal" get you once the chemistry no longer resembles anything in the training set, where out-of-distribution means that local neighborhood was not an environment the fit saw.
So here is the stunt I came up with: take one cell and every element from hydrogen to francium, $1 \leq Z \leq 87$, put them in a periodic box at equal atomic fractions, attach a foundation MLIP with coverage, and ask how sampling different configurations via swapping sites and relaxing the cell samples the PES.
This is not some insane equiatomic high-entropy alloy ... although it is an insane chemistry ... but it mixes metals, nonmetals, halogens, noble gases, lanthanides, and radioactive species that barely exist long enough to sit in a box. The chance anyone synthesizes anything close is next to nil. For fun let me call this fictitious initial state the Everything Bagel Phase. Phase is loose here, it names the starting configuration, not a claim that 87 elements form a stable homogeneous thermodynamic phase.
The digital experiment
Equal mole fractions, $x_i=1/87$, so $N=87m$ with $m$ copies of each element. The probe3 below uses $m=1$ ($N=87$), which may introduce finite-size noise effects. A serious run wants tens or hundreds of atoms per species but not trying to spend more than few minutes on this with my compute.
I kept the composition closed ... and of course we can't capture nuclear decay since we have classical nuclei on a learned PES, no explicit electrons or plasma chemistry. Also we are working with periodic boundaries. The search below is a labeling on this PES, so it is not a claim that the cell found the thermodynamic ground state of an 87-component mixture.
The oracles here are just whichever few MLIP ASE calculators will take the whole cell. The calculator has to take every $Z$ from 1 through 87, and the thing to note on the Matbench Discovery diatomics page is that Po, At, Rn, Fr, and Ra are already off the scored curves so the metric stays compatible with Materials Project compositions, the same cut MACE-MPA-0 [1] and M3GNet [2] follow when their element tables skip $Z=84$ through $88$, which puts those models out on this cell. I stayed on the three that accept the full cell, mattersim_v1_5m, orb_v3 (orb-v3-conservative-inf-mpa), and orb_v3_direct_inf (orb-v3-direct-inf-mpa) [3], with orb-models 0.5.4. Both ORB models one-hot atomic number out to 117, and their reference-energy tables are filled through Ra, with Po, At, Rn, and Fr in the table and a filler near $-4.9$ eV from Ac onward. Open MatterSim v1.0.0-5M [4] is one of the three, its embedding runs to $Z=94$ and its paper describes support for the first 89 elements, and the H through Fr cell has a finite energy on all three. The swap and cell-relax loops are in the footnotes.3
Random packing at some guessed "average density" mostly tests overlapping cores. There is no meaningful average density for H, Xe, W, Hg, F, and Fr together I can think of. A less pathological path is a radius-aware pack at low density, then a soft-repulsive cleanup, then the MLIP. Figure 1 is one instance of H through Fr ($N=87$) in a cubic cell of side $13.18$ Å ($V=2290.6$ Å$^3$), density from a covalent-radius volume proxy at packing fraction $0.64$ ($\rho \approx 6.64$ g/cm$^3$), and the soft-overlap push got the shortest pair to about $1.44$ Å.
|
| Figure 1. One atom of each element H through Fr in a soft-overlap-relaxed periodic cell. Left, colored by atomic number $Z$. Right, the same cell colored by chemical class. |
Local minima searching
Starting with a trial, which is one proposal of two random atom pairs swapping labels, then FIRE relaxes the cell and the positions together on a FrechetCellFilter at $P=0$, and the state updates only when $E/N$ drops. The shortest pair permitted is at least $1.0$ Å, and the volume is constrained to stay between $0.45$ and $2.2$ times the initial volume.
There are sixty trials on one seed for each of mattersim_v1_5m, orb_v3 (conservative, unlimited neighbors), and orb_v3_direct_inf (direct forces, same cutoff), so one hundred twenty pair swaps were proposed per model, of which MatterSim kept fourteen, ORB v3 conservative kept thirteen, and ORB v3 direct kept fifteen. Rather than compare absolute $E/N$ across models, in Figure 2 the drop from each model's own start against accepted trials is shown. MatterSim finishes $0.20$ eV/atom down, its largest single drop being Mg with S and Re with Ni ($-0.042$), ORB v3 conservative finishes $0.19$ eV/atom down, its largest single drop being Pb with Y and Dy with W ($-0.043$), and ORB v3 direct finishes $0.16$ eV/atom down, its largest single drop being Mo with As and Sn with Ni ($-0.031$), with a short further relax of those three landing at $-4.949$, $-5.191$, and $-5.137$ eV/atom.
The first trial is the same on all three, Ti with At and Eu with Ni, kept by MatterSim and by the direct model, and the conservative model leaves the state unchanged. Density barely changes, and those values and shortest/largest first-neighbor distances are shown in the table below ... just as point of reference, stainless steel is about $8$ g/cm$^3$.
| Packed | MatterSim | ORB cons. | ORB direct | |
|---|---|---|---|---|
| Density (g/cm$^3$) | 6.64 | 6.66 | 6.68 | 6.60 |
| Shortest pair (Å) | 1.77 | 1.75 | 1.79 | 1.72 |
| Largest first-neighbor pair (Å) | 3.33 | 3.40 | 3.43 | 3.47 |
Because every element appears once, a partial radial distribution function (RDF) by element is a single distance, so the structure info is shown in Figure 3, each entry is how many atoms of one class have the other class as their closest neighbor.2 MatterSim piles metals next to metals ($11$ to $16$), ORB conservative sits in between, and ORB direct does not follow that metal-metal rise, while lanthanide-halogen contacts fall from $5$ on the packed cell toward $1$ under ORB direct. Alkali-halogen stays rare, and metal-halogen is zero for every model. With one atom per element these counts are small, so this is a packing-and-disagreement check, and not so much a motif one.
The final structure in Figure 4 is the MatterSim cell after those sixty trials, $-4.949$ eV/atom at $\rho \approx 6.66$ g/cm$^3$. Sixty trials are a local sample of the labeling, not a ground state.1
|
| Figure 2. Sixty trials on the packed $N=87$ cell, plotted as accepted trials only. Each trial swaps two random pairs and FIRE-relaxes cell and positions. The curves are the energy drop from each model's own start for MatterSim, ORB v3 conservative, and ORB v3 direct. Leader labels mark the first kept trial and each model's largest single drop. |
|
| Figure 3. Nearest-neighbor class pairing after sixty trials. Each panel is a count matrix, row is the center atom's class, column is its closest neighbor's class, for the packed start and each model's incumbent. Class abbreviations marked on the axes are defined in the footnotes. |
|
| Figure 4. MatterSim cell after sixty trials, $E/N\approx -4.949$ eV/atom, $\rho\approx 6.66$ g/cm$^3$. Left, colored by atomic number $Z$. Right, the same cell colored by chemical class. |
What the swaps say
Of course trying to synthesize the Everything Bagel is a stunt rather than providing utility. My main interest was just to see what foundation MLIPs would do if handed some ridiculous chemical system cell. I also was curious about the agreement between them, although in this case its just really two types and one of a different flavor. The MLIPs did return finite energy, and under the volume constraint the drop from each model's own start seemed reasonable and controlled. While the kept atom pairs are not the same, and after the first trial those three searches are already on different structures, so there is a split in the search/acceptance.
I'm not entirely sure this answered my initial question about how well these would do with something so out of distribution, although truth is my question may have actually not really been well posed because there is nothing to validate against. But it is an interesting exercise about MLIPs and their behavior under extreme conditions.
Footnotes
-
You can view and download the same final cell as Figure 4, rotatable in MatterViz. ↩
-
In Figure 3, Ae* is alkaline earth, Ln† is lanthanide, and MG‡ is other main-group (B, Al, Si, P, S, Ga, Ge, As, Se, In, Sn, Sb, Te, Tl, Pb, Bi, Po). ↩
-
Core ASE loops (calculator attach omitted) code block below. Each swap trial relaxes the cell and the positions together. ↩↩
ASE Loops
from ase.optimize import FIRE from ase.filters import FrechetCellFilter def relax(atoms, fmax=1.0, steps=40): """FIRE on cell and positions at zero pressure.""" filt = FrechetCellFilter(atoms, scalar_pressure=0.0) FIRE(filt, logfile=None, maxstep=0.04).run(fmax=fmax, steps=steps) return atoms def swap(numbers, rng, n=2): """Swap n random pairs of species labels.""" out = numbers.copy() for _ in range(n): i, j = rng.choice(len(out), 2, replace=False) out[i], out[j] = out[j], out[i] return out
References
