CrystaLLM

A recent preprint, github repo, and web-app just dropped that makes use of a LLM trained on CIF [1]. Its pretty nifty utility and is different from the recent Microsoft paper [2] on the MatterGen approach to making stable crystal phase predictions.

I haven't tested out CrystaLLM that extensively but from what I can see its going to be a nice research tool for computational materials scientist. I mainly waiting for an API key so I can make calls and then do other things with the generated crystal structures. Here is a demo of a screen recording I took, showing the web-app:

I'm excited to see how this tool improves my computational workflows. There are two things I noticed need some improvement. The first is the inference times, it can easily take up to 45 seconds or more to generate a crystal structure. The second is more fundamental in that no physics is used to inform the structure generated, its strictly based on the CIF database used for training.

References

[1] L.M. Antunes, K.T. Butler, R. Grau-Crespo, Crystal Structure Generation with Autoregressive Large Language Modeling, (2024). arXiv.

[2] C. Zeni, et al., MatterGen: a generative model for inorganic materials design, (2023). https://www.microsoft.com/en-us/research/publication/mattergen-a-generative-model-for-inorganic-materials-design.

Reuse and Attribution

Materials Project REST API w/ Julia

The Materials Project (MP) is one of the most successful computational materials science databases for exploration of materials properties [1]. On top of that, it's also large enough to do some serious machine learning for materials design and discovery. There are for sure limitations on the data generated¹, but I'm not going to touch on that. What I'll focus on is actually how to grab this data via the API. The reason I'm creating a post is:

1. To remind myself on how to use HTTPS request for a REST API.
2. Show how straightforward it is to do in Julia.

For most, the way to access the MP data is to use the python MPRester package, which integrates with the very useful pymatgen package. However, I do a lot in Julia and although it is very easy to install python packages and use them from within Julia, it adds an additional dependency and overhead. So here I'll so how simple it is to use the REST API with only two Julia packages: HTTP and JSON. Both of these packages are implemented in pure Julia and are very mature within the Julia package eco-system, so one does not need to worry about supported operations. You can install these packages in the REPL with:

using Pkg
Pkg.add("HTTP")
Pkg.add("JSON")

Background

I was not familiar with REST API until about 2 years ago. A REST API (Representational State Transfer Application Programming Interface) is a set of protocols and standards used for exchanging data between systems. It utilizes HTTP requests to access and manipulate data, which can be in a variety of various formats but typically in JSON or XML. The most common use is to enable interactions between client and server in web applications, allowing for operations such as retrieving, updating, or deleting data stored on the server. Its a favorable architecture because it is scalable and performant, while also standardizing communication among different systems.

Since it uses HTTP protocol almost anything connected to the internet can use a REST API. Furthermore any modern computing language that has a library with functions that support HTTP can by defacto become a API. Thus its very easy to implement calls in Julia.

Creating a function to grab an MP entry

We can now implement a function that grabs the summary data for the a MP entry id. To use the MP REST API we need to have a base url, endpoint, and an operation. The base url for the materials project REST API is just: https://api.materialsproject.org. What is an endpoint? An endpoint is just a particular location and is associated with a particular operation or set of operations that can be performed on a resource, such as retrieving, creating, updating, or deleting data. These endpoints are accessed through standard HTTP methods like GET, POST, PUT, and DELETE. Since we have no admin privileges, we can only use GET.

Okay, there is one last thing we need. Many APIs require authentication, meaning you need to be an approved user. To do so they typical use a unique digital key, which is nothing but a token that consist of symbols². To get a MP api key, you need to sign-up. We now have everything we need.

Note

For this function I'm just going to use the summary endpoint which provides a fairly comprehensive dataset for a materials project ID. You can modify the endpoint to get more specific data (i.e., /materials/thermo/ )

function get_mp_summary(id::String, api_key::String, all_fields=true)
    base_url = "https://api.materialsproject.org"
    endpoint = "materials/summary/$(id)?_all_fields=$(all_fields)"
    query_url = joinpath([base_url,endpoint])
    headers = ["accept" => "application/json", 
        "X-API-KEY" => api_key]
    response = HTTP.get(query_url,headers)
    data = JSON.parse(String(response.body))
    return data
end

As you can see its a very small amount of code. The endpoint variable provides the specifics about our query, which points to a materials project id, and then uses the ? to indicate a new query that states if all the data fields should be returned or not. In this case the query for all fields is yes (i.e., true). To create the HTTP url we just combine everything into query_url. The next aspect is the header variable, which is a dictionary that specifies the we are expecting a JSON formatted data and the value of the api key. Finally, we make the HTTP request and then parse the returned JSON to a Julia dictionary. Thats it!

The data

What we get is a fairly deep dictionary structure, so its useful here to go through at least the first two layers. I'll illustrate for mp-510604, which is Mn$_2$O$_3$. The first key for the dictionary is the data:

data = get_mp_summary("mp-510604",MP_API_KEY)
keys(data)

KeySet for a Dict{String, Any} with 1 entry. Keys:
  "data"

The data key only has a single entry that is a Dict{String,Any}, this is where all the material structure and property data is. So we now want to go deep into the data, lets list all the keys in the Dict{String,Any} for data:

keys(data["data"][1])

KeySet for a Dict{String, Any} with 70 entries. Keys:
  "e_ionic"
  "chemsys"
  "weighted_surface_energy_EV_PER_ANG2"
  "material_id"
  "homogeneous_poisson"
  "deprecated"
  "shape_factor"
  "uncorrected_energy_per_atom"
  ⋮

Once you understand the structure of the data, you can then proceed how you intend to use the MP, thats it. As noted earlier, you can change the endpoint to look at other material properties that are calculated from the DFT data.

Footnotes

---

1. One of my pet peeves is that DFT calculations have become so revered that they are often used without sufficient caution or scrutiny. As numerical methods improve, with better functionals for exchange-correlation (XC) and corrections such as DFT+U, the uncritical acceptance of large datasets using standard DFT calculations, i.e., GGA or meta-GGA, seems questionable. My bias is that the quantum physics of materials is in truth a many-body problem as well as not merely a ground-state one. Thus, if you're using large "approximate" datasets to train ML models, then you're likely to encounter problems when moving away from in-silico! Feel free to correct me. ↩

2. My guess is the token is actually just a public or private ssh-key for the REST-API server that is assigned to you. ↩

References

[1] A. Jain, et al., Commentary: The Materials Project: A materials genome approach to accelerating materials innovation, APL Materials 1 (2013) 011002. https://doi.org/10.1063/1.4812323.

Reuse and Attribution

Materials Informatics Book

Should I write a technical "how-to" textbook?

Why am I asking this question, well, I've come to somewhat of a cross roads in materials informatics. I have digested a lot of research papers and review monographs [1-4], but constantly run into the following problem:

How do I implement the technique discussed, reproduce the results, and extend it towards my specific domain topic?

The answer over and over is, these resources aren't going to enable that or provide a way to do so. Yes in some cases there are Github repos that provide the source, but what I find is the data input pipeline is so complicated or convoluted that figure out how to get my data to work is too much effort. It would just be better to actually implement the model from scratch based on my specific data preprocessing/pipeline.

So why would me writing a address this challenge? Well, for one no one else has written such a book. You have three textbooks available on this topic:

• Isayev, O., Tropsha, A., & Curtarolo, S. (2019). Materials informatics: Methods, tools, and applications. John Wiley & Sons. url.
• Rajan, K. (2013). Informatics for materials science and engineering: Data-driven discovery for accelerated experimentation and application. Butterworth-Heinemann. url.
• Kalidindi, S. R. (2015). Hierarchical materials informatics: Novel analytics for materials data. Elsevier. url.

There are a few other monographs [3-4] that try to focus on specific subdomain areas of materials science. The books above are actually pretty good if your looking for a foundational understanding of data science and machine learning applied to materials science and engineering. The problem with those books is they are more reference text for people involved in materials informatics. They won't help you get going in front of a computer or information systems.

This is what is missing and I want to provide a solution. One reason is that I'll learn more by writing a book. The second reason is I think the tools are now available to make writing this type of book much smoother. Writing literate programming is a regular thing (e.g., Jupyter notebooks) and therefore writing while coding is straightforward, usually.

The dilemmas I face are what programming language to use, what framework to write in, how much to cover, and best examples/case studies to use. For the programming language its between Python and Julia. I'm torn because I prefer to use Julia but Python is more broadly adapted and has very mature and standard packages (e.g. scikit-learn, pytorch). For the framework its also a challenge. I'm favoring Quarto at the moment and it won't matter if I use Julia or Python. Similar case for Jupyter Books. I haven't used Jupyter Books and I'm not too interested in adapting the MyST. There are other options to explore as well, such as Books.jl which is geared towards PDF and website generation.

For the content the book would cover, I need to be very thoughtful. My rough outline would be something like:

1. What is data and information
2. Describing data: Probability, Statistics, and Visualization
3. Processing & Transformation of data
4. Pattern extraction and reduced representation
5. Regression, optimization, and prediction
6. Neural network models
7. Autonomous solution seeking

The first 3 chapters are probably self-explanatory, the chapters 4,5, and 7 would correspond to unsupervised, supervised, and reinforcement learning. This is a moving target so it would change based on my particular interest and focus.

The key point is that each chapter provides the background and the code to actually do something at the computer. My goal would also be to try to do as much as possible from scratch. Meaning, that if it made sense I would actually have a section in the chapter on neural networks that builds the layers, does forward and backward propagation, and trains using minimal packages (e.g. NumPy). Why do so if in the end we all are going to implement and deploy using pytorch or `tensorflow? Because for most the act of doing is what solidifies understanding and comprehension. After coding up a simple NN, when someone talks about backpropagation, you'll know what is actually being down, at least from the most minimal implementation ¹.

Status

I have yet to really start writing, but plan to have some kind of a draft by end of 2024. My goal would be to make the draft available online first. For the physical copy I intend to go the self-publishing route using Amazon Kindle services.

As for the applications and case studies used, well, I really want these to be real in the sense that they have either been done in academia or industry. I want to try and avoid "toy problems" not because they aren't useful but because I want to avoid the issue with a creating a insurmountable barrier to applying whats in the book the readers specific interest/problem.

My hope for this potential book is that grad. students and researchers who want to get into this area, but don't have any hands-on experience, can more easily do so by working through the book. The book obviously wouldn't be at the forefront of research methods, but it would be as if you were taking a graduate level lab at a major university.

References

[1] K. Takahashi and L. Takahashi, "Toward the Golden Age of Materials Informatics: Perspective and Opportunities", J. Phys. Chem. Lett., vol. 14, no. 20, pp. 4726-4733, May 2023, doi: https://doi.org/10.1021/acs.jpclett.3c00648.

[2] C. Li and K. Zheng, "Methods, progresses, and opportunities of materials informatics", InfoMat, p. e12425, Jun. 2023, doi: https://doi.org/10.1002/inf2.12425.

[3] T. Lookman, F. J. Alexander, and K. Rajan, Information science for materials discovery and design. Springer, 2015.

[4] I. Tanaka, Nanoinformatics. Springer, 2018.

Footnotes

---

1. There are different numerical implementations to achieve backpropagation and the book I would write showing how to implement a NN would focus on the most basic approach. ↩

Reuse and Attribution