Dual Numbers

A while back when I was doing some exploration of writing a simple NN code to improve my understanding of neural networks and deep learning in general, I came across dual numbers. They're a type of number that generalizes the concept of real and complex numbers. But what makes them so interesting is that they can encode both a function value and its derivative in a single number. This means that we can use them to simplify the calculation of derivatives and solve complex problems efficiently.

So how does one think of dual numbers? What's the difference between a dual number and a complex number? One way to think about dual numbers is that they consist of two parts: a scalar part and a skew part. The scalar part is just a regular real number, while the skew part is a multiple of a new number, often denoted as $\epsilon$, that satisfies the property $\epsilon^2=0$. This means that every dual number can be written as $a+b\epsilon$, where $a$ and $b$ are real numbers.

What's most interesting is that the skew part of a dual number is that it provides an approximation of the first derivative of a function evaluated at a particular point. By using the dual number representation of the function at that point, one can calculate both the function value and its derivative in one shot.  One reason dual numbers have applications in deep learning is that algebra on dual numbers provides the chain rule for calculus, therefore they can be used to compute derivatives of complicated functions involving multiple variables and interdependencies.

As an example, say I want to evaluate the function $f(x)=x^2+2x$ at $x=3$. The dual number representation of $f(3)$ is $f(3+\epsilon)=f(3)+f'(3)\epsilon$, where $f'(x)=\frac{df(x)}{dx}$. We can compute $f(3)$ directly as $f(3)=3^2+2\cdot3=9+6=15$. To compute $f'(3)$, we can take the derivative of $f$ with respect to $x$: $f'(x)=2x+2$. Evaluating this at $x=3$, we get $f'(3)=2\cdot3+2=8$. Therefore, the dual number representation of $f$ at $x=3$ is $15+8\epsilon$.

One of the benefits of dual numbers is the derivative of the composition of two functions, $f(g(x))$ requires only the derivatives of the individual functions. Specifically, if $f(x)$ and $g(x)$ are two functions, then the dual number representation of their composition $f(g(x))$ is $(f(g(x)), f'(g(x))g'(x))$. This is especially useful when dealing with complex functions involving multiple variables and complicated interdependencies.

Dual numbers are actually a useful mathematical concept because they have practical applications in a wide range of fields. It's pretty cool that one can encode function values and derivatives in a single number, which makes it possible to simplify the calculation of derivatives and solve complex problems efficiently. On my computational blog, I have an example using dual numbers to calculate the derivative of an interatomic potential, Dual Numbers Pluto blog.

Reuse and Attribution

Research hype seems problematic

The recent news  [1,2] about room temperature superconductors (RTSC) at relatively low pressures has been getting a lot of attention, both negative and positive. The negative press is coming from the fact that the PI of the study has had questionable research in the past. The PI has also been resistant to requests for sharing data and samples that were synthesized. 

If it turns out that the nitrogen-doped lutetium hydride is indeed a superconductor at ambient temperatures and pressures around 1 GPa this would indeed be a waypoint on the journey toward superconducting materials. For many, 1 GPa may seem fairly high compared to ambient pressure, which is around ~0.0001 GPa. However, the ability to engineer a coating or conduit that wraps and applies suitable compressive pressures to RTSC should be something feasible. You can think of how Corning's Gorilla glass works, which is an alkali-aluminosilicate material that uses some clever surface composition engineering to create gradients of strain to arrest microcracks/pits that form on the surface; this is done by creating compressive stresses in the material. The difference here is you wouldn't modify the RTSC material directly.

Going back to press on RTSC, I'm glad to see there is a lot of debate going on. One thing that appears to be very clear is that the peer review process for these high-impact journals is not very good. Seems to me that such an impactful article would have received the same criticism that is being displayed in the public discourse. Such criticism would have probably made it much more challenging for the authors to publish their findings as they would have had to overcome many requests for raw data by the reviewers, although I'm not sure journal editors entirely support such requests as the reviewer could be a potential competitor. What I like is that there is a lot of community review going on. Independent researchers and groups are eagerly trying to reproduce the findings and we will probably know shortly what the outcome is. Some early preprints/papers [3,4] are indicating they aren't observing the same resistivity measurements as reported in the original work; not looking too good for the controversial PI.

A parallel event that is going on in quantum computing is the ongoing debate and coverage of the quantum computing wormhole publication [5]. I've worked a bit on quantum algorithms for NISQ devices so I'm a little familiar with what can be done using them. I know nothing about research in quantum gravity or EPR=ER, but I can tell you that the initial (seems they've updated it) coverage by Quanta magazine was awfully misleading. The main message that should have been conveyed is that the simplified model being simulated facilitates the mathematical relation between the dynamics of entanglement in quantum systems and wormholes predicted in general relativity. It does not mean that running the quantum device creates spacetime wormholes in the physical lab, however, anyone reading the original article or related popular stories would be inclined to think this is what happened.

This leads me to start thinking about what is going on with hype in research. Why is it that science is becoming about what hype one can generate around the research? I've seen a lot of good posts on Linkedin commenting on this. It appears that it is strongly linked with the prospectus in securing more funding for their research. I assume the thinking behind this is that if funding agencies and program managers get excited, they won't want to miss out on all the fun! Other comments indicate that it's mostly because much of the science being done is actually not that impactful and very incremental, so things get overblown in importance and meaning. Whatever it may be it seems this is going to create huge issues in the future because popular articles on scientific research that overhype will eventually lead to serious decisions that affect both social and economic life for everyone.

References

[1] N. Dasenbrock-Gammon, E. Snider, R. McBride, H. Pasan, D. Durkee, N. Khalvashi-Sutter, S. Munasinghe, S.E. Dissanayake, K.V. Lawler, A. Salamat, R.P. Dias, Evidence of near-ambient superconductivity in a N-doped lutetium hydride, Nature. 615 (2023) 244–250. https://doi.org/10.1038/s41586-023-05742-0.

[2] H. Pasan, E. Snider, S. Munasinghe, S.E. Dissanayake, N.P. Salke, M. Ahart, N. Khalvashi-Sutter, N. Dasenbrock-Gammon, R. McBride, G.A. Smith, F. Mostafaeipour, D. Smith, S.V. Cortés, Y. Xiao, C. Kenney-Benson, C. Park, V. Prakapenka, S. Chariton, K.V. Lawler, M. Somayazulu, Z. Liu, R.J. Hemley, A. Salamat, R.P. Dias, Observation of conventional near room temperature superconductivity in carbonaceous sulfur hydride, (2023). https://doi.org/10.48550/arXiv.2302.08622.

[3] P. Shan, N. Wang, X. Zheng, Q. Qiu, Y. Peng, J. Cheng, Pressure-induced color change in the lutetium dihydride LuH2, Chinese Phys. Lett. (2023). https://doi.org/10.1088/0256-307X/40/4/046101.

[4] X. Ming, Y.-J. Zhang, X. Zhu, Q. Li, C. He, Y. Liu, B. Zheng, H. Yang, H.-H. Wen, Absence of near-ambient superconductivity in LuH$_{2\pm\text{x}}$N$_y$, (2023). https://doi.org/10.48550/arXiv.2303.08759.

[5] D. Jafferis, A. Zlokapa, J.D. Lykken, D.K. Kolchmeyer, S.I. Davis, N. Lauk, H. Neven, M. Spiropulu, Traversable wormhole dynamics on a quantum processor, Nature. 612 (2022) 51–55. https://doi.org/10.1038/s41586-022-05424-3.

Edited 27 Mar, 2023: In the original version it was stated that Gorilla glass is a borosilicate glass, this is incorrect and the post has been updated to reflect that Gorilla glass is an alkali-aluminosilicate.  Corning does produce a product called Willow glass which is a borosilicate.

Reuse and Attribution

Parsing research articles with a Zotero workflow

I started thinking about needing to document my reference and journal reading process since I may want to overhaul it later on based on the rapidly evolving tools that are coming out such as elicit.org or scispace.com.  So here is how I do things for the most part.

My go-to reference manager: Zotero

If you are anyone who is conducting research in an academic, government, or even industrial lab setting you most likely are familiar with a reference manager. The list is exhaustive nowadays, but for me, there is only one that I've consistently kept going back to and that is Zotero. The thing I like about Zotero is that it's multiplatform, open-source, no-cost, easy to use, and has a good amount of features integrated. The biggest issue is the cloud storage cost. This is really only a problem if you want to have all your attached pdfs stored on the cloud service so that you can access them using the browser interface.

To get around this limitation you can use the ZotFile extension which allows you to change how Zotero stores and renames files. If your using Google Drive or DropBox, this means you can create a folder on your cloud storage and then use ZotFile to store all linked PDFs there. If you have multiple devices with the same OS and file structure, then opening the attached files on any of your devices with Zotero desktop will work. If this is not possible with your working environment, for example, you have Windows and Linux machines. Then you will just want to make sure you create links to your local cloud storage path for the Zotero entry. You could also open the folder on your cloud storage such that anyone with the links can access the files and then add these as links in the Zotero entry. That way your file is always accessible no matter where you try to grab it from.

There is one other Zotero add-on I like to use since I do a lot of my technical writing in LaTeX. The Better Bibtex extension makes generating .bib files extremely easy and . Furthermore, it makes using your preferred bib entry naming schema consistent across all references straightforward. You also don't have to worry about keeping a .bib file updated.

Grabbing Literature

There are several ways to get journal articles so I'm not going to list all of them. For me, the easiest and most natural is to just use google scholar. I like google scholar mainly because it grabs any PDFs that have been posted on the internet and ties them to the reference.

Parsing Literature

Once I've added an entry to my Zotero library, which includes adding relevant PDFs, GitHub repo links, and tagging, making sure you tag and grab relevant links will make it easier latter on. I then go about creating a note item for each entry. The nice thing with recent versions of Zotero notes is that they are markdown+rich text meaning you can be pretty detailed.  So how do I initial parse without having to read all the papers I've added? I add three sections in the note. The first is a summary section. For this, I either use ChatGPT, SciSpace, or paper digest to do this for me. Then I look through the paper/document and screengrab any figures that stand out to me for whatever reason. Finally, I mark the priority level. Do I think this is a high-priority paper to read or not? The reason to do that in the note and not the tag is because the priority for reading is a transient state of an entry; it will change over time and eventually have a null status once it's read. 

Once I'm done creating the notes, what I can do for a specific folder that may represent a topic or a specific project is generate a Zotero report that provides all the metadata text for the entries. I can then go through it again to see what stands out to me. This in my opinion is a really nice feature, because it lets me visually go through the titles and my notes and see what I want to focus on first. There are some fancy tools out there that can create graphs of connectivity and other relationships between papers which would probably also be very useful if your trying to narrow down papers to read.

Reading Literature

Once I'm done selecting which papers I want to read from the Zotero report I generated, I will then print out the papers. Yes, I know printing doesn't make much sense in our multi-monitor research setups, but for me, I can't seem to shake the desire to want to read a paper in physical form. There is something about being able to flip back and forth between different sections and how I represent concepts and results in my mind's eye. I believe there is some strong evidence for better information recall when reading books in physical form.

For reading the papers, there is no real best approach in my opinion, you just have to sit down and read in the best way that works for you. I personally try not to spend too much time marking up the document. I will typically just add some kind of marking to indicate to myself some passage or content I find interesting or important. Once I'm done reading I then go to the digital PDF of the paper in Zotero and make annotations and highlights of the parts I've indicated on the physical form. This is important because when I'm writing and want to reference the document, I use these annotations and highlights as a guide for why I wanted to reference the paper in the first place.

Referencing

If you use MS Word with the Zotero plugin, then it's pretty straightforward to create in-text citations and a bibliography. If you are a $\LaTeX$ user, as mentioned above, I've found the best way to set up your documents is to use a cloud storage service and the Better Bibtex plugin. This lets you automatically update the .bib file for a given folder in your Zotero library and save it on your cloud storage. This way if you use something like Overleaf, you just have to create a shared link for the .bib file in your cloud storage and then create an Overleaf project file based on the link. For google drive, you can follow the steps here.

Reuse and Attribution