log in

Quantum computing’s reproducibility crisis: Majorana fermions

Hacker News - Mon Apr 12 17:07

Equipment to perform experiments to find Majorana signals

Experiments to find Majorana signals are performed by loading a nanowire into a dilution refrigerator capable of cooling it down to close to absolute zero.Credit: HGA Architects and Engineers

A shadow has fallen over the race to detect a new type of quantum particle, the Majorana fermion, that could power quantum computers. As someone who works in this area, I’ve become concerned that, after a series of false starts, a significant fraction of the Majorana field is fooling itself. Several key experiments claiming to have detected Majorana particles, initially considered as breakthroughs, have not been confirmed. One recent case ended in a high-profile retraction from Nature (see Nature 591, 354–355; 2021), which I initiated with my colleague Vincent Mourik, a physicist at the University of New South Wales in Sydney, Australia. We raised concerns after obtaining additional data from the original experiments that were not included with the published paper.

Much is at stake. Majorana particles are in theory their own antiparticles, and were predicted in 1937 by Italian physicist Ettore Majorana. Computer giant Microsoft hopes to use Majorana particles to build a reliable quantum computer: the particles should make for exceptionally stable quantum bits. The scientific excitement around them is on a par with gravitational waves and the Higgs boson.

Experimentally, researchers are at loggerheads over whether Majoranas have been detected at all, let alone whether they’re an asset for quantum computing. As scepticism of the claims creeps beyond the cognoscenti, the field is at risk of getting a bad reputation, despite its untapped promise.

Challenging science

Producing Majoranas in the laboratory is very hard. Experiments combine cutting-edge fields such as nanotechnology, superconductivity, device engineering and materials science. In the most developed approach, researchers must first grow a nanowire crystal — a feat in itself — to produce a column of atoms 100 nanometres (one-thousandth the width of a human hair) across. Then they must connect the wire to a circuit sensitive enough to measure single electrons travelling through it. The whole experiment must be done at about one-hundredth of a degree above absolute zero, in a magnetic field 10,000 times that of Earth’s.

Under those extremes, when all the electrons in the wire are magnetized, Majorana particles should emerge from the two wire ends. In theory.

More than 100 groups have tried this. Two dozen have reported Majorana manifestations. These usually appear in the form of a characteristic electronic signal: a narrow peak in current as voltage is varied across the nanowire. I was a member of one of the first teams to observe this, in 20121. More papers soon appeared. Detections of a quantized value of the current, first predicted in theory and then reported in experiments published in Science2 in 2017 and Nature3 in 2018, were interpreted by many to be the ultimate evidence of Majoranas.

In 2020, these observations came under scrutiny after replication experiments were conducted. Science published an experiment led by researchers at Pennsylvania State University in University Park contradicting the 2017 report4. My group reproduced patterns from the 2018 Nature study, but demonstrated that they need not originate from Majorana5. We did a cross-check on both ends of the same nanowire, but found a current peak on only one end. This violated the basic expectation from the theory that Majoranas always come in pairs. The rate of rebuttal is speeding up: researchers have not been able to confirm the findings6,7 of two separate papers claiming to have found Majorana regimes in nanowires8,9. And reports of current peaks in a new iron-based superconductor, Fe(Te,Se), that were attributed to Majoranas1012 in Science and Nature Communications will need to become more nuanced after a Physical Review Letters publication this year13.

The lesson: Majorana particles aren’t necessary to produce the current peak signals. At least since 2014, we have known of more-mundane explanations, such as other quantum states that are not Majoranas14, accidental signals caused by imperfections in the nanowire, or fascinating but previously explored cooperative behaviour of numerous electrons (see ‘Mixed signals’). Yet, affirmative papers kept coming out without even mentioning alternative explanations, creating the impression that a debate is raging between Majorana optimists and pessimists.

Mixed signals. Graphic showing imperfections in nanowire.

Source: Peng Yu/Frolov lab

Reflection needed

As someone who has published and reviewed positive and negative Majorana claims, I sense a wider problem. The controversy has already begun to erode confidence in the basic experimental method of passing current through quantum objects, even though this powerful technique has been used in many great discoveries, including Nobel-prizewinning observations in superconductivity, the quantum Hall effect and tunnelling.

It has already begun to affect me. Prospective graduate students ask whether I’m stopping Majorana research. Grant reviewers assume it is the methodology, not selective data reporting, that causes confusion in the field.

In my view, nothing is wrong with the basic method often dubbed ‘quantum transport’. I feel that selective data presentation is the main problem. If all papers included full or at least appropriately selected sets of data, quantum physicists could assign correct explanations, Majorana or not.

But I think that researchers are cherry-picking — focusing on data that agree with the Majorana theory and sidelining those that don’t. A case in point: a 2020 Science paper on Fe(Te,Se) reported quantized behaviour of current, which the authors saw in a single vortex, out of 60 assessed10. I contend that data-selecting researchers can be enabled by some journals and reviewers who might be insufficiently stringent. (When asked about the 2020 paper, a spokesperson for Science said that results and conclusions, including alternative mechanisms to explain the observed quantization, were presented carefully.) Time and time again, I and other reviewers argued for journals not to publish papers based on selective data presentation, only to see them appear in other (or sometimes the same) journals. Sometimes there really is no need to present all of your data, if a single graph tells the whole story. But for Majorana particles, simply searching through the data to identify peaks of the right height is not enough to stake a detection claim, especially when alternative theories exist.

It is all too easy for selection bias to take over in hypothesis-driven experimental research. The ‘best’ data are often considered to be those that fit the theory. So deviations are too readily dismissed as experimental or human error that can thus be discarded.

Another problem is the breadth of peer review needed to check Majorana claims. Scrutiny is hard in any multidisciplinary field. Referees tend to be expert in one subject and struggle to judge others, and that leaves gaps. For example, a theoretical physicist might be comfortable assessing the calculations but not the experimental process, and a materials scientist, who understands how to grow nanowires, might skip the theory part. But a holistic view of the whole study is needed to properly assess it.

It is an all too familiar story. In a Nature survey of the ‘reproducibility crisis’ across chemistry, biology, physics, engineering and medical sciences (see Nature 533, 452–454; 2016), selective reporting of results was a top culprit. We’ve seen this for decades. Physicist Robert Millikan’s oil-drop experiments of more than a century ago famously omitted some data points. He did get close to the actual value of the electron charge — but science cannot depend on these sorts of fluke. Some Majorana papers are turning out to be unreliable because of how data are selected.

Ways forward

The behavioural norms across the condensed-matter physics community need updating. There is only one solution, and it is more accountability across the board. The following steps will help both Majorana research and fields far beyond.

Open data. Scientists should disclose all data in a repository and comply with sharing standards, such as FAIR (findability, accessibility, interoperability and reusability)15. Some curation is unavoidable. The volume of data collected in a modern physics laboratory is high: computer scripts control the equipment, which might run 24 hours a day. A remedy is to clearly explain the protocol that is used to perform any data selection — so others might reuse or scrutinize it. Remember, data selection is a form of data processing.

Journals, funders (including corporations), research labs and universities should demand such open data practices, as they do in clinical trials, genomics, Earth sciences and a handful of other disciplines. Sharing data improves reliability, fosters collaborations and speeds up progress. The high-energy physics community, for example, could teach others how to share study protocols so that each paper is repeatable or reproducible16.

Although it is not widely known, access to further data is already required by many publication policies and government codes of research conduct. Notably, the United States does not have a national code, in contrast with other countries investing heavily in research. Further efforts are needed to make such sharing automatic and not ‘upon request’. As the case of the Majorana paper recently retracted from Nature showed, seeing full data can be crucial for evaluating an experiment.

Critics will counter that simply sharing data does not capture all that goes on in the lab, that experience and insight — craft — have value that cannot be described in a protocol. I argue that robust, useful science is built on reliable processes that can be revisited, verified and re-examined as many times as necessary.

Open process. Reviewers need to be more questioning of extraordinary claims. Are the results too good to be true? Have enough data been presented? Have other explanations been considered? Cross-checks should be conducted, making it harder to stake an unreliable claim. For Majorana physics, this is as basic as comparing the magnetic- and electric-field dependence of current peaks with what would be expected theoretically. If done consistently, this would thwart many false claims.

But even the most rigorous reviews can be ignored. If the paper is rejected, the authors are free to disregard all input they were given and send their manuscript to another journal. I have seen Majorana papers that have received multiple negative reviews and rejections on scientific grounds published with only minor changes in another high-profile journal. Opening up the notoriously opaque publication process is key to cutting down on the proliferation of bad research.

Editors should take responsibility: it is they who decide, even if they lack in-depth expertise for that particular paper’s topic. Each accepted paper should have its editor’s name published alongside it. For each retraction, the editors should provide their view on what happened. All journals, especially high-impact ones, need to have community oversight. Editorial retraction should be applied widely, because waiting on authors to retract papers on their own can take an eternity. At the moment, most journals do not even have the capacity to run their own investigations into claims of mistakes in their papers. They should build this capacity, with the help of the research community.

Journals deserve praise for publishing negative results and for normalizing verification studies in physics. Researchers willing to share their results should receive well-deserved attention. For example, the American Physical Society ran an invited session on negative Majorana results at its virtual 2021 March meeting.

What of Majorana research? It remains viable and important. But, in my view, the key discoveries have yet to be made. A concentrated effort is now needed to improve our nanowire materials, experimental techniques and data analysis, as well as to tease out alternative explanations. Reliable proof is needed that the particles are indeed their own antiparticles — with our eyes on the full data.

Only then will we be ready to develop Majorana quantum computers.