Every living cell on Earth has proteins. Typically thousands of them, that serve as molecular machines to digest food, sense the environment, or anything else a cell must do. However, many genes, and the proteins they code for, have unknown functions. In humans, the function of about 1 out of 5 of genes is unknown. To explore these dark corners of the genome, scientists can break up DNA to disable a gene, making their encoded proteins nonfunctional, and watch what happens to cells as a result, inferring the lost function from what goes wrong.

When CRISPR/Cas9 came online in 2012, it offered scientists a tool to do exactly this – cut genes. The Cas9 enzyme searches through DNA, using a "guide RNA" to look for a specific sequence, and makes a cut when it finds a match. The gene, split in two, is repaired by the cell, but often with a large chunk missing. Many scientists assume that if a chunk of a gene is missing then the protein that it encodes will not function, or even be produced.

Researchers at the European Molecular Biology Laboratory in Heidelberg, Germany used CRISPR to make cuts in 136 different genes. In about a third of cases, proteins were still produced from these "damaged" genes and, furthermore, many of the proteins remained partially functional. This strange phenomenon, of damaged DNA producing functional protein, does more than punctuate life’s remarkable adaptability and resilience.

It means that dozens, or hundreds, of studies that used CRISPR/Cas9 to knock out genes, but failed to validate that the encoded protein was fully removed, could be incorrect or misleading.

While many labs that use CRISPR to knock out genes do validate that the encoded protein is no longer produced, other labs fail to check. Searching for one protein in a cell is time-consuming and technically challenging; testing for protein function takes even longer. There are some methods available to look for specific proteins, but many CRISPR/Cas9 studies fail to run these experiments – or scientific journals don’t ask for the data. 

Nature Methods, the same journal that published the paper from the Heidelberg laboratory, recognized shortcomings in CRISPR validation early on. In 2017, they highlighted a genome-editing consortium, in collaboration with the US National Institute of Standards and Technology, that aims to develop standardized procedures for CRISPR research, including publishing guidelines that include which guide RNAs were tested, how they were designed, and which controls were used in experiments.

The problem with major scientific developments, especially CRISPR/Cas9, is that experimental tools often explode in popularity before scientists and editors can implement standard procedures. When DNA sequencing was developed in the 1970s, for instance, there was little need for standards because it was so challenging to decipher the sequence of even a short piece of DNA. A decade later, however, GenBank, a DNA sequence repository, came online and journals began to mandate that researchers deposit their sequences. This requirement, together with reporting standards issued by journals like Nature, have ensured that a rapidly growing collection of DNA sequences can be vetted and independently analyzed by the scientific community. The same is true for methods like x-ray crystallography, with journals requiring that protein structures be independently validated and uploaded to publicly-accessible databases.

But while some scientists were shocked by the new study, others took a laissez-faire approach to the findings. On Twitter, many vented their rage at what they felt was a lack of careful controls by the scientific community. Raphael Ferreira, a postdoctoral fellow at Harvard Medical School, was inspired, perceiving this study as a game changer for the CRISPR community.

“I was surprised by the results in a really positive way, as this paper rings the wake-up call for every scientist using CRISPR/Cas9,” Ferreira said.

Despite the enthusiasm, however, Ferreira will not change how he performs his own experiments. “The few times I have [used CRISPR/Cas9] in mammalian cells, I have always confirmed them with a Western blot,” referring to an experimental method to detect specific proteins.

Victor de Lorenzo, a research professor at the National Center for Biotechnology in Madrid, agreed, claiming that, “…the only way to ensure that a protein is altogether removed is by making a Western blot.”

Down the hall or across the street from my office, dozens of scientists use CRISPR/Cas9 to uncover protein functions. One of these researchers is Shashank Gandhi, a PhD student at the California Institute of Technology (CalTech) that has published CRISPR/Cas9 methods to delete genes in chicken embryos. Though he agrees with Ferreira and de Lorenzo, Gandhi asserts that validations could be taken a step further, and believes that journal editors should take action.

“I think that journal editors should encourage authors to present supplemental data on how the knockouts were validated,” insists Gandhi. “I know that Nature requires that information as part of a "research summary" document that is submitted to the journal with each paper submission.”

If Nature, which is widely considered the premier academic research journal, takes action to ensure that CRISPR knockouts are validated, then perhaps other publishers will take notice. In the meantime, Gandhi and others are not taking any chances.

“I use several approaches to validate my CRISPR knockouts. For starters, I design and test multiple [guide RNAs] targeting the same gene for all my knockout experiments. Secondly, wherever applicable, I try to perform rescue experiments to establish loss of function phenotypes,” says Gandhi, referring to an experiment in which a deleted gene is restored to test whether that protein’s phenotype returns, confirming a link between a gene and the function that was lost when the gene was broken.

While all of the scientists that I spoke with agreed that researchers could do more to double check their experiments, it is unclear what actionable steps could be taken. Perhaps a combination of scientific, academic, and institutional changes could alleviate the potential for misleading studies. Faster experimental methods to detect proteins, standardized publishing procedures, and an academic database that describes which guide RNAs have been tested in each organism, would all serve to enhance the rigor of current studies.

Unfortunately, academic institutions and scientific publishers are hulking bureaucracies with slow-moving policy changes. Ensuring that CRISPR/Cas9 produces repeatable experiments – rather than blemishes on the scientific record – will require the collective action of scientists. It will demand self-governance.