Bill Marshall

A Tale of Disruptive Science, Irreproducibility, and Unanticipated Consequences

Posted by Bill Marshall in From The Trenches, Portfolio news


I’ve spent a significant amount of my career focused on the use of nucleic acids as therapeutic agents.  The ability of nucleic acids to act through exquisitely selective binding to other nucleic acids, proteins or other biologically active molecules provides a seemingly limitless opportunity to target the drivers of disease processes.  This approach has emerged as a major modality in drug development and has driven significant investment in nucleic acid therapeutics.

To develop nucleic acid therapeutics against intracellular targets requires delivery into a range of tissues and cells and this is a significant challenge with this modality.  Thus an inordinate amount of time has been spent investigating promising new approaches and evaluating their potential to deliver on the promise of the technology (no pun intended!).  It’s a very difficult problem that has consumed decades of academic and industrial research time and dollars with relatively limited success.  Almost all current approaches require a parenteral route of administration despite significant efforts to develop approaches for oral bioavailability.

Disruptive Science

Given this prelude, one can only imagine my unbridled excitement back in 2012 in reading a peer reviewed paper, hot off the presses, suggesting that nature had evolved a system that allowed for the transfer of small nucleic acids from plants to mammals.  The manuscript in Cell Research entitled, “Exogenous plant MIR168a specifically targets mammalian LDLRAP1: evidence of cross-kingdom regulation by microRNA” reported that a plant specific small RNA (sRNA) from rice grain, microRNA-168a, could be found in the blood of mammals after feeding them rice.  Not only that, but the rice could be raw or cooked and most amazingly this sRNA molecule eventually ended up in the liver where it regulated the production of an enzyme important for cholesterol metabolism!

The publication suggested that rice had evolved a system for the modification and/or encapsulation of sRNA in a way that made it very stable to environmental conditions and able to transit through the gut of mammals, into the blood stream, into tissues and into cells that controlled important biological processes.

Furthermore, despite the fact that there are thousands of sRNA’s in a rice grain that modulate biological functions in the plant, there was apparently a system in place that specifically prepared MIR-168a in a way that transited it safely through an extremely hostile environment for RNA while maintaining biological activity.

The implications of this report were certainly disruptive as it provided a provocative opportunity to solve two significant challenges in the nucleic acid therapeutics field, namely; creating orally bioavailable nucleic acid drugs and simultaneously identifying a low cost manufacturing platform for their production.  This nirvana prompted me, like any good biotech entrepreneur, to develop a business plan that would allow us to potentially exploit the underlying biology to create orally administered nucleic acid therapeutics.

The concept was enthusiastically greeted by several potential investors and prompted us to put a plan in motion to generate sufficient proof of concept to justify an investment in such an enterprise.  One important aspect was to identify a potential partner in the agricultural biotechnology space that would be interested in participating and would bring both plant sRNA analysis and transgenesis expertise to the table.

We were fortunate to work collaboratively with a group of scientists at Monsanto on a phased research plan.  This plan provided a strategy for initial proof of concept research through reproduction of the reported findings, and followed this with a detailed research strategy to identify the systems and processes responsible for packaging sRNA in plants in a format that delivers oral availability in mammals.

Irreproducibility

To make a long and complicated story short, in the first phase of the plan we showed quite convincingly that we could not detect the presence of MIR-168a from the plant in the blood or liver of mice despite feeding them several different rice based diets.  Moreover, we could show that the effects on cholesterol metabolism originally reported could be explained by nutritional imbalance rather than by cross-kingdom gene expression control.  I won’t delve into the details of such experiments as they can better be reviewed in this publication (p. 965).

Needless to say, our observations were very disappointing.  At the same time, they provided a compelling set of data that prompted us to discontinue further work in the area.  All of this was accomplished quickly and in a very capital efficient manner.  This is a nice example of providing a small amount of resources to assess the potential of a reported innovation.  Based on the outcome we moved quickly to kill the program, but could have moved equally quickly to further develop the opportunity had we succeeded in reproducing the findings.  In my view this is a necessary model in early innovation funding as others have advocated.

The low rate of independent reproducibility of experimental results reported in the bioscience scientific literature is a significant issue generally, but exponentially for biotech entrepreneurs.  After all it is the lifeblood from which we sprout new ventures.

Estimates from two pharmaceutical company groups suggest that the rate of reproduction is less than 25% (here) or worse (here).  There are a variety of reasons for this state of affairs and a set of guiding principles have been proposed for improving the quality of the scientific literature (here too).  In addition, a collaborative Reproducibility Initiative has been created to independently evaluate the rate of scientific reproduction of published data.

The original findings reported in Cell Research generated a huge dialog in the nucleic acids field and a significant amount of skepticism.  In addition, the possibility of cross-kingdom transfer of dietary nucleic acids created an enormous amount of discussion and potential disquiet about the impact of dietary nucleic acids in agricultural biotechnology.

Our collaborative research team decided to make the investment of time and effort to write up the findings and attempt to publish them.  We felt this validation experiment was an undertaking that provided important, clear outcomes.  We also felt it was important to have this report included as part of the body of scientific literature regarding the potential for cross-kingdom RNA transfer.

Unfortunately, negative reproducibility results are rarely reported in the literature.  Academics strive to make new breakthrough discoveries, companies often cut their losses and move on quickly to other projects, and typically scientific journals aren’t very interested in publishing negative results.

After an unsuccessful first submission of our manuscript to Cell Research, the journal that published the initial paper (and who told us it would be hard to publish a paper of largely negative results), we were eventually successful in publishing the findings in Nature Biotechnology.  The Editor made it clear, as a collaborator in the Reproducibility Initiative, that Nature Biotechnology is open to considering replication papers.

Nature Biotechnology realized the broad ranging implications of the originally reported results across a large swath of pharmaceutical and agricultural biotechnology.  After an extensive peer review process, our study was deemed to be of sufficient quality to be published in this highly respected journal.  The paper was well received and generated a significant amount of interest in the fields of nucleic acid research and agricultural biotechnology, as well as the general press.

And then the unexpected consequences part of the tale…

About five months after the paper published, I received an interview request from a scientific writer at a local publication who was interested in discussing RNAi technology in medicine and agriculture for an article she was working on.  The interview focused almost completely on the Nature Biotech paper we’d published.  The author seemed quite well versed on the paper and I looked forward to seeing the eventual article.

Much to my amazement, an article entitled “Muzzled by Monsanto” appeared in The Boulder Weekly and was written by the reporter with whom I had spoken.  To make another long story short, the article suggested that our publication was really intended to squelch scientific research into the transfer of dietary nucleic acids from plants to animals because of the potential negative impacts in agricultural biotechnology.

What an unbelievable turn of events.  Our efforts to demonstrate scientific rigor and reproducibility were now cast as a central player in a dubious conspiracy theory purported to squelch research into the area.  Of course this couldn’t be further from the truth.  Our efforts were driven by an opportunity to fundamentally change the trajectory of the development of nucleic acid therapeutics and in parallel launch an innovative science driven company.

It should be noted that several groups have independently published papers questioning cross-kingdom dietary transfer of sRNA from plants to animals (here and here, reviewed here).  A recently reported study also suggests that contamination is the predominant non-dietary source for foreign sRNA in animal tissue extracts.

Our experience is a fairly typical example of how a biotech entrepreneur converts disruptive scientific observations into innovative technologies with product potential.  This is also one that highlights a major and common challenge facing innovative life science companies, namely the lack of reproducibility of results published in bioscience scientific literature.  Hopefully programs like the Reproducibility Initiative and the recently announced U.S. NIH focus on combatting irreproducibility should help to correct the problem.

Bill Marshall

Bill Marshall

CEO of Miragen
This entry was posted in From The Trenches, Portfolio news and tagged , , , . Bookmark the permalink.
  • http://www.hollyip.com/ Suleman Ali

    The reporter’s behaviour shows how important transparency is in the system, and how everyone in society has to be kept on board as technology develops. That’s very difficult to do given how many anti-science people we now have, but public opinion is important as it influences law-making, court decisions etc.