Lost Ideas in Science

Believe it or not, humanity is completely capable of ignoring a $1,000,000,000,000 bill sitting in plain sight. It happened in 1747, when James Lind ran a controlled scientific study — maybe the first ever — and discovered that citrus fruits cured scurvy.

In Lind’s day, scurvy killed more British sailors than "shipwreck, storms, all other diseases, and enemy action combined”. Which is to say that Lind's cure was not just going to save countless lives; it was an incomparably powerful military technology. The stakes couldn't have been higher. And yet, despite having been proven with the most advanced scientific methods the world had ever seen, it was promptly lost for many decades after.

According to British Historian C.C. Lloyd, the British might have “lost the American War of Independence thirty years [after Lind’s study] because his cure was not adopted”. If Lloyd’s suggestion is correct, the discounted net assets of America is what gives you a roughly $1,000,000,000,000 bill that the British may have just left lying on the ground.

Why Do We Lose Ideas?

"Show me the incentive and I'll show you the result."
- Charlie Munger

James Lind's scurvy cure didn't mesh well with the science of 1747. Science was then 150 years away from discovering vitamins and so, naturally, it would have been hard to understand how it was the vitamin C in citrus that was the cure. The top minds of the day, viewing Lind's results, quickly came to believe that the acid in the juice must be melting some big black scurvy blob that sailors got in their stomachs. "Great!" they said, "let's boil the juice to make it more acidic and work even better!"

Unfortunately, as we now understand, boiling the juice just destroys the Vitamin C content. Which means the boiled "cure" was no cure at all. Now there is nothing wrong about this; it's just good old normal science seeking understanding and trying out hypotheses. And 150 years later this did give us understanding, and that understanding was bigger than a simple cure for scurvy.

The problem is that somehow for the intervening 150 years everyone didn't just go back to using the limes that were proven to work.

Theories, like those of James Lind, which happen to integrate poorly with existing science of their day are bound to suffer. By failing to integrate into science they lose out on what is surely humanity's best protocol for rewarding, preserving, and developing openly shared knowledge. Patents might offer an alternative, but Lind's theories were also hopelessly unpatentable, since citrus fruits are freely available.

NFT's for science

The idea that NFTs could help rescue enormous scientific truths from obscurity might strike you as a little fanciful. You might even think... "aren't NFTs just some hyped-up blockchain art thing? Why should they help science funding at all?" So to start with let's take the simplest argument, and here we'll just quote it from our 2018 essay on NFTs for Science:

Historically, great knowledge is valued like great art. Bill Gates paid $40 million for Leonardo Da Vinci’s Codex Leicester. Einstein’s manuscripts sell for millions. And a single page of Darwin’s writing fetches hundreds of thousands of dollars[...] People buy Darwin’s original pages because they are connected to his idea — the fact that information from The Origin of Species is freely available is not a bug for its high value; it’s a feature. Today’s Darwin would type The Origin of the Species in a Google Doc or a versioned LaTeX file... But now with Planck, and the digital scarcity blockchain enables, she would create a unique NFT that serves as that first manuscript copy.

This "science as art" argument received some validation when we sold the first scientific result NFT (for $24,000). But we believe that this is just the beginning, and have been testing ways in which funds and proceeds from NFTs can be channeled to improve science funding more broadly. The aim, ultimately, is to enable progress and to try to help save some of the consequential ideas we might otherwise lose.

Case Study: Seth’s Robert's Appetite Theory

“Science is great, but it’s low-yield. Most experiments fail. That doesn’t mean the challenge isn’t worth it, but we can’t expect every dollar to turn a positive result. Most of the things you try don’t work out — that’s just the nature of the process.” https://fivethirtyeight.com/features/science-isnt-broken/#part4

You might think, “James Lind’s scurvy cure was 150 years ago! That couldn’t happen today!” But you’d be wrong. Planck looked for promising candidates for consequential ideas that seemed possible we might lose, and the most promising candidate we found was the late Berkeley Professor Seth Roberts' appetite theory.

In 2004, Berkeley Professor Seth Roberts discovered that consuming flavorless calories reduced his appetite. Consuming flavorless or unfamiliar calories made his eating habits more consciously controllable and helped him lose weight. And beyond weight loss, such a tool for conscious eating seemed to have the potential to help people eat more nutritious food, more easily go vegan, etc. Seth was a serious empirical scientist and the evidence he published for his theory was intriguing, if unorthodox. Though preliminary, it was an enormous discovery.

Soon, the diet went viral. It was covered enthusiastically in mainstream outlets like NYTimes and influential blogs, too, such as Steve Levitt’s Freakonomics, Alex Tabarrok’s Marginal Revolution and Andrew Gelman’s StatModeling. Seth Robert’s book, “The Shangri-La Diet: The No Hunger Eat Anything Weight-Loss Plan” became a national best-seller.

Glowing anecdotes appeared, including a notable one from Reddit co-founder Aaron Swartz. In that essay, Swartz confidently alludes to the "clinical trials" that would surely be carried out on Seth's Theory. His essential optimism, his confidence that the world would obviously figure out a way to run a clinical test on something so promising, is striking.

But the lab studies never came.

Within 5 years of its publication Seth’s idea had become little-discussed, never having undergone a single clinical trial. Some important outlets like Marginal Revolution LessWrong, and StatModeling kept the flame alive, but the discussion of Seth's theory all but vanishes.

By 2017, a few years after Seth had passed away, Neurobiologist Stephan Guyenet wrote that he expected to remain forever “puzzled” by Seth’s theory because “some sort of research” on it was “unlikely to ever happen.” Stephan’s answer reads more than a bit tragic; obesity kills millions each year, interest in using conscious eating to help improve the world, as in vegetarianism, is skyrocketing.

Failing to follow up on a promising theory in such a world feels a bit like an echo of our tragic failure to follow up on James Lind's scurvy cure.

How we tested “Seth’s Appetite Theory” the Web3 way

P1anck is active in the field of innovation and discovery, where incentives are infamously poor (e.g. Nelson 1957). For our Open Science NFT Concept, we sought to consequential ideas that would otherwise go unfunded and unfound. Seth’s Appetite Theory was perfect because any experiment results concerning the theory could not easily be commercialized for profit (hey - it’s just good ol’ flaxseed oil!).

We arranged to hire Cate Babcanec, a public health PhD with experience in science and technology acquisitions, to help facilitate a clinical study. Excitement grew amongst researchers and statisticians on Reddit and Twitter as we crowdsourced a study design to test Seth’s theory.

Then, we built a big website to recruit volunteers and explain the study. Of course, the design was shared upfront and the resultant data will be public.

Web3 legend Zooko even promoted our study to help recruit volunteers.

We randomized volunteers into a treatment and control arm as planned, thereby performing a randomized control trial (RCT), the gold standard of experimental research.

People volunteered to literally drink olive oil for five days to test a beloved but untested theory.

We collected survey results from volunteers three times a day, analyzed them , aggregated the data, and sold the results as $24,000 that funded the replication of the study.

Finally, @austingriffith minted and distributed ~160 “Open Science Astronaut” NFTs to burner wallets via http://planck.nifty.ink. These were given to people who participated in Planck’s Web3 Open Science RCT and comprised of clinical data visuals generated by amazing @dhiahop who plotted “Hunger” on the vertical axis and time on the horizontal one. The lower line is the local regression (Loess) of the “treatment” condition, the test of Seth’s Theory. I mean, just look at it up there. Elegant polynomial smoothing but with the low-fi charm of a cryptopunk. The NFT’s creation also auratically indexes (h/t @MPtherealMVP ) the beginning of a journey that might take us to some very cool places. This is a beautiful thing.

The Center for Open Science also actually owned one NFT, too, for a few exciting minutes as the first Open Source Science NFT went live for auction! Soon after, it sold for the mindblowing amount of 13 ETH ($24,000)!(https://t.co/JoV28pKxQR)

More so than a dollar-denominated value, the NFT really symbolizes participation in the first randomized control test of Seth Roberts’ Appetite Theory. It is also the the first “blessed” NFT, which means 10% of sales goes to the Center for Open Science. This sets a precedent of selling NFTs to pledge proceeds to them, an idea Vitalik recent lent legitimacy to https://vitalik.ca/general/2021/03/23/legitimacy.html.
Therefore, the NFTs buyers also participate in the study.

Planck was able to sell the result by bridging a compromise between two types of people who might want to buy this NFT.

The first type of buyer is a curator of science and innovation NFTs in the @Cooopahtroopa’s $value sense - this NFT would be a fine first edition in any such collection. The collector might find value in owning the internet’s first of many kinds:

  • first randomized study of Seth’s appetite theory
  • first successful crowdsourced citizen science study using #opensourcescience methods: it was published for review on a major stats blog and then participants volunteered online.
  • first crowd-funded replication of a study, which the NFT sale helps fun
  • and, of course, the first native NFT of a scientific study (and using solid web3 methods, at that!)

The second type of buyer may only want to fund the replication of Seth’s theory, knowing that buying this NFT would help accomplish this. These buyers value the study for simply having been run.

Consider what’s motivating these buyers. They have not seen the full results before they purchased the NFT. They are rewarding a study based on its design and subject matter! This is called “results blind science” and it can help alleviate publication bias.