Open Source and Scientific Studies: the biggest scams of our era
Each decade got its own scam: in the 2000s it was hardware (with Y2K or Intel nerfing CPUs for packaging reasons), 2010s was websites, apps and ransomware and now in the 2020s we have: Open Source and Scientific Studies 💫
Why are Scientific Studies a scam?
Ok, they’re not all a scam. But they’re used by scammers to manipulate non-statistically-educated people into buying into their claims. People hear “Scientific Study” and immediately think it’s some sort of godsend. “I read it in a Scientific Study, it must be true, right?” That’s why most ads on Social Media will leverage “Scientific Studies” to push their crap.
But the more you know about Science (and statistics), the LESS you trust any “conclusive” Scientific Study. Or, as Richard Feynman put it:
Science is the organized skepticism in the reliability of expert opinion
Here’s a funny example. A few months ago I saw this Reddit post by someone asking the “ideal rest period between sets for hypertrophy”. The second most upvoted comment just points to a paper published in 2016 and concludes: “Three minute rest times led to greater hypertrophy”.
Done. God has spoken.
But if you look into the study, there’s A LOT to be skeptical about:
- Small sample size: only 21 individuals split into 2 groups
- Very short duration: 8 weeks
- Very narrow population: all young experienced resistance-trained men
So, just right off the bat, if you’re a woman, or you’re not a young experienced lifter, or you’re not the particular ethnic group selected by this study, you should take it with a grain of salt.
But on top of that, the authors of the study themselves LITERALLY say:
however, our data were statistically underpowered for these analyses. We therefore cannot rule out the possibility that the greater training load achieved by the longer rest period group was responsible for the greater training adaptations
They are LITERALLY SAYING: “What we found might NOT be related to the rest time, but to a confounding variable.” They explained that the LONG group (the group resting for 3 minutes) pushed more volume because they had more time to rest and recover:
Total aggregate load volume over the 8 weeks was greater on an absolute basis for LONG compared to SHORT (51,385kg ± 9420kg vs 44,755kg ± 12,166kg, respectively)
So basically the rest period most likely had NOTHING to do with the difference in growth. It was just that by resting for a longer time, you get less fatigued, and you can achieve greater training volumes. And they provide a final word of caution:
Sessions for the LONG lasted more than twice as long as those for the SHORT. The cost-benefit tradeoff must therefore be taken into account if training time is an important factor.
Yet, people take it at face value. And the worst part? The findings are not even statistically significant (only anterior quadriceps thickness passes a significance test with p=0.04).
Why is Open Source a Scam?
I’m sorry again, another incendiary take to get your attention: not all Open Source is a scam, by far. On the contrary, true Open Source software was a great advantage for human progress. I’m a huge supporter of Open Source software (and a contributor myself).
But lately, “Open Source” has been used as a cheap way to promote products. All the AI-boom companies are pushing their vibe-coded projects to Github with dubious licenses and no installation instructions and claiming to be “Open Source”.
Take Open WebUI for example, the “open-source, self-hosted alternative to ChatGPT”. The code is there on Github, so yes, it’s “open”, but the license forbids you from fully modifying the software if you’re serving more than 50 users.
Truly Open Source software must comply with OSI’s (Open Source Initiative) definition of Open Source, which means: you can not only see the code, but also use it, modify it, and share it freely. You might also hear the term FOSS (Free and open-source software) which is a less formally defined, but equally relevant term to define truly Open Source. Open WebUI’s license is not OSI-approved and does not meet the requirements to be called FOSS.
So, important for everybody to know:
the fact that the project is on Github does NOT make it Open Source.
For the record, this phenomenon of non-FOSS projects on Github is not new. It happened many times before including big projects (Redis or Elastic for example), but in the past 2 years I’ve seen it blow up.
But you might be wondering, why is this a problem? Well, in the same way that it happened with the Scientific Studies, people see a project on Github and immediately think “Oh, it’s Open Source”, only to get bitten by the non-free license months down the implementation path.
Software is like sex: it’s better when it’s free
- Linus Torvalds
The good news is that, while Scientific Papers require detailed scrutiny, verifying if a project is truly Open Source is fairly simple. Just check the license, if it’s one of the official OSI Licenses, then you’re good. If not, run away. The most developer friendly licenses are MIT, GPL, Apache or BSD.
The bad news is that it doesn’t end there. Checking the license is one thing, but you can’t ever fully trust a project just because it’s on github, as it’s plagued with malware. For example, check out how scammers are sending job offers with backdoors in LinkedIn job offers.
Exploiting authority
These scams are basically riding on the authority of the two abstract transitive entities: Science and Open Source. Science is legit, but not all science is legit. Open Source is legit, but not all open source projects are legit.
The bottom line is: don’t trust claims just because there’s a transient authoritative entity behind it. Always be skeptical.