Not All Evidence Is Created Equal

We live in an era of unprecedented access to health information — and unprecedented confusion about what to believe. A 2023 survey found that 65% of Gen Z and Millennials get their primary health advice from social media rather than doctors, where influencers confidently promote conflicting rules — carnivore, keto, seed-oil avoidance — backed by testimonials and cherry-picked studies.

A personal story is not a clinical trial. A single study is not a consensus. A confident presenter is not a substitute for decades of rigorous research.

This article is a practical guide: how to read an abstract, spot the statistical tricks used to mislead, and apply a five-step checklist before you change anything about your diet.

Scientists have developed a hierarchy of evidence quality. Think of it as a pyramid: the base is wide and easy to produce but unreliable, while the top is narrow, hard to achieve, but represents the strongest knowledge we have.

We build our recommendations on levels 4 and 5 — randomized controlled trials, systematic reviews, and meta-analyses. Not influencer opinions. Not single studies. Not trends.

An anecdote tells you what happened to one person; it can't separate cause from coincidence. Observational studies of large populations are more informative, but people who eat more vegetables also exercise more and smoke less — hidden confounders. Randomized trials assign people at random, the only reliable way to rule those out; meta-analyses pool many trials, which is why they sit at the top.

Understanding why misinformation spreads is just as important as knowing what good evidence looks like. Four major sources dominate the health misinformation landscape:

The pattern is clear: the sources people trust most are the ones with the weakest evidence and the strongest financial incentives to mislead.

This deserves special attention. In 2016, a JAMA Internal Medicine investigation revealed that the sugar industry had secretly paid Harvard scientists in the 1960s to publish research blaming dietary fat — not sugar — for heart disease. 1 That single deception shaped dietary guidelines for decades and likely contributed to millions of preventable deaths.

It's not isolated. A systematic analysis found industry-funded food and beverage studies were 4 to 8 times more likely to produce results favourable to the sponsor. 2 Coca-Cola funded a network shifting blame for obesity from sugary drinks to inactivity; the dairy industry funds studies showing benefits of milk; the meat industry funds studies questioning the harms of processed meat.

Industry research isn't always wrong — but always check who paid before you trust the conclusion. Most journals require a funding disclosure near the end of the paper.

Most people never read past the headline. Journalists rarely read past the abstract. You can beat both by scanning the abstract for a handful of specific signals.

1. Study design. Strong: randomized, controlled, double-blind, meta-analysis, systematic review. Weaker: observational, cohort, pilot, in-vitro, mice. An RCT tells you more than a cohort; a cohort more than a rat study.

2. Sample size. A trial of 30 tells you far less than a trial of 3,000. Small samples produce flashy results that often evaporate on replication. Be sceptical below ~100 participants.

3. Duration. A 12-week weight-loss study says nothing about long-term outcomes. Any longevity or cardiovascular claim built on a three-month trial is suggestive, not proven.

4. Effect size. Large and meaningful, or small and marginal? A drug that lowers cholesterol by 40% is meaningfully different from one that lowers it by 2%.

5. Conclusion language. Good abstracts hedge: findings suggest, warrants further study. Bad ones over-claim: proves, demonstrates conclusively. Science almost never proves. It builds evidence.

Even a well-designed study can be presented to mislead. Three tricks account for most of the confusion.

Relative vs. absolute risk. The most common sleight of hand. A drug that cuts heart-attack risk from 2% to 1% can be reported as "halves heart-attack risk" (relative: 50%) or "reduces risk by 1 percentage point" (absolute). Both are true; they feel completely different. Always look for the absolute number.

Statistical vs. clinical significance. With enough participants, a tiny effect can be "statistically significant" (p < 0.05) and clinically meaningless. A supplement that raises HDL by 0.3 mg/dL clears the bar without changing anyone's health.

P-hacking. Run 20 tests on one dataset and roughly one will come out "significant" by chance. Researchers under pressure test many endpoints and report only the positive few. Pre-registered trials and systematic reviews are much harder to cherry-pick.

Two more concepts. Number Needed to Treat (NNT) is how many people must take a treatment for one to benefit — NNT of 5 is excellent, 200 useless. Confidence intervals give the range of plausible effects — a 30% benefit with a 95% CI of −5% to 65% includes "no benefit."

Practical red flags that a health claim may not be trustworthy.

No peer-reviewed citation. Legitimate claims can always be traced to published research.

"Studies show" without naming the study. Vague phrasing masks weak, misrepresented, or non-existent evidence.

Animal or in-vitro reliance. Cell and mouse work generates hypotheses but often doesn't translate; many compounds that cure cancer in mice have zero effect in humans.

Cherry-picking. Citing one study while ignoring twenty with the opposite result is selling a narrative, not doing science.

The presenter sells what they recommend. Profiting from the product they promote compromises objectivity — regardless of credentials.

Appeals to "ancestral" or "natural." Pre-modern populations died young of infection, childbirth, and malnutrition. Natural is not a scientific category; works in a controlled trial is.

Miracle testimonials. A single transformation story is the weakest possible evidence — people recover from illness for many reasons that have nothing to do with the intervention they credit.

Next time you see a bold health claim, run it through these five questions before changing anything.

1. Peer-reviewed citation? No link to a published paper — stop. If there is one, open it and read at least the abstract.

2. What kind of study? An RCT or meta-analysis is far stronger than a single observational study, which is far stronger than a mouse experiment or case report.

3. Who funded it? Read the Funding and Conflicts of interest section. A financial stake in the result should adjust your confidence downward — not to zero, but meaningfully.

4. Effect size large enough to matter? Absolute terms or only a ratio? Clinically meaningful, or statistically significant but practically tiny?

5. Does the body of evidence agree? One study is never enough. Does a systematic review or major guideline body (WHO, AHA, EAT-Lancet, NICE) reach the same conclusion?

Fail any of steps 1–3 and the claim shouldn't change your behaviour. Pass all five and it's worth taking seriously.

At Evida Life, we follow strict evidence principles:

Peer-reviewed sources only — studies scrutinised by independent scientists before publication.

Meta-analyses preferred — single studies mislead through small samples or flukes; we prioritise systematic reviews.

Continuously updated — science is not static; we update recommendations as new evidence emerges.

Transparent uncertainty — we mark where evidence is strong, emerging, or still needed. Honesty about uncertainty is a mark of scientific integrity.

Gold-standard focus — we prioritise randomized, double-blind, placebo-controlled trials over observational data.

No commercial conflicts — we do not sell supplements, take industry funding, or accept money for product reviews. Our only incentive is helping you live longer and better.

We draw evidence from the world's most respected scientific institutions:

NutritionFacts.org — daily-updated nutrition reviews by Dr. Michael Greger and team, covering every study in the English-language nutrition literature. PubMed — the U.S. National Library of Medicine's database of over 36 million biomedical citations. The Lancet and NEJM — the highest-impact medical journals. EAT-Lancet Commission — the most comprehensive framework for healthy diets within planetary boundaries, authored by 37 scientists from 16 countries. 3 Longevity-region research — the Buettner / National Geographic surveys of the world's longest-lived populations. Cochrane Reviews — gold-standard systematic reviews with strict conflict-of-interest rules.

The bottom line: apply the checklist. If a claim doesn't pass, it belongs in the interesting but unproven bucket — not in your meal plan.