Perspective on scientific truth versus scientific evidence; maintaining integrity in global food systems

The IARC Working Group report on the Carcinogenicity of Red Meat

The first case study is the IARC (International Agency for Reseach on Cancer, a suborganisation to the World Health Organisation WHO) Working Group report on ‘Carcinogenicity of consumption of red and processed meat’, which was published on 26 October 2015 in The Lancet. The IARC group consisted of 22 members from 10 countries. According to both the press release by WHO, with a four-page question and answer (Q&A) session on its website, as well as a short article in The Lancet (Bouvard et al. 2015), the group had ‘assessed more than 800 epidemiological studies…’, and, on this basis ‘classified consumption of red meat as probably carcinogenic to humans (Group 2 ‘classified consumption of red meat as probably carcinogenic to humans (Group 2A)’ and processed meat as ‘carcinogenic to humans (Group 1)’. Three years later, the extensive documentation was published as a monograph (IARC 2018). At the time of writing this perspective (July 2023), according to PlumX metrics, the Lancet article has been cited 1190 times, and was used 81 times in public-policy documents, including several times in United Nations global-policy documents. During the second quarter of 2023 alone, the Lancet article was quoted 51 times and used 12 times in public-policy documents. While the IARC evaluation is already 8 years old, it is still prevalently used and cited.

In the Lancet article, the IARC group members briefly summarised the balance of their evidence on which they base their evaluation. For instance, they wrote in the eighth paragraph that ‘Chance, bias, and confounding could not be ruled out with the same degree of confidence for the data on red meat consumption, since no clear association was seen in several of the high-quality studies and residual confounding from other diet and lifestyle risk is difficult to exclude. The Working Group concluded that there is limited evidence in human beings for the carcinogenicity of the consumption of red meat.’ Two paragraphs later the authors then went on to explain ‘the mechanistic evidence for carcinogenicity was assessed as strong for red meat’ (they mentioned N-nitroso compounds (NOC), haem iron and hormonally active agents (HAA)), which was then qualified by the WHO Q&A 11th answer, namely that ‘Eating red meat has not yet been established as a cause of cancer.’. As the only quantifying piece of information, the text mentions ‘A meta-analysis of colorectal cancer in ten cohort studies reported a statistically significant dose–response relationship, with a 17% increased risk (95% CI 1.05–1.31) per 100 g per day of red meat and an 18% increase (95% CI 1.10–1.28) per 50 g per day of processed meat.’. The Lancet article concluded that for its carcinogenicity evaluation, the ‘Working Group took into consideration all the relevant data, including the substantial epidemiological data…and the strong mechanistic evidence.’. The authors did not explain how ‘no clear…limited evidence’ in the statistical associations become ‘substantial epidemiological data’, nor how ‘strong mechanistic evidence’ squares with ‘no establishment of causality’. The contradictions between their own few paragraphs of text and their conclusion to identify red meat as probably carcinogenic were left for the reader to resolve. The same can be said for the separate evaluation of processed meats.

The methodological weaknesses of the IARC evaluation have since been variously described and published. A comprehensive critique came from Klurfeld (2018), who was himself member of that IARC Working Group. He explained the statistical inadequacy of using relative risks of less than 200% for casting any judgement, let alone just 17% as used by IARC. For instance, relative risk of lung cancer from smoking cigarettes would be 1000–3000% (p. 6). He also explains the difference between relative risk and absolute risk, where if values of 17% are applied to what are low incidence rates such as specific cancers, then the resulting increased absolute risk is outside fair measurability, or indeed significance for the consumer. Furthermore, to illustrate the inadequacy of statistical associations due to confounding factors for making such far reaching evaluations, Klurfeld (2018) also quoted one of his own previous analyses showing that red meat eaters have a statistically robust higher relative risk of 28% over white meat eaters of dying in accidents (p. 8). Surely, however, red meat consumption does not cause more accident deaths. Whatever association red meat may have with mortality or chronic disease, it is small and contextual, thereby depending on inter-individual differences and vulnerabilities, dose, preparation methods, quality of background diets, and overall lifestyles (Johnston et al. 2023). Regarding the mechanistic evidence, Kruger and Zhou (2018) showed in a review of all available material that studies of NOCs and haems ‘have not provided sufficient documentation that the mechanisms studied would contribute to an increased risk’. Boobis et al. (2016) thoroughly explained that the whole methodology of hazard identification utilised in the IARC monograph series generally ‘have become outmoded and serve neither science nor society’ (title). Also in 2016, the Editorial of The Lancet Oncology equally called for an improved methodology for carcinogen assessment, as the current one could be flawed (The Lancet Oncology 2016).

The balance of the epidemiological evidence on meat consumption was also comprehensively questioned by the NutriRECS consortium evaluation from the year 2019, which concluded after reviewing each available epidemiological study, that there is ‘only low to very-low certainty evidence’ for any mortality risk, and that therefore adults might ‘continue current unprocessed red meat consumption.’ (Johnston et al. 2019, abstract). A publication in Nature Medicine in October 2022, reviewing the same evidence again, arrived also at essentially the same conclusion, that ‘while there is some evidence that eating red meat increases risk of chronic disease, it is weak and insufficient to make stronger or more conclusive recommendations.’ (Lescinsky et al. 2022, abstract). Moreover, in 2021, the Scientific Council of the World Farmers Organisation released a review (WFO 2021, the author of this article was also one of the co-authors of this WFO review), where it was demonstrated that the IARC statement of ‘17% increased risk per 100 g per day of red meat’ can be attributed primarily to just one study conducted in 2005, and that the data in that study was significant only for pork meat, and only after statistical enhancement, according to IARC’s own assessment.

Overall, it can be concluded, that the 2015 IARC evaluation has been discredited along every dimension. It has been shown methodologically, epidemiologically and mechanistically to be devoid of supporting scientific evidence. It does not meet the before-mentioned National Academies of Sciences, Engineering, and Medicine (2019) robustness standard of being a ‘holistic web of knowledge reinforced through multiple lines of examination’. Given the very high-profile public-policy significance of this document, why has this IARC evaluation not been retracted or been reassessed, given all of the substantial counter-evidence described above? It seems also fair to ask, on what basis did the 22 IARC Working Group members reach their evaluation to begin with, if there was such degree of insufficient scientific evidence as shown by their own documentation? Is it possible that they followed an agenda they established beforehand, for which scientific evidence is helpful, but not mandatory?

The Global Burden of Disease 2019 investigations

The second case study concerns the Global Burden of Disease (GBD) 2019 study. When it was published, some observers (concerned scientists) noticed a conspicuous jump of dietary risks and deaths associated with diets high in red meat, compared with the previous GBD 2017. There was a 36-fold increase to a total of 896,000 deaths (GBD 2019 2020). Deeper analysis of the publication showed that the theoretical minimum-risk exposure level (TMREL) was reduced to 0 g per day, making red meat toxic from the first bite. Additionally, it seemed that none of the well-established protective health effects were assigned. Underlying evidence for such a strong statement seemed to be lacking in the documentation. Furthermore, against the stated guidelines of The Lancet, no PRISMA (www.prisma-statement.org) or GATHER (https://www.who.int/data/gather/statement) statements could be found, both of them being instruments in health-related sciences to ensure that the necessary minimum of information is provided for independent review.

The observers eventually succeeded to publish a letter in The Lancet in April 2022, asking whether the estimate of deaths attributable to red meat intake in GBD 2019 is reliable (Stanton et al. 2022). In the authors’ reply, the GBD collaborators then admitted that the setting of TMREL to zero was methodologically not adequate and will be revised for GBD 2020 (Murray 2022). Furthermore, the main author Christopher Murray has separately confirmed that GBD 2019 is not PRISMA-compliant (A. Stanton, as per an email conversation received from C. Murray, 2023). The observers note that according to the guidelines of The Lancet and the Committee on Publication Ethics (COPE) and the International Committee of Medical Journal Editors (ICMJE), it is mandatory to immediately correct all such known errors. This has so far not been the case, while at the same time GBD 2019 keeps on being referenced in important public-policy documents such as the Europe’s Beating Cancer Plan of the European Union. In a further letter to The Lancet, another group of observers are much concerned about the ‘troubling assumptions behind GBD 2019 on the health risk of red meat’ (Gordon-Dseagu et al. 2022, title). Finally, Christopher Murray was himself the senior last name author for the above-cited Nature Medicine publication in October 2022 arriving at the conclusion that the evidence against unprocessed red meat ‘….is weak and insufficient to make stronger or more conclusive recommendations’ (Lescinsky et al. 2022).

Similarly, to the IARC study of 2015, the next steps are still outstanding. Scientific protocol, guidance and rules unequivocally demand a retraction and correction of such medically relevant key stone publications, if they have been found to be erroneous. Why is GBD 2019 still unretracted? How is it possible that a GBD evaluation can reach results with such far-reaching implications for global public policy, if there was insufficient scientific evidence as Murray stated himself?

Sustainable livestock at United Nations Food Systems Summit 2021

The third case study pertains to global public-policy proceedings at the United Nations Food Systems Summit (UNFSS) in 2021, which were supposed to be underpinned by science. In the Summer of 2019, United Nations Secretary General Antonio Guterres announced he would convene a Food Systems Summit as part of the Decade of Action to achieve the Sustainable Development Goals (SDGs) by 2030, as follows: ‘The Summit will launch bold new actions to deliver progress on all 17 SDGs, each of which relies to some degree on healthier, more sustainable and equitable food systems.’ (UN 2019). The initiative became delayed by the outbreak of the pandemic, and it is not publicly known how the leadership structures of the Summit proceedings were chosen. By November 2020, the Summit preparations de facto kicked off in a large symposium of panels, discussions and presentations called ‘Bold Actions for Food as a Force for Good’ (all online because of the pandemic). It was hosted on an online platform provided by World Economic Forum and could be attended only by becoming a paying member of the World Economic Forum. This event was important because, for the first time, the organisation of the Summit proceedings was made comprehensively public, with its five action tracks, their staffing and agendas. The November 2020 event had the following as its first objective: ‘to agree on an initial set of bold actions and science-based principles to change the food system’ (UN 2020).

As one of the five action tracks, Action Track 2 (AT2) had the theme of ‘Shift to sustainable consumption patterns’. Its chair was Dr Gunhild Stordalen, who is the executive chair of the EAT Initiative, founded by the Stordalen foundation, the Stockholm Resilience Center under Professor Johan Rockström and the Wellcome Trust (EAT 2023). On the EAT website it says that Stordalen is ‘a published scientist…with a passionate commitment to food system transformation’, and that she is a Young Global Leader of the World Economic Forum, as well serving on the World Economic Forum Stewardship Board on Food Systems (EAT 2023). Stordalen described her ambition for her AT2 leadership as follows: ‘International summits rarely change the world by themselves, and especially not now given the global political climate. Our goal is therefore to take full advantage of the Summit to build an unstoppable global movement for change that we can keep growing well beyond the Summit, to force the kinds of far-reaching changes that the world now desperately need.’ (EAT 2020). How these changes will be achieved was made clear in the preceding EAT-Lancet Report, where Johan Rockström was the second author, and Christopher Murray (of the GBD) the last name author, as follows: ‘The scale of change to the food system is unlikely to be successful if left to the individual or the whim of consumer choice. This change requires reframing at the population and systemic level. Hard policy interventions include laws, fiscal measures, subsidies and penalties, trade configurations and other economic and structural measures….countries and authorities should not restrict themselves to narrow measures or soft interventions. Too often policy remains at the soft end of the policy ladder’ (Willett et al. 2019, p. 478). Essentially, the entire leadership of AT2 was staffed with persons closely associated with the EAT Initiative. The public positioning of the choice for AT2 leadership left no doubt, although not explicitly mentioned like this, that the AT2 aims were to significantly reduce the amount of meat consumption, accompanied by a corresponding reduction of livestock, and, by self-declaration as described above, to force through this transformation by authoritarian ‘hard’ measures against the will of the consumer where necessary.

By May 2021, the Summit preparations had reached a major impasse. Most of the various stakeholder groups representing farmers, civil society and industry could not find common ground with representatives from AT2 on the role of livestock in a global food system. The transformational reduction of livestock envisioned and argued for by AT2 did not find approval among most of the other concerned stakeholder groups. To unblock the situation, it was decided to arrange a solution cluster called ‘Sustainable Livestock’ cutting across all action tracks, with three focal-point individuals nominated respectively by the World Farmers Organisation (WFO), the International Livestock Research Institute (ILRI) and CIAT Bioversity. The author of this article was the focal point appointed by WFO and therefore has knowledge of the proceedings first-hand. The cluster was installed on 17 May 2021 and tasked to curate a two-pager solution paper among an open-membership working group with approximately 70 different organisations. Four weeks later, 15 June 2021, the cluster had created such a document, which found approval by all these stakeholders. Its tenor could be paraphrased with ‘Much improvement is necessary, but livestock is part of the solution, not part of the problem’ (the author’s summary, P. Ederer). Two days later, on 17 June 2021, the three focal-point individuals were asked by the Summit leadership group to incorporate additional input from other stakeholders that had chosen not to participate thus far. These were representatives from Chatham House, Wellcome Trust, Oxford University, New York University, the Good Food Fund, 50by40, the Good Food Institute, Compassion in World Farming and FAIRR, with each of them being closely associated with AT2. A week later, inputs received from this group were as follows: ‘That is nonsense’, ‘On farmer-driven roadmaps: in other areas such as energy, would it be acceptable for oil producers to decide a roadmap?’, ‘It is absurd to proposition a growth in the livestock as a solution’, ‘It is irrelevant that livestock farming has provided food, clothing, power, manure and income and acted as assets, collateral and status. Fossil fuel has done many of the same things’, ‘We are a 38 trillion investor network calling for sustainable agriculture system which importantly includes a shift to plant-rich diets and lower quantities of high-quality meat and dairy consumption’ and similar (these quotes are taken from email conversations that were shared among many members of the solution cluster and therefore are neither private nor confidential).

It is legitimate for the individuals from these organisations to hold these views and express them as part of a stakeholder exchange. However, the focal points, and indeed all proceedings throughout the Summit, had been asked to ground proposals in either solid scientific evidence or existing practiced solutions. It was the view of the focal-point leaders, that the contributions of the 17 June 2021 group met neither of these two criteria. Since the solution cluster ‘Sustainable Livestock’ again could not agree on a joint position after the 17 June 2021 group began to weigh in, the proposal was then to create three different two-page solution papers with one jointly drafted opening paragraph, respectively called A, B and C papers, and then each respective stakeholder group could provide its scientific and practice evidence for each preferred solution pathway. The three solution papers would then become part of the Summit proceedings and be the starting point for so-called post-Summit Coalitions of Action. The three papers are available as online Supplementary material to this article (until Summer 2022, they were available for public download on the UNFSS website). The A-paper called ‘Best Practices and Technologies’ was similar to the first solution paper presented in mid-June 2021, calling for an emphasis on best practices, technologies and diversity to be a guide for rapid change, without defined targets for livestock numbers, but an acknowledgement that not enough livestock products are available to the global population (again, the author’s summary, P. Ederer, see Supplementary material for the original papers). The B-paper called ‘Grazing for Soil Climate and People’ had a particular focus on grazing livestock. The C-paper called ‘Aligning Production to Consumption’ was essentially a summary of the positions of organisations closely cooperating with the EAT Initiative and AT2 that livestock numbers and animal-sourced food consumption need to be significantly reduced. By finding this compromise formula of three separate solution papers, the impasse was solved, and the Summit could proceed to its concluding pre-Summit meeting at the end of July 2021 in Rome, with every stakeholder group satisfied that its position was somehow represented.

Each of the three A, B, C papers was extensively referenced to scientific journal articles, public-policy documents or existing practice descriptions. The Scientific Council of the WFO (SC-WFO) checked the quality of the references of each of the three papers (the author of this paper being a co-author of the SC-WFO as well). In the A and B papers, the evaluation found three references each that were not correct, of 45 and 56 respectively. However, among the 53 references of the C-paper, 17 were wrong or irrelevant sources unrelated to the statement, 22 were relevant but said either nothing about or even the opposite of the statement, and 18 sources employed demonstrably weak or manipulative methodologies that could easily be disqualified (multiple mention possible). Only 11 of 53 sources were correctly attributed and supported the statement. That was the quality of the science provided by the C-group, which supposedly was the competence centre of AT2 and EAT on the subject (see Supplementary material for the detailed draft evaluation of the SC-WFO analysis. It was not officially entered into the UN FSS process, and is thus made available here for the first time to a wider audience).

Two examples shall showcase how manipulated the sources of the C-paper could be. In one instance, a statement was made ‘that significant reduction in global consumption of meat and dairy is needed….to minimise the use of antimicrobials’. This was referenced in footnote 15 to a well-known and respected Joint Scientific Opinion prepared by the European Medicines Agency and European Food Safety Authority in 2017 about antimicrobial agents. However, nowhere in that 245-page dense scientific treatise do these two agencies mention or imply that antimicrobial-agent use would be reduced, let alone minimised, via a reduction in the global consumption of meat and dairy. The second example is the well-cited Springmann et al. (2016). It was referenced twice in the C-paper of AT2 related stakeholders. Marco Springmann was also the co-author in the EAT Lancet 1.0 Commission, is a current Commissioner in the EAT Lancet 2.0 Commission, and was among the group who provided additional input on 17 June 2021. His 2016 article is called ‘Analysis and valuation of the health and climate change co-benefits of dietary change’. According to PNAS where it was published, the article has been cited 707 times, and has been viewed a sizeable 460,000 times, so it is a successful article. In that work, the authors claim to have calculated that if all of humanity switched to a vegan diet, then 8.1 million deaths could be avoided per year on grounds of improved health. The authors claim to have calculated this value based on the Global Burden of Disease (GBD) survey 2010. The vegan diet that they composed would avoid the GBD health risks of diets being low in fruit for 4.9 million avoided deaths, and low in vegetables yielding 1.8 million avoided deaths. Avoided deaths because of diets high in red meats falling away due to it being vegan, would contribute only 38,000 cases (according to GBD 2010, each value was extrapolated by some factor to adjust for population growth to then reach a total of 8.1 million avoided deaths). Clearly, the avoided death toll of the preferred diet had almost nothing to do with being ‘vegan’, and all with eating sufficient amounts of vegetables and fruits.

Springmann et al. (2016) is not a one-time exception in his publication record. In 2018, Springmann et al. published a follow-up study where they proceeded in the same fashion (2018 being funded by the Wellcome Trust and EAT, both among the 17 June 2021 contributors). They wrote ‘Progressively replacing animal-source foods with plant-based foods led to progressive reductions in premature mortality of up to 12% in 2030’. But in the next sentence, they admitted that only a tenth of that reduction would be due to a reduction in red meat, the other 90% is due to an increase in vegetables, fruits and legumes. These latter food groups can be easily increased without having to reduce red meat as they are not mutually exclusive. So, framing these benefits as beneficial for a ‘vegan’ animal-sourced-food-free diet is false. Moreover, the analysis sayid nothing about white meat, dairy or eggs, which are also animal-sourced foods. The argument that these diets are called ‘vegan’ as a proper description for the parallel analysis of the environmental impact is disingenuous because similar manipulations can be identified in those just as much in both the 2016 and the 2018 works. (In the same article, Springmann et al. (2018) also modelled that a reduction of obesity and overweight by curtailing the provision of calories would decrease premature mortality by 8% (p. 456, fig. 2). This disregards that obesity is not a simple function of overeating calories, and paying no attention as to how this would be practically implemented. It is, however, a further example of the manipulative practice of science by Springmann et al. 2018). Nor has Springmann distanced himself from these articles in the meanwhile. In a letter to the editor of Lancet Planet Health in June 2023 (Springmann 2023), Springmann wrote that ‘Eating a nutritionally adequate diet is possible without wrecking long term health, the planet or the pocket’, and cited the 2018 article Springmann et al. (2018) as his primary evidence. There is a little sign in Springmann’s work of NAS 2019 recommendation not to overstate the implications of research.