Flawed findings: conspiracy theories masked as academic analysis
By Katherine Rich
In an era when consumers are sceptical of ‘fake news’, a further concern is brewing.
Public health is one field where such concern is relevant – where sometimes the effort to advance knowledge and public benefit is threatened by the use of exaggeration and flawed methodologies wrapped in emotive but persuasive language.
Of concern within food circles is what seems to be a new academic fashion of over-analysing email exchanges and using them to affirm food industry conspiracy theories.
A good example of this is a paper published last year in the Journal of the American Medical Association, which used relatively innocuous correspondence to allege that the American Sugar Association had undue influence on the academic work of three Harvard professors. The story went global and the reputations of the respected professors, no longer alive to defend themselves, were damaged. (see my previous Soapbox article in Food Navigator-Asia “Sugar Review: Rewriting history to expose a non-existent conspiracy”).
Social science is an important endeavour, but the techniques used in these sorts of “studies” do not qualify as scientific process in any sense of the term. Its worrying effect is to cast serious debate and the genuine science behind it into the shadows just when science-based illumination and debate is most needed.
The most recent glaring example of this is a paper published in the journal ‘Critical Public Health’ under the names of Gary Sacks, Boyd A Swinburn, Adrian J Cameron, and Gary Ruskin, entitled ‘How food companies influence evidence and opinion – straight from the horse’s mouth’.
Once again it’s a paper based on academic analysis of the correspondence of others, which extrapolates the exchange to imply it is representative of the entire global food industry.
It’s hard not to conclude that with this paper we might have reached peak social science silliness.
So what is this email, which is treated like a food industry Watergate tape, all about?
It was written by Dr Michael Knowles, a man whose life’s work has been to ensure good science underpins food decision-making. In it he shares a view with a colleague, Dr Alex Malaspina, about how the food industry should respond to “biased, non-scientifically-based recommendations” concerning key industry issues such as obesity and causative factors, sugar, and low/no calories sweetener safety. Knowles’ suggestion is to encourage learned discussion among scientists in professional societies. And that’s it.
But the authors have chosen to interpret this suggestion in the darkest possible light. Despite not even knowing Knowles, his organisation, his work, or his motivations, the authors present his support of evidence-based discussion as being some kind of Machiavellian food industry plot. This is unfair and not substantiated by the evidence.
What follows below is a detailed analysis of the academic paper. It’s long, probably to be read only by those with a deep interest in this issue, but I believe it’s important that all misleading claims, exaggerations, and selective quoting are addressed in a way that both sets the record straight and exposes the lengths that some papers go to, to discredit and demonise the food industry.
From this analysis my hope is that the reader will agree it’s preposterous to suggest one email is ‘smoking gun’ evidence that the food industry seeks to infiltrate and manipulate professional associations, but that’s basically what the paper concludes.
Here’s why I believe the academic paper is flawed and the attacks on Dr Knowles unsubstantiated and appalling …
Inconceivably small databases – one email
The ‘horse’s mouth’ of the paper is a single (yes, just one!) email from 2015 between the two former employees of Coca-Cola. Both men have strong qualifications and are still passionately involved in positive global food and beverage discussions.
One. Single. Email. The smallest database possible. Hardly the Rosetta Stone.
It’s difficult to conceive how a selective analysis of one email can be accepted by a reputable, refereed journal like Critical Public Health as an adequate basis for making sweeping claims in respect of entire industries and groups of directors and managers. An unknown graduate researcher could not expect to gain such traction.
The authors make such evidence sound much grander than it is by using phrases that minimise the fact that this is one email from two former industry executives. A few examples where the authors have well and truly over-egged the pudding with grand talk:
- “This paper provides an analysis of an email exchange”
- “The results provide direct evidence”
- “The paper reveals industry strategies”
- “This paper analyzed direct communications”
- “The email exchange provides a high-level perspective”
- “The tactics displayed by these food industry leaders”
- “Despite the limitations of the evidence analyzed, the email exchange provides a high-level perspective …”
One innocuous email between learned colleagues reasonably suggesting that science underpin food discussions cannot make a conspiracy of Watergate proportions.
Omission of key context
Is it reasonable to take an email sent to a man who left a company many years ago and claim that it’s representative of a company today? The paper’s authors apparently think so.
The authors rely heavily on the fact that they’re analysing an email between two “former senior executives of Coca-Cola to gain insider insight”. They’ve treated Knowles’ typed words with the same solemnity as if they’d been uttered by the Coca-Cola Company CEO at the time, Muhtar Kent.
Both gentlemen had retired years before the email was written – the recipient hadn’t worked for Coca-Cola for at least 15 years, while Knowles had retired in 2013.
Though the paper points out that the two are former employees, that doesn’t stop the authors from furthering the conspiracy theory and extrapolating their findings to cover the entire global food industry.
The length of time both gentlemen had been gone from Coke is significant context that the authors have not fairly reflected, making it patently absurd to promote the idea that this single email (interpreted through a rather warped lens) is important “insider insight” that reflects the Coca-Cola company or the whole food industry in 2017.
To recap: that’s one email between two guys who don’t work for Coca-Cola, and hadn’t for many years. Undeterred, the authors imply that the email is indicative of multiple “industry strategies”.
Self-referenced truths: the 'I’m right, because I say so' approach
The paper’s references fall into two main groups – the authors’ own work or sensational general media stories. It happens a lot in academic work, but it always seems somewhat dubious when papers include the practice of making assertions such as “it is well established that …” and then quote the author’s own work as the evidence. In no sense can continual quoting and re-quoting of one’s own papers be seen as providing a satisfactory form of independent verification.
Examples in this paper:
- “The tactics used by companies in the food industry to influence public policy have been well documented (Mialon, Swinburn, & Sacks, 2015 Mialon, M., Swinburn, B., & Sacks, G. (2015).” - quoting themselves.
- “The content of the email communication was analyzed using a framework for classifying the tactics employed by food companies in their efforts to influence public policy in their favour (Mialon et al., 2015) Mialon, M., Swinburn, B., & Sacks, G. (2015).” - referencing themselves and quoting their own invented framework that sees industry offering a view on anything or participating in debate as being ‘corporate political activity.’
- "These findings support previous analyses that have examined the tactics used by the food industry (Mialon, Swinburn, Allender, & Sacks, 2017.” - \agreeing with themselves.
When the conclusion is that this paper “reveals industry strategies”, would a reasonable person expect there to be a stronger evidential platform than one email, academics quoting themselves, and a few newspaper stories? Most people would say yes.
The halo effect of academic publication
The words ‘research’, ‘academic paper’ and ‘journal’, create the impression that there is some higher process applied compared to that of an op-ed in a newspaper, a call to radio talkback, or even a food industry CEO writing for Food Navigator-Asia.
With this paper, it’s hard to see any other process at play than the authors looking at an email and expressing an opinion. How this paper managed to get published by such a respected journal is for the journal to justify.
It would be interesting to know who the peers in peer review were, to judge whether it was an objective process, and whether those peer reviewers and all the authors actually read Knowles’ email in full before giving the paper the big tick. It’s hard to understand how anyone could read the email in full, and within full context, and judge that it was anything other than a knowledgeable man suggesting science-based discussion.
Bullying by association
The use of institutional affiliation, disciplinary qualification, and historical reputation does not constitute nor deserve to be treated as independent verification of otherwise unsupported assertions and allegations. The email’s author and recipient (and their backgrounds) cannot be the evidence.
- “At the time the email was sent, neither Knowles nor Malaspina were employed by Coca-Cola …” - but this paper’s authors do treat the one email with the same seriousness as if Knowles and Malaspina were still employees and speaking on behalf of the company and wider food industry.
- “But emails reveal that Malaspina remains in contact with Coca-Cola executives …” - so what? Many people who spend their lives working in a certain company or field continue to be in contact with their colleagues and friends. This is completely normal.
- “… while Knowles continued to serve as a trustee of ILSI through 2016 and of the ILSI Research Foundation through 2017”. - again, so what? Being a trustee of an independent research foundation, which is a matter of public record, is not a sinister or controversial activity.
The sin of selective quotation
The paper grossly distorts Knowles’ message by selectively quoting what he said. This sin is three-fold: where his words and sentiment have been inaccurately summarised, presented out of qualifying context, or edited so the message is changed to create a negative impression.
The effect is to create the impression of a master plan to manipulate discussion, when most people reading the email in full, and with knowledge of Knowles’ work, would see it was no more than two colleagues discussing how to further science-based discussion. By selectively quoting parts of the email that are most helpful to their argument, the authors overlook Knowles’ comments about the need for an evidence base, transparency and scientific consensus.
For example, the authors claim Knowles suggests the food industry must generate its own evidence relating to the causes of obesity and food safety, “both through the manipulation of external organisations and through directly commissioned work”, and use this following quote from his email as proof of that:
“As to the generation of credible, consensus science on the issues hitting the industry – obesity and causative factors, sugar, low/no calorie sweetener safety – in particular we have to use external organizations in addition to any work we directly commission.”
The sentence has been clipped and is not presented accurately. In his email, Knowles’ sentence finishes … (and that needs to be very carefully reviewed in light of the BMY article)! He then goes on to give examples of ILSI (International Life Sciences Institute), Scientific Societies, Medical Associations, National Academies of Science, and EU/government research.
(ILSI is an organisation that aims to “improve public health and well-being by engaging academic, government, and industry scientists in a neutral forum to advance scientific understanding in the areas related to nutrition, food safety, risk assessment, and the environment.” When ‘tripartite’ is used in the context of ILSI it refers to ‘industry, academia and government’. ILSI’s Board has equal numbers of industry and public representatives and is always chaired by a leading academic.)
Under ILSI, Knowles reasonably says that contentious issues need to be addressed by all stakeholders i.e. under a tripartite procedure, which is probably and simply ILSI’s way of getting industry, academia, and government to work together.
The paper continues presenting as proof that Knowles was seeking to “exert influence over scientific bodies and medical associations”, by quoting this from the email: “We do have good contacts in some [medical associations] and we should encourage them to address public health matters and ‘suggest’ appropriate topics”.
Once again, that’s not the full quote. It’s a small difference with a big impact. A full stop has been added, when in Knowles’ email there’s a dash that leads to a significant and reasonable qualification. The context is important because it goes on to explain what Knowles means.
The full section reads: We do have good contacts in some [medical associations] and we should encourage them to address public health matters and ‘suggest’ appropriate topics – I belong to the Apothecaries Society (it’s a very old, traditional medical – 90% medics – organization in the City of London) and they may be interested – they give courses in ‘catastrophe medicine’ so perhaps our issues qualify. I will talk with my friend Sir Colin Berry who’s a Past President of the Society and has advised us on several issues in the past, worked with ILSI and Tox forum – a vg toxicologist and medic.
It’s clear that this is an information exchange of ideas between people with a science background talking about how to encourage science-based discussions. Only the most sinister interpretation would lead to the suggestion that this is evidence of a dark strategy to manipulate global food policy.
Still under the heading ‘Exert influence over scientific bodies and medical associations’, the authors use as further evidence the quote:
“We can … suggest some form of debate on the issues [at the National Academies of Science], ensuring of course that the debaters are balanced!
Again, this is selective quotation and offering part-sentences for negative effect.
Here’s what Knowles actually said:
National Academies of Science: In EU we do have an association of these bodies and the UK’s, The Royal Society, has recently opened a ‘food science’ theme and we can talk them through the IFST, of which I’m a Fellow, to suggest some form of debate on the issues, ensuring of course that the debaters are balanced!
In the context of the whole email it’s clear that Knowles’ comment is not a suggestion that science or evidence-based discussion should be manipulated, which is the impression created by the edited quote presented without context.
A further quote is presented in the paper as evidence of exerting influence:
“We all belong to one or more of these [scientific societies] and we should have leadership roles in the key ones and push for individual issues to be addressed by public conference/workshops.”
This comment is presented in the paper out of order in terms of comments made in the full email, but once again Knowles is not accurately quoted and it’s an unfair omission.
This is the full quote:
Scientific Societies: we all belong to one or more of these and we should have leadership roles in the key ones and push for individual issues to be addressed by public conference/workshops in the manner of ILSI above.
As noted earlier, in speaking about ILSI, Knowles was clear that to him this meant, the generation of credible, consensus science [for issues such as obesity,] that such contentious issues need to be addressed by all stakeholders i.e. under the tripartite procedure used by the institute to engage people in the discussion.
Under a section headed ‘Relationships with policy-makers and opinion leaders’, the authors say Knowles “advocates broad-based collaboration with government and key opinion leaders, and involvement in nutrition-related government reviews of the evidence base”, and use this quote from the email as evidence: “Contentious issues need to be addressed by all stakeholders [including industry, government, and academia].”
But again, this is a partial quote that misleads readers as to what Mr Knowles actually said, which is here: ILSI was formed by you for the very reason that such contentious issues need to be addressed by all stakeholders i.e. under the tripartite procedure. He then goes on to emphasise that work should be done in the traditional manner of ILSI – in a transparent manner with the best international experts and the full proceedings published…
Sipping the Kool-Aid - exaggeration, false claims, half-truths
As pointed out, there are some disappointing half-truths in the paper that contribute to the dark (but nonsense) conclusion that Knowles’ email is evidence of a global conspiracy to trounce scientific debate about food.
Knowles’ comments are presented out of the clear context that he was promoting more discussion in learned societies based on the science.
The paper makes many claims that are not backed up by the contents of the email. They include:
“The tactics used by the food industry to influence public policy have been well documented …” - this is opinion, not fact. No references are given.
- “The results provide direct evidence that senior leaders in the food industry advocate for a deliberate and co-ordinated approach to influencing scientific evidence and expert opinion.” - no they don’t. When the selectively quoted emails are examined in context there is no evidence of anything except two colleagues talking about furthering evidence-based discussion amongst experts.
- “The paper reveals industry strategies to use external organizations”. - no, the single email reveals a single scientist sharing his view about encouraging science-based discussion.
- “It also demonstrates the importance of identifying and managing potential conflicts of interest when assessing the evidence base and when making policy decisions related to nutrition and NCD prevention.” - this is always good practice, but the information from the single email does not “demonstrate” this at all.
- “The email exchange notes several tactics, outlined by these senior food industry figures, by which the food industry deliberately seeks to shape the evidence base on diet- and public health-related issues, and establish relationships with governments, key scientific bodies and medical associations in order to have influence over them.” - by selectively quoting only parts of the email, the authors overlooked comments about the need for an evidence base, transparency and scientific consensus.
- “Furthermore, Knowles praises industry efforts (through the IFIC Foundation) to criticize government recommendations. Specific tactics include highlighting the limitations of existing (non-industry sponsored) evidence and focusing on doubt in science.” - no, he does not. This is completely unfair and wrong. This allegation is not backed up by any quotes. There is nothing in the email that has Knowles saying this. It could be referring to his comment: This IFIC media call is a great example of how the industry should respond to biased, non-scientifically-based recommendations. He says nothing that praises industry efforts “to criticize government recommendations”. He does say that responding with science-based comments hopefully will also demonstrate to governments that they must have credible scientists in their advisory committees, or else they risk being made to look foolish. Fair enough. Most people would think that’s a reasonable position. Nothing in this comment can be reasonably interpreted as Knowles praising industry for criticising government recommendations.
- “Knowles highlights the need for food industry representatives to use their positions on influential scientific bodies and medical associations to direct debate and discussion towards issues of interest to the industry.” - Dr Knowles does not say this. When the email is read in full and his comments are in their correct context, it’s clear Knowles was aiming for, as he refers to it, credible, consensus science on issues hitting the industry – obesity and causative factors, and that contentious issues need to be addressed by all stakeholders i.e. under a tripartite procedure. [academia, industry and government]
- “He also suggests that industry representatives should seek key leadership roles in such societies.” - this is not a fair reflection of what Knowles actually said in the email: Scientific societies: we all belong to one or more of these and we should have leadership roles in the key ones and push for individual issues to be addressed by public conferences / workshops in the manner of ISLI above. Calling for public workshops? Raising issues such as obesity? Seeking consensus of contentious issues through public discussion? This is hardly cloak and dagger stuff. Whether they are from industry or academia, scientists will be members of professional groups. Where better to have science-based discussions that can inform policy?
- “Knowles emphasises the need for the food industry to guide global debate and, in particular, to use their academic contacts to facilitate this.” - this misrepresents what Knowles said. He suggests participation and engagement, but nowhere that industry “guide” anything. The authors are overstating what he actually said.
- This paper provides direct evidence that senior leaders in the food industry advocate for a deliberate and co-ordinated approach to influencing scientific evidence and expert opinion. Importantly, the food industry seeks to do this by co-opting academic contacts, infiltrating major scientific bodies and medical associations, and influencing the generation of scientific evidence. External organizations, and the people associated with them, are depicted as pawns to be used to overcome the global scientific and regulatory challenges faced by the industry. - the authors seem to think that scientists in industry have no right to be members of such organisations. As for the suggestion that external organisations and the people associated with them are pawns – that is emotive stuff that is usually edited out of quality academic journals.
In their grand conclusion to the paper, the authors state:
“Companies that profit from the sale of unhealthy food have a clear conflict of interest in relation to NCD prevention, and are unlikely to have the public’s health as a motivating factor (Moodie et al) The tactics displayed by these food industry leaders to influence the scientific evidence base and global debate in relation to nutrition and food represent a substantial risk to efforts to address NCDs globally.”
There’s nothing in the email that supports any of this. Basing a claim of tactics by food industry leaders to influence scientific evidence base and global debate on this one, perfectly reasonable solitary email is ridiculous.
Finally: “The public health and medical community need to be aware that they are viewed as tools through which food companies can overcome threats to their profits.”
It’s hard to believe an academic journal would publish an emotive, non-science-based comment like that, but once again, that’s up to Critical Public Health to justify.
When the email by Michael Knowles is read in full, it becomes very clear there is no ‘smoking gun’ here – rather, a reasonable discussion about informing decisions in a transparent way that uses science and evidence.
All he is saying is let’s have a look at the science, and let’s have some more debate in the learned societies, based on that science. There’s nothing wrong with that. In fact, it’s the perfectly correct thing to do.
The authors have packaged up one email between two retired colleagues, selectively clipped comments and presented them with all the fury and outrage of Watergate. Michael Knowles deserves an apology, and the paper should be retracted.
Final note: disclosure statement
The authors end with this disclosure statement:
“GR is employed by U.S. Right to Know, a 501(c)(3) nonprofit organization. The major funders of U.S. Right to Know are the Organic Consumers Association, Dr. Bronner’s Family Foundation, CrossFit Foundation and the Westreich Foundation. GS, BAS and AJC are the academic partners on a supermarket intervention trial that includes Australian local government and supermarket retail (IGA) collaborators. No funding was provided specifically for this study.”
A more complete statement might have contained the following important sentence: “In New Zealand, BAS is involved in legal proceedings against the New Zealand Food & Grocery of which Coca-Cola is a member company.”
(as published in Food Navigator-Asia, 25 September 2017