in

Review of studies sent by Dynacare and LifeLabs

Commissioned analysis of studies sent by labs

Allergies to food are established by elevated expression of IgE on testing. There has, however, been increasing marketing directed at identification of ‘food sensitivities’ by some private, for-profit labs, through food-specific immunoglobulin G (IgG) testing.

CBC Marketplace commissioned Jason Busse, Epidemiologist and Biostatician, to review 8 articles; 5 randomized controlled trials, 2 observational studies, and 1 review that had been provided by Dynacare and Lifelabs.

These papers were all very problematic.

In brief:

  1. The randomized controlled trials largely enrolled small number of patients, 2 reported large loss to follow-up (>25% of patients), and 1 was open label meaning that patients were aware of their treatment assignment. Three were funded by, or were authored by employees of, for-profit laboratories that appear to sell IgG food testing services. None of them were registered, which is a requirement of all major medical journals as a condition for publication. Analyses of all were problematic – either non-significant results are ‘spun’ by focussing on positive trends, significant results are reported that are not clinically important, improper observational comparisons (baseline vs. elimination diet) are reported, or invalid subgroups are reported that use post-baseline factors (i.e. diet compliance).
     
  2. The observational studies, both funded by for-profit laboratories or involving authors that are employed by private laboratories, are both very small (80 patients in total for both studies). One compares results to baseline among unblinded patients, and the other focusses on surrogates that are unimportant to patients.
     
  3. The review calls itself a systematic review, but it is not. It is an uncritical, narrative review that fails to identify the considerable problems with the primary studies it chooses to highlight.


In Summary

IgG food testing to derive therapeutic diets has not been validated nor supported by research, and food-specific IgG is to be expected, marking the presence of exposure and tolerance to a food. Accordingly, the Canadian Society of Allergy and Clinical Immunology, the American Academy of Allergy Asthma and Immunology, the Australasian Society of Clinical Immunology and Allergy, and the European Academy of Allergy and Clinical Immunology have all published position statements strongly discouraging the use of IgG testing to identify adverse food reactions. 1-4 None of the studies provided for review provide evidence to challenge these position statements. Further, I wonder about the selection of the 8 studies, which seems selective in that they all support IgG food testing, and other trials (e.g. Mitchell et al.  2011) that found negative results seem to have been missed.

The following are detailed critiques of the articles that you provided for review:

1. Atkinson et al., 2004

Food elimination based on IgG antibodies in irritable bowel syndrome: a randomised controlled trial

Risk of bias: This randomized controlled trials was at low risk of bias.

Funding: No funding source reported, although all testing was done thorough a private for-profit company (YorkTest Laboratories)

Trial registration: the trial was not registered, which is mandatory for all major journals to ensure was authors planned to do is what actually was done and reported.

Primary outcome(s): Authors declared a co-primary outcome of the IBS symptom severity score (0-500) on which they advise a reduction of 50-points is the minimally important difference (MID; the smallest difference that patients can detect), and a global rating of change. The global rating is problematic as it will be affected by recall bias in that patients are asked to recall how they felt 3 months ago for comparison.

Other serious problems:

the authors declare success of their IgG elimination diet on the basis that the mean difference in IBS symptom scores is 39; however, this is less than the MID of 50. A better analysis would have been to compare the proportion of patients between diet groups that achieved a reduction of 50-points or more. The authors then compound the problems of their analysis by focussing on the subgroups in each treatment arm that were fully adherent with the diets. This is a post-randomization factor that presents a serious threat to the balance in prognostic factors achieved by randomization.

All secondary outcomes are non-significant. It may be that the authors were lucky enough to find a significant difference in the 2 outcomes that they designated as their primary outcomes, but my confidence in this would be greatly strengthened if they had registered their trial in advance and openly declared their primary outcomes before starting the study.

The difference in primary outcomes between groups focusses on statistical significant vs. clinical importance (as noted above). Further, the associated p-values are 0.024 for IBS scores and 0.048 for global impact scores. A recent editorial in JAMA has called for lowering the p-value threshold to 0.005 to provide greater safeguards against spurious findings,5 which neither of these effects would meet.

2. Drisko et al., 2006

Treating Irritable Bowel Syndrome with a Food Elimination Diet Followed by Food Challenge and Probiotics

Risk of bias: This is an observational cohort study of 20 patients, in which everybody was treated, and then followed up at 6 and 12-months. As such, the study starts as low quality of evidence.

Funding: Private, for-profit funding by the BioCommunications Research Institute, which seems to be part of the Riordan Clinic: https://riordanclinic.org/research-studies/

Primary outcome(s): not declared

Other serious problems:

This study is called a “pilot study”, which typically means a small randomized controlled trial to establish the feasibility of a larger, definitive trial. Pilot studies are always underpowered to detect differences in treatment effects. However, this study seems to have no control arm, and is simply a 1-year follow-up of 20 patients who were all assigned a food elimination and rotation diet.

The authors note: “At one year after trial completion, a follow up questionnaire to assess gastrointestinal status was obtained; this was to evaluate for the role of placebo effect in this intervention, which is known to be quite high in IBS”. I agree that non-specific effects are important to consider when evaluating treatment effect for IBS, but the authors did not provide any placebo or have a control group, so I can’t see how they possibly accounted for this.

All treatment effects were compared to baseline data, among unblinded patients, which places the results at high risk of non-specific effects.

3.Wilders-Truschnig et al., 2007

IgG Antibodies Against Food Antigens are Correlated with Inflammation and Intima Media Thickness in Obese Juveniles

Risk of bias: This small, observational study of 60 adolescents explores the association between anti-food IgG levels and low-grade inflammation and early atherosclerotic lesions. Both of these outcomes are surrogates, meaning they have no direct importance to patients (no adolescent make an appointment with their doctor due to their concerns over early atherosclerotic lesions). As such, the clinical relevance of any findings is uncertain.

Funding: not declared, but one of the authors is employed by a private, for-profit labatory (Laboratoires Réunis Junglinster, Luxembourg) that performs IgG food testing.


4. Bentz et al., 2010

Clinical Relevance of IgG Antibodies against Food Antigens in Crohn’s Disease: A Double-Blind Cross-Over Diet Intervention Study

Risk of bias: this randomized controlled trial reported very high loss to follow-up (43%, 17 of 40 pts), which presents a serious risk to the findings (high risk of bias is typically assumed with 20% loss to follow-up), as the large numbers of missing patients risks losing the prognostic balance achieved through randomization. The authors also used a per-protocol analysis (instead of intention-to-treat) which is associated with exaggerating treatment effects, and is not recommended for randomized controlled trials exploring effectiveness.

Funding: funded by a private, for-profit laboratory (Evomed MedizinService GmbH, Darmstadt) that performs IgG food testing.

Trial registration: the trial was not registered, which is mandatory for all major journals to ensure was authors planned to do is what actually was done and reported.

Other serious problems:

This study is called a “pilot study”, which typically means a small randomized controlled trial to establish the feasibility of a larger, definitive trial. Pilot studies are always underpowered to detect differences in treatment effects. This may explain why the authors did not bother to do a sample size calculation (e.g. calculate how many patients they would need to detect a difference in treatment effect if one existed).

The authors captured pain, and well-being, and combined these outcomes with stool frequency to present a composite score. The effect on the composite was not significant (p=0.07) and yet the authors try and declare victory by stating “The estimated effect seems to have a clinically relevant effect, but is not significant (p = 0.07).” They note a reduction of stool frequency of 11% in the first trial phase, but which was reversed in the second phase (sham diet showed lower stool frequency than the specific diet) – they address this problem by only focussing on the first phase results, even though there was no evidence of a period effect (p=0.08).

5. Mullin et al., 2010

Testing for food reactions: the good, the bad, and the ugly

Risk of bias: this study is presented as a systematic review, but it has no features of a rigorous review. For example: (1) no search strategy is provided, (2) there was no risk of bias assessment of primary studies, (3) there was no assessment of the overall quality of evidence. This appears to be a largely narrative review that was called a systematic review.

Funding: none declared.

Other serious problems:

This review simply repeats the findings of select studies without critically apprising them. For example, the findings of Atkinson et al (2004) that suggest diet adherence shows better results for IBS symptoms is highlighted – despite the fact that re-analysing results according to a post-randomization variable presents a serious risk of bias to the prognostic balance achieved by randomization.

6. Alpay et al., 2010

Diet restriction in migraine, based on IgG against foods: A clinical double-blind, randomised, cross-over trial

Risk of bias: this small, 35-patients study is reported to be a randomized controlled trial, but there is no mention of how patients were randomized or allocated to treatment arms.

Funding: not declared, but two of the authors are employed by private, for-profit laboratories (Vivitro Ltd., Turkey and Invitalab, Germany).

Trial registration: the trial was not registered, which is mandatory for all major journals to ensure was authors planned to do is what actually was done and reported.

Other serious problems: although described as a randomized controlled trials, all the analysis reported focus on observational data –the authors claim success because the number of headache days and number of migraine attacks are less in the elimination diet arm as compared to baseline. This is the wrong comparison; what should be compared is the results of the elimination diet vs. the provocation diet. We know that non-specific (placebo) effects are prominent in trials of migraine sufferers, and so both arms would be expected to improve. The only results I can find that seems to compare treatment arms is as follows: “In comparison between elimination and provocation phases, for the number of headache days and attack count, reduction was ≥30% in 15 (50%) and 12 (40%) patients, respectively, and reduction was ≥50% in 6 (20%) and 4 (13%) patients, respectively.” If I perform a chi-squared test for the difference I proportions, I get p=0.44 and p=0.49, respectively, suggesting no difference between treatment groups.

7. Aydinlar et al., 2012

IgG-Based Elimination Diet in Migraine Plus Irritable Bowel Syndrome

Risk of bias: This small trial that enrolled 28 patients suffered from 25% loss to follow-up (7 of 28), which presents a considerable threat to the prognostic balance between treatment arms.

Funding: funded by a private, for-profit laboratory (Immuno Diagnostic Laboratories, Turkey).

Trial registration: the trial was not registered, which is mandatory for all major journals to ensure was authors planned to do is what actually was done and reported.

Other serious problems:

No primary outcome was declared, and no sample size calculation was reported.

Many inappropriate comparisons were reported that focussed on observational data instead of the randomized comparison. The authors focus on the results of observational data (elimination diet vs. baseline) instead of the proper comparison (elimination vs. provocation diet), possibly because the observational results are almost always significant.

The very large number of comparisons that were looked at would result in some statistically significant findings by the play of chance alone.

Results focus only on statistical significant, and not clinical importance.

8. Jian et al., 2018

Food Exclusion Based on IgG Antibodies Alleviates Symptoms in Ulcerative Colitis: A Prospective Study

Risk of bias: this is an open label study – meaning that patients knew if they were receiving the treatment or sham diet. This presents a serious risk of bias in that the outcomes were patient-reported.

Funding: the study was funded by “intestinal barrier research fund of Academician JieShou Li”, which seems to be not-for-profit, but I can’t be sure.

Trial registration: the trial was not registered, which is mandatory for all major journals to ensure was authors planned to do is what actually was done and reported.

Other serious problems:

No primary outcome was declared, or sample size calculated.

A number of outcomes are surrogates, and have no direct importance to patients (e.g. albumin, transferrin, and prealbumin levels)

Health related quality of life is not different between treatment groups, and so the authors focus instead on inappropriate observational comparisons of baseline vs. elimination diet.

If we focus on the patient important outcomes that the authors report in their Abstract to support the use of an IgG elimination diet: “After intervention, the Mayo score was significantly lower in the intervention group than in the control group (2.41 ± 0.89 vs 3.52 ± 1.15, P < 0.05). The number of patients with extraintestinal manifestations decreased from 7 to 2 in the intervention group and from 6 to 5 in the control group.”

A difference between groups in the Mayo score of 1.11 may be statistically significant, but it is not clinically important. Lewis et al. reported that a reduction of ≥ 3 points on the Mayo score and the partial Mayo score reflect a clinically meaningful change (Lewis JD, et al. Use of the noninvasive components of the mayo score to assess clinical response in ulcerative colitis. Inflamm Bowel Dis [Internet]. Dec, 2008. pp. 1660–1666.)

The authors do not report whether the change in extraintestinal manifestations between treatment groups was significant, and I suspect this is because it is not. If I perform a chi-squared test for the difference I proportions, I get p=0.23, suggesting no difference between treatment groups.

References

1. Carr S, Chan E, Lavine E, Moote W. CSACI Position statement on the testing of food-specific IgG. Allergy Asthma Clin Immunol. 2012 Jul 26;8(1):12. Available from: http://www.aacijournal.com/content/8/1/12

2. Bock SA. AAAAI support of the EAACI Position Paper on IgG4. Journal of Allergy and Clinical Immunology. 2010 Jun;125(6):1410. Available from: http://www.jacionline.org/article/S0091-6749(10)00512-9/fulltext

3. Australasian Society of Clinical Immunology and Allergy. Unorthodox Techniques for the Diagnosis and Treatment of allergy, Asthma and Immune Disorders – ASCIA Position Statement [Internet]. 2007 [cited 2014 Mar 10]. Available from: http://www.allergy.org.au/health-professionals/papers/unorthodox-techniques-for-diagnosis-and-treatment

4. Stapel SO, Asero R, Ballmer-Weber BK, Knol EF, Strobel S, Vieths S, et al. Testing for IgG4 against foods is not recommended as a diagnostic tool: EAACI Task Force Report. Allergy. 2008;63(7):793–6. Available from: http://onlinelibrary.wiley.com/doi/10.1111/j.1398-9995.2008.01705.x/abstract

5. John P. A. Ioannidis. The Proposal to Lower P Value Thresholds to .005. JAMA. 2018;319(14):1429-1430. doi:10.1001/jama.2018.1536