Errors of omission: Why we are deeply concerned about research on autism therapies

Last year, we completed a project to determine whether psychoeducational therapies for young autistic children are effective. Initially, we were excited: We had been able to track down 150 studies completed over the past five decades comparing treatment and control groups. Locating such a large number of studies would usually allow for a high degree of confidence in our conclusions.

As part of our protocol, we rated and summarized the rigor of the studies and planned to do a statistical analysis of only those that we deemed to be high quality. We quickly became disappointed, however: Across intervention types, there were too few high-quality studies for us to compute reliable statistics.

Inspired by conversations and social-media interactions with fellow autism researcher Michelle Dawson, we conducted two follow-up analyses, scrutinizing the ethical conduct of researchers who authored the studies we had compiled. One focused on the researchers’ disclosure of conflicts of interest (COIs), and the other on their reporting of adverse events. The results only deepened our concerns about how scientists conduct and report their research on autism interventions for young children.

Altogether, our analyses revealed three ethical problems: Most studies are poorly designed, fail to disclose COIs and lack attention to adverse outcomes. Disclosing COIs and taking steps to mitigate the biases they may introduce will lead to better-quality research and instill greater trust from the autism community. Proactively measuring and reporting adverse outcomes will enable families to adequately weigh the potential benefits of interventions against any potential negative consequences.

Conflict resolution:

We found that 70 percent of studies were co-authored by a researcher with a COI, which we defined as any situation in which a person holds a vested interest that might bias a study’s outcome. For example, a researcher who provides an intervention for profit stands to benefit from evidence suggesting the method is effective. That researcher’s interest in fairly evaluating the intervention conflicts with his potential for financial gain.

Having COIs does not necessarily constitute ethical misconduct, but failing to disclose them might — especially if a journal requires it. We found that only 6 percent of the studies with COIs had actually reported them.

One conflict we found especially worrisome was when the investigator was the developer of the intervention. Developers stand to gain from career advancement, training others to use their intervention via for-profit workshops and selling manuals describing the intervention. However, none of the studies we evaluated had disclosed this particular COI, even though most interventions were studied by the same researchers who had developed them.

We surmise that the low quality of autism-intervention research to date is related to this paucity of COI reporting. Biases associated with having COIs may lead autism-intervention researchers to make questionable methodological decisions that compromise research integrity and quality. For instance, a researcher with a COI may choose a flawed study design in which participants are not randomly assigned to treatment and control groups, or may use a measure that is vulnerable to placebo effects. These types of flaws increase the likelihood of getting favorable results, even when the interventions are not actually effective. COIs may also bias researchers’ interpretations of their results, leading them to claim their studies provide strong evidence of effectiveness when in fact the results are mixed.

Additionally, when researchers collectively and routinely fail to disclose COIs, there is no pressure from either the research community or the autism community to reckon with the effects these COIs may have. Essentially, COIs have remained out of sight and out of mind for many researchers — meaning that the problems they introduce are not being addressed.

Unfortunately, it is not always clear what constitutes a COI; there is no exhaustive list of the kinds of relationships that should be considered as such. We therefore urge researchers to think carefully about any potential COIs that they or their team members may have and to disclose them. Such disclosures may compel researchers to design unbiased studies and to be clear about how they prevented their COIs from influencing their research results (such as by excluding themselves from data collection and statistical analysis).

Facing adversity:

Our other follow-up looked at the extent to which autism-intervention researchers measured and reported adverse events, adverse effects and harms. Adverse events include any occurrence of psychological or physical distress; other adverse outcomes that might be reasonably attributed to an intervention are deemed adverse effects. Harms refer to adverse events that remain long after an intervention has ended. For example, if a child develops a fever while receiving an intervention, researchers should document this as an adverse event. If a child breaks her arm on a play structure during an intervention session, this should be reported as an adverse effect. If a child becomes fearful of a particular toy used during an intervention and remains afraid of it several years later, this should be recorded as a harm.

We found that, similar to COIs, adverse events are rarely reported. Only 7 percent of the 150 studies we examined mentioned adverse events. None mentioned harms, nor described systematic procedures for measuring adverse events.

Because autism research has a history of using punishments that induce pain as an intervention strategy — some of the first behavioral therapists gave electric shocks or slaps to children to discourage certain behaviors — we believe it is especially important for autism-intervention researchers to proactively monitor adverse outcomes. Although contemporary interventions tend not to use punishment, unintended negative consequences are still possible.

To get some insight into these unintended consequences, we compiled a list of the adverse effects reported in the literature, along with any reported reasons for a participant’s withdrawal that met the definition of an adverse effect but were not labeled as such. We also scrutinized results indicating that children had less optimal outcomes when they received the treatment of interest than when they were in a control condition.

The adverse effects we identified included major and minor physical trauma, child distress and dislike of the intervention, caregiver anxiety and stress, and poorer outcomes in several areas, such as social communication, restricted and repetitive behavior, language, play and social-emotional behavior. The majority of the participants in the studies we reviewed likely did not experience adverse effects, but our findings still suggest researchers should monitor for and record them.

We were not able to find any data on the potential harms of autism interventions, but many autistic adults have expressed that they experienced harm — even trauma — as a result of the interventions they received as children. Researchers should conduct follow-up evaluations of study participants long after interventions are over, to document possible harms that arise or persist over time.

We hope that bringing these issues to light will lead to greater awareness of these pitfalls, improve researcher practices and ultimately lead to better services to support autistic children and their families.


Kristen Bottema-Beutel is associate professor of teaching, curriculum and society at Boston College in Massachusetts. Shannon Crowley is a doctoral candidate in the curriculum and instruction program at Boston College. Micheal Sandbank is assistant professor of special education at the University of Texas at Austin. Tiffany Woynaroski is assistant professor of hearing and speech sciences at Vanderbilt University Medical Center in Nashville, Tennessee.

The post Errors of omission: Why we are deeply concerned about research on autism therapies appeared first on Spectrum | Autism Research News.

Order by: 
Per page:
  • There are no comments yet
Related Feed Entries
The post Spotted around the web: Suicide rates, TCF4, rapid screening, face masks in an MRI appeared first on Spectrum | Autism Research News. Source: Spectrum News
2 days ago · From Spectrum News
Mutations that affect a histone called H3.3 can lead to a neurodegenerative condition marked by developmental delay and congenital anomalies, according to a new study. Histones act as spools for DNA, making it possible to pack the strands of genetic material tightly within the nucleus. They also ser…
2 days ago · From Spectrum News
Fusing dissimilar spheres of neurons enables researchers to model the circuit differences seen in a genetic condition linked to autism, a new study shows. Merging the 3D clusters in a lab dish at will — “almost like Legos” — can reveal how defects arise as neural circuits form, says lead investigato…
3 days ago · From Spectrum News
Mutations in the gene DNMT3A disrupt how other genes are turned on and off, and can lead to a range of neurodevelopmental conditions, a study in mice has found. The results hint at how mutations in DNMT3A, seen in people with autism and a related condition called Tatton-Brown Rahman syndrome, interf…
4 days ago · From Spectrum News
Art of Autism Creatives Share ‘Wassup’ In the New Year Compiled By Keri Bowers with help and special thanks to New Art of Autism Ambassador Linish Balan This Mash Up is dedicated to all of us who are emotionally and physically exhausted from the high-intensity dramas of 2020. Our wish is we can tap…
4 days ago · From The Art of Autism
0 votes
25.11.2020 (25.11.2020)
0 Subscribers