Three researchers expose critical methodological fallacies and flawed analysis in the TransPulse project conclusions, or: There is no evidence for parental affirmation benefits.
The following article was reviewed by three researchers, from Canada, UK and Australia:
Travers R.,, Bauer., G, Pyne., J., Bradley., K.,, Gale., L., & Papadimitriou., M. (2012) Impacts of Strong Parental Support for Trans Youth. Trans PULSE Project; (Accessed 30 March 2023)
Author: CK Moruzi, Canada
Based on a survey analysis, Travers et al conclude (or strongly imply) that there is a causal relationship between parent support of their child’s new Gender identity and the health, psychological well-being and life outcomes of those identifying as transgender. Their aim is to provide data on the impacts of parental support on trans youth aged 16 to 24, and to shape policy and inform service providers and the greater community on the needs of this population. Their aim is commendable. However, there are a number of issues that must be considered if this report is used to inform current policy.
The data that they use is from 2009-2010. It is well documented that there has been a significant change in the profile of those identifying as transgender. Since it is possible that the data in this survey is not reflective of transgender youth today, it may be imprudent to base policy on this data rather than data on current trans youth.
While the researchers collected data on 4 levels of perceived parental support (very supportive, somewhat supportive, not very, not at all), they chose to report only on the two extremes: very supportive and not at all. Transparency by presenting all 4 levels would allow for a better understanding of the relationship, as the relationship may not be linear as is implied.
While the researchers acknowledge that there were no statistically significant differences on particular outcome variables, they discuss these findings, nonetheless, as though they are significant, which negates the point of hypothesis testing. Perhaps a power analysis would have helped in making the claim that null effects were simply due to lack of statistical power.
All of the variables used, with the exception of the Rosenburg Self Esteem and CES-D Scale are self-made Likert-type scales and are likely have insufficient reliability or validity. If respondents are able to determine what the researchers are attempting to examine, which may be the case here, the use of such variables is especially problematic. Even their predictor variable of parent support is not a precise measure as it is difficult to disentangle general parental support from support of transgender identity. Exploration of data such as this is important to inform further research, but it lacks the rigor necessary for policy-making.
The researchers do not conduct any multivariate analysis to consider the role of any covariates that might contribute to the findings. In order to make such strong claims about the role of parent support, more sophisticated analyses are required.
One such variable that they did not control for, which may be most significant with regards to the current trans youth population, is when the child first exhibited behavior that might be considered to be indicative of being transgender or when their transgender identity was declared. It is entirely possible that those who have supportive parents are those who exhibited signs of being transgender early on (child onset). There could, in fact, be two populations, early onset vs adolescent onset. If this is the case, the causal claim they make regarding parent support falls apart and the results may be due to the two different populations. There is a significant increased risk of psychopathology in the current cohort who identify as transgender in adolescence. If transgender identity declared in adolescence is generally associated with a greater risk of psychopathology, this further complicates the relationship the researchers are attempting to declare.
The youth outcomes the researchers want to predict are, indeed, important. However, they are likely to be the result of a complex set of interacting variables including those that mediate parent support. In order to develop policy that accurately reflets the need of transgender youth, and what measures are necessary to support those needs, more comprehensive study is necessary. The Trans Pulse survey and its analyses lacks the rigor required to make informed policy.
Author: Anonymous, UK
The authors of the 2012 study themselves explicitly warn that "we cannot make causal assertions" using their study findings. In another paper based on the findings of the same 'Trans Pulse' questionnaire, the authors explicitly warn that their “findings must be interpreted cautiously. our research is both exploratory (hypothesis-generating) and analytical in nature, and given the cross-sectional study design, we cannot make causal inferences.” When listing the study's limitations, they repeat: “our study is cross-sectional; thus, we cannot make causal assertions.” However, in an earlier data presentation, they label slides on psychological functioning 'Impact of parental support', clearly suggesting that their findings were causative.
The study is not peer-reviewed or published in a journal, but merely a self-published document posted onto the internet. The findings come from a single question (N10) in an 87-page survey: "In general, how supportive of your gender identity or expression are the following people?" Nowhere is the notion of support explored or defined. Nowhere are the views of parents assessed. Respondents may each have had different ideas and standards of what 'supportive' meant.
Recruitment used a chain-referral method known as 'respondent-driven sampling' and took place in 2009-10, via “16 well-connected trans people” centred around the Sherbourne Health Centre in Toronto (later supplemented by a further 22 ‘seeds’), who recruited their friends, who in turn recruited their friends etc, using a financial incentive. Respondent-driven sampling is a new way of accessing 'hard-to-reach' minority populations. Researchers have criticised it as "heavily dependent on often unrealistic assumptions" and subject to "substantial bias". [source] "Caution is needed when interpreting RDS study findings." [source] "The extent to which RDS-derived estimates are valid and generalizable remains a source of controversy in the peer-reviewed literature." [source]
The significant dissimilarity of survey respondents to the large groups of present-day gender dysphoric adolescents is provided by the pen portraits of the project's co-ordinators, published within the survey. Among these were Will, "a former lesbian feminist turned tranny poststructuralist" whose hobbies included "completing home renovations and reading critical theory". [p. 10 of the survey]; Nael, who is "passionate about queer, transgender, post-colonial and diasporic theory". [p. 15 of the survey]; and Liz (aka Raven), "a 2-Spirited Transsexual Warrior" and former "Prostitute, Bank robber, Heroin/Cocain addict, and thus ending up in Federal Prison for 5 years." [p. 21 of the survey]; and Caleb, "a white, 24 year old, queer-identified trans guy who is currently completing his MA in Sociology" whose "current research interests include the negotiation of trans subjectivity ... theorising the body... anti-psychiatry, madness, and whiteness studies." [p. 28 of the survey]. The survey was peppered with cartoons about transition surgery. The 2012 report clarifies that their analysis is based on only 84 socially-transitioned adolescents and young adults. Of these, 28 or 29 (34%) had 'very supportive' parents, 21 (25%) had parents 'somewhat supportive', and 35 or 36 (42%) had 'not very' or 'not at all' supportive parents.
The recruitment method was especially ill-suited to test the hypothesis the researchers developed (i.e. that there was association between parental 'support’ and positive self-reported psychological functioning), for it overlooked children who were happy in their home environment and had consequently not required the support of advocacy groups. This group was likely to be under-represented in the sample.
Findings from this population are not generalisable to a present-day adolescent-onset female which is very different from the transgender population of the early 2000s. One indication of the difference is that, in an accompanying powerpoint presentation (slide 8) 59% of respondents stated their gender dysphoria was childhood-onset. As we are not told the ages of the participants (only that they belong in the 16-24 bracket) they might all be young adults aged 23 and 24 (and the pen-portraits offered in the survey strongly suggests that this is largely a survey of adults) again, making the results non-generalisable. The survey questionnaire was also peppered with humorous cartoons about sex reassignment surgery, again suggesting strongly that it was intended for an adult audience. The cartoons and the pen portraits also represent a form of biasing participation, for someone who did not identify themselves in either might be less likely to participate. Finally, we might question how many 16 year olds would bother to complete an 87-page questionnaire.
The report tells us very little about the respondents. We do not know their sex, or their age, or the age from which they identified as trans. We do not know their health needs or any comorbid diagnoses, we don't know their family situations (siblings, cultural background, educational attainment, employment status and income, religion, parenting). We don't even know their gender identity ("a broad definition of 'trans' was used"). Any one of these might be a significant confounding factor. For example, participants are asked to rate their mental wellbeing but we are not told anything about their mental health diagnoses or what treatment they might be getting. 100% of the 'very supportive' group had adequate housing, but only 45% of the rest did - which meant (as defined in this study) that they were either homeless or in insecure accommodation (eg sofa-surfing) or in prison. This strongly suggests that we are not comparing like with like here: for obviously, if you are homeless, your mental health is very likely to suffer. Only 15% of the 'somewhat to not-at-all supportive' considered they had very good mental health, vs 70% of the 'very supportive' group. Thus the findings in Figure 2, on depression and suicidality, could simply reflect prior mental health problems. Indeed, the mental health disparities could alone account for all of the other findings. The mental health problems experienced by this population were real and tragic; indeed, one of the coordinators killed himself in 2012.
Critically, the underlying survey data does not support the study's hypothesis that the level of parental 'support' has an impact on children's psychological outcomes. In the study report, the answers for the 'very supportive' group are contrasted with all others - whether 'somewhat supportive', 'not very' or 'not at all' supportive. Because the study's authors have not split up 'somewhat' and 'not at all supportive', it is not possible to conclude that strength of supportiveness is correlated with good outcomes. In fact, data from the same survey that was presented in an earlier powerpoint presentation shows that it threw up contradictory findings that undermine the authors' conclusions, but which they masked by grouping together the somewhat/not very/not-at-all categories (slide on page 29 of the 2012 Trans Health Plenary presentation). Taking as an example respondents' reported life satisfaction, data given in the presentation shows that just over 70% of 'very supportive' parents were satisfied with life, fewer than 20% of the 'somewhat supportive' group were satisfied, and almost 45% of those with ‘unsupportive parents’ were ‘satisfied with life’. Clearly this complicates any notion that parental support drives life satisfaction, so in the 2012 study report, the authors simply masked the disparity between 'somewhat supportive' and 'unsupportive', combining both to make a figure of 33% with 'parents who were not strongly supportive’ figure.
In fact, had the study authors had a different agenda, they might have presented the same data in a different combination to imply the opposite conclusion, that there was little or no association between parental support and life satisfaction. We know that about 20 (=84 x 34% x 71.5%) young people with very supportive parents were satisfied with life. About 4 (=84 x 25% x 18%) with somewhat supportive parents were satisfied. And 15 or 16 (=84 x 42% x 44%) with less or unsupportive parents were satisfied. If we combine figures for 'very' and 'somewhat' supportive we find 49% of the most 'supportive' groups reporting life satisfaction, vs 44% of the least supportive group.
In the same powerpoint, almost 70% of those with ‘somewhat supportive parents’ had made a ‘past-yr suicide attempt’, vs over 30% of those with ‘unsupportive parents’ - again, complicating the idea that a lack of parental support is associated with suicidality. In fact, the data can be combined in a different way to negate that parental 'supportiveness' impacted past year suicidality, as follows. 9% of 'very supportive' subjects (=84 x 34% x 9%) reported a past year suicide attempt, and 68 of the 'somewhat supportive' group did too, meaning that, when combined, 34% of the 'very supportive' and 'somewhat supportive' participants made a past year suicide attempt (= (84 x 25% x 68%) + (84 x 34% x 9%) / (84 x 59%)). This is the same figure, 34%, among the 'unsupportive' participants.
Further, it seems that the authors either mis-represented or misunderstood their own data, for in the powerpoint presentation, the figure for ‘past-yr suicide attempt’ is put at almost 10% (probably 9%, possibly 8% - the representation of the figures makes it impossible to be certain) but in the report it has shrunk to 4%. How reliable can this report be considered, as a basis for policy-making on a matter as vital to health as the drivers of suicidality?
Author: Anonymous, Australia
This is a non-peer reviewed publication describing the findings of a non-representative cross-sectional anonymous survey of 84 Canadian trans-youth (age 16-24) (Travers et al., 2012).
This study is being used in policy documents to support claims that parents who affirm their child’s gender identity is the strongest protector against suicidality and self-harm. I will discuss a few of the limitations of this study in some detail, as it illustrates many of the weaknesses of much of the research in this area.
First, a cross-sectional study design provides very low certainty evidence (NHMRC Level IV). It is not able to determine causal relationships. It cannot exclude reverse causality – for example, a young person with preexisting mental health conditions and suicidality, might be the cause of parents being non-affirming, rather than a consequence.
Second, another weakness of this survey is that it did not distinguish between “generally supportive” parents and “affirming” parents. By this I mean we can think of at least four potential groups of parents: a) generally supportive and affirming; b) generally supportive but not affirming; c) not generally supportive and not affirming; d) not generally supportive and affirming (In fact, there are more groups – one parent supportive and another not etc.) This survey only asked, “how supportive of gender identity/expression are your parents?”. The respondents could tick one of five options (four ranging from not at all to very supportive, and one not applicable).
The study concluded that parent support for gender identity/gender expression was directly related to decreased suicidality and improved mental health. However, there are other plausible possibilities behind this association. One, that it is general parent support that is protective, not specifically support for gender identity/gender expression. We can imagine for example a young homeless person, maybe who is escaping family violence, and who has no general support from parents would also likely list their parents as non-supportive of gender identity. Which is it – the lack of general support or the lack of support of gender identity - that is associated with self-reported lower mental health? Or in another example, a generally supportive parent may affirm and, also, ensure their child receives psychological support and possibly antidepressant medication. Which is it that leads to self-reported improved mental health – affirmation of gender identity or a generally supportive, and well financed, parent who makes sure a child receives appropriate psychological and psychiatric care? The study needed to ask more questions than it did to tease these types of issues out.
It may be that this is an example of “healthy adopter” bias – parents who follow physicians’ advice tending to have other features which are protective of their children’s health. It was this type of bias that is thought to have been one of the factors that distorted early SIDS studies’ findings which seemingly supported prone neonate sleeping, and likely contributed to the deaths of 10s of 1000s of the infants of parents who followed expert physicians’ advice (see discussion in: Clayton., A. (2023). Gender-Affirming Treatment of Gender Dysphoria in Youth: A Perfect Storm Environment for the Placebo Effect-The Implications for Research and Clinical Practice. Archives of sexual behavior, 52(2), 483–494. https://doi.org/10.1007/s10508-022-02472-8).
Third, a further problematic weakness of this study design is the lack of any multivariate regression analysis to exclude the role of confounders in its reported findings. For example: homelessness, bullying, total household income, sexuality, alcohol and drug abuse, ethnicity, education level, are all elements that may be associated with both parental support and mental health/suicidality outcomes. This study’s lack of accounting for these types of confounders means it is invalid to claim any causal association between parental support of gender identity and mental health/suicidality outcomes.