Critical Appraisal session with children’s services, September 10, 2015

I recently delivered a critical appraisal session at one of the trust’s specialist children’s services site. The participants for this session were all involved in children’s services. There were several physiotherapists, a physiotherapy technical advisor, an occupational therapist, a speech and language therapist, and a school nurse. The paper we appraised was:
Mitchell, Darcy B., and Hauser-Cram, Penny. Early predictors of behaviour problems: two years after early intervention. Journal of early intervention, vol. 32 no. 1, December 2009, pp. 3-16.

Link to full text

This was a cohort study. We appraised it with the help of the CASP cohort studies checklist.

There was some discussion as to whether the study was focussed enough. Participants pointed out that the children involved had a range of diagnoses. While the study stated its research questions clearly, there was some debate as to whether it was maybe “trying to do too much”.

When discussing cohort recruitment, we found that we would have had to refer back to recruitment criteria for the EICS, a longitudinal investigation of 190 children with disabilities. The cohort was drawn from children who were enrolled in this study. Questions of the cohort’s representativity were discussed – we wondered if they represented the sort of population that would generally access specialist services in the states where the cohort was recruited. A big drawback of the study was that the demographics of the children were not presented in an easy-to-digest table. We eventually agreed that as far as we could tell this was an acceptable way of recruiting the cohort, though probably not the best.

Regarding the measures for exposures and outcomes, we found that most participants required some clarification to identify what was an exposure and what was an outcome, because again, the paper wasn’t terribly clear. We found that the researchers used lots of validated scales to measure exposures and outcomes, which as far as we could tell was okay. A point of discussion was the administering of the scales. The paper only mentions mothers as sources of parental input on scales and questionnaires, so we wondered if this was a source of bias or something that left the picture incomplete, especially as quite a high proportion of the parents were married and presumably raising the child together. We also wondered what the child behaviour checklist looked like and which kinds of problem behaviour it covered. We also wondered which kinds of behaviour would have been classified as externalising or internalising, as the gender divide between them that was found in the study didn’t seem to reflect participants’ experience.

We thought that there were a number of confounding factors that the authors hadn’t taken into account. As one participant put it, a lot can happen in two years. We thought that it was good that they measured stressful life events and family climate, but thought that a number of other things could also influence children’s outcomes, such as: parental physical and mental health, the way parents cope with stressful life events, the stress of caring for a disabled child in and of itself and how parents cope with that, severity of disability, how much they used specialist children’s services. The point was made several times that there is possibly a big proportion of the story here that the numbers cannot tell. Confounding factors implicit in the design were, as mentioned, the presence of potential bias caused by only taking into account mothers’ reports and the fact that there might be families with children of this category who don’t access specialist services.

We thought that follow-up was good and complete enough to answer the question. We thought it was long enough to answer the question, but there was some debate over how good that question is. Two years are not very long in the life of a disabled child. Also, the studied period of the children’s lives included their transition into kindergarten/pre-school, but not their transition into school, which participants agreed would have been more interesting to know about.

The results, to us, didn’t seem particularly surprising – indeed, several people thought the paper was “stating the obvious”. The bottom line results were that, out of the factors that were studied, two seemed especially important as predictors of behavioural problems at age 5: the number of stressful life events, and the family climate. Children’s gender or the specific disability group they belonged to did not appear to influence behavioural outcomes.

Participants believed the results as they fit in with their general experience. The local population seen at the site is quite different from the one in the study. The study was concerned mainly with white children of whom 55% were boys. The disabilities found in the study population were also quite different: equal numbers of children with Down’s, motor impairment, and developmental disabilities of unknown etiology. Our local population has a different mix of ethnic origins, includes disproportionately more boys than girls, and a large percentage of children with autistic spectrum disorder (a group virtually non-existent in the study).

Nevertheless participants mentioned that there were some things they could take away from the study. It was mentioned that the results would be helpful to then build on – for example, do a study with the families to find out what would help them deal better with stressful events. Participants also agreed that in their clinical experience, it is always important to take the family’s situation into account, but that they “can’t do everything” and would benefit from a presentation from workers involved in areas like housing.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s