Comparing multiple means from a dependent variable against multiple means from an independent variable -- Predict mean scores based on mean scores?

Question

I have about 300 students who replied to a psychological test formed of several resilience and vulnerability issues. A typical Likert scale, with "mean scores" as the result. From these students, about 10 have ADHD and about 30 have "Other medical health disorder—OMD.". Their school performance was also classified as "below", "average", and "above" by their teachers.

I want to know which psychiatric profile (ADHD vs. OMD) is more related to the results students obtained in accordance with their school performance.

.

This helps in the following practical aspect: If I have a student with ADHD or if I have a student with OMD, I can predict what their school performance tends to be.

Check this table out. The mean score of students with ADHD was 33. OMD was 39.9. Students with school performance "below" scored 30.5 Students with school performance "average" scored 32.4 Students with school performance "below" scored 29.53

So, which is the most adequate statistical procedure to deal with this ?

Below there is a snippet of this dataframe using R.

library(tidyverse)
library(arsenal)

df_2 = structure(list(adhd = 
    structure(c(1L, 1L, 1L, 1L, 1L,   1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
     1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
     1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
     1L, 1L, 1L, 1L, 1L), 
     levels = c("0", "1"), class = "factor"), 
     omd = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
     1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 
     1L, 1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 
     1L, 1L, 1L, 1L, 1L, 1L), levels = c("0", "1"), 
     class = "factor"), 
     school_performance = structure(c(1L, 2L, 1L, 3L, 3L, 3L, 2L, 2L, 
     2L, 2L, 2L, 1L, 3L, 2L, 2L, 2L, 1L, 1L, 1L, 2L, 2L, 3L, 2L, 3L, 
     1L, 2L, 3L, 1L, 2L, 2L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 
     1L, 3L, 3L, 1L, 3L, 3L, 3L, 3L, 2L, 2L), 
      levels = c("1", "2", "3"), class = "factor"), 
     reactivity_total = c(33L, 23L, 10L, 21L, 8L, 14L, 29L, 25L, 54L, 
     32L, 31L, 19L, 42L, 26L, 50L, 10L, 37L, 44L, 62L, 8L, 23L, 39L, 
     21L, 20L, 38L, 10L, 17L, 22L, 39L, 14L, 11L, 80L, 54L, 8L, 20L, 
     36L, 30L, 19L, 27L, 29L, 37L, 22L, 15L, 46L, 19L, 26L, 40L, 29L, 
     25L, 30L)), row.names = c(NA, -50L), class = "data.frame", 
     na.action = structure(c(`2` = 2L, `5` = 5L, `20` = 20L, 
      `21` = 21L, `30` = 30L, `185` = 185L, `282` = 282L, 
      `332` = 332L), class = "omit"))   

tableby(school_performance ~ 
          reactivity_total,
        df_2) %>% 
  summary(text = T, digits=2) %>%
  as.data.frame()
#>                         1 (N=15)      2 (N=17)      3 (N=18)  Total (N=50)
#> 1 reactivity_total                                                        
#> 2     -  Mean (SD) 30.73 (14.55) 26.47 (12.84) 28.50 (17.57) 28.48 (14.98)
#> 3         -  Range  8.00 - 62.00  8.00 - 54.00  8.00 - 80.00  8.00 - 80.00
#>   p value
#> 1   0.732
#> 2        
#> 3

tableby(list(adhd, omd) ~ 
          reactivity_total,
        df_2) %>% 
  summary(text = T, digits=2) %>%
  as.data.frame()
#> Warning in as.data.frame.summary.tableby(.): as.data.frame.summary.tableby is
#> returning a list of data.frames
#> $adhd
#>                         0 (N=48)       1 (N=2)  Total (N=50) p value
#> 1 reactivity_total                                             0.092
#> 2     -  Mean (SD) 27.75 (14.47) 46.00 (22.63) 28.48 (14.98)        
#> 3         -  Range  8.00 - 80.00 30.00 - 62.00  8.00 - 80.00        
#> 
#> $omd
#>                         0 (N=48)       1 (N=2)  Total (N=50) p value
#> 1 reactivity_total                                             0.668
#> 2     -  Mean (SD) 28.29 (15.03) 33.00 (18.38) 28.48 (14.98)        
#> 3         -  Range  8.00 - 80.00 20.00 - 46.00  8.00 - 80.00

^{Created on 2024-12-21 with reprex v2.1.0}

Shawn Hemelstrand · Accepted Answer · 2024-12-22 01:53:14Z

I had a bit of a longer response to this question until I read this:

Their school performance was also classified as "below", "average", and "above" by their teachers.

This seems like a very flimsy way of assessing their school performance. Wouldn't it be more direct to simply use their GPA or something analogous? What you propose here invites a lot of subjectivity, and if your goal is prediction, then this is even more problematic. Will you have the same teachers next year? Will this rating system replicate in another cohort? Have you checked the inter-rater reliability to even see if their scoring is consistent? These would definitely be necessary if you are going to use subjective and indirect ways to quantify performance.

As to what you should use for such an analysis...if your goal is prediction, I would consider a lot more than just the mental health classifications. I'm not a clinical psychologist, but I do know that 1) mental health disorders can be highly comorbid (e.g. anxiety and depression) and may have some compounding effects that should be considered 2) age probably has a determinant factor on both grades and mental health status 3) previous scores on standardized tests or some other academic predictor may be very helpful for prediction. My assumption is that you would use these combined data to predict with more precision than simply one categorical predictor which is a more loose amalgamation of disorders (OMD seems very hand wavey to me).

Regardless, you can just fit this to a regression, but this is again why the "performance" variable matters. If you keep performance as the DV with this sort of ordinal relationship, you may need to consider fitting this to an ordinal logistic regression. If you had something more continuous (e.g. GPA or something like it), then a simple regression with mental health status (and any other predictors) would suffice.

Notice I kept this answer firmly on prediction. If you just want to know differences between groups in just this class, then you would just compare their descriptive statistics and move on. But I assume this question was posed for larger pursuits.

Ty for your comments. I agree with you. However, the data I'm working on comes from Brazil and it's very common that every school teacher uses their subjective perception and classifies students school performance as "below", "average" or "above". I imagine this is not limited there, as teachers have their perception about students. My question is: given that everyone took the same test, some of them had ADHD or other problems, and that I know students were classified beforehand by their teachers, how can I use the scores they had in the test to predict what their classification will be? — Luis
– Luis, Commented Dec 22, 2024 at 20:51
Like I mentioned before, you could use ordinal logistic regression since the response is essentially an ordered category, using psychiatric profile as the predictor and performance as the outcome. I would still check the inter-rater reliability to see if their ratings are consistent and if that invites measurement error into the model. It also wouldn't hurt to get the inter-item reliability too since you mention Likert scales. — Shawn Hemelstrand
– Shawn Hemelstrand, Commented Dec 23, 2024 at 0:59

Stack Exchange Network

Comparing multiple means from a dependent variable against multiple means from an independent variable -- Predict mean scores based on mean scores?

1 Answer 1

Your Answer

Hot Network Questions

Comparing multiple means from a dependent variable against multiple means from an independent variable -- Predict mean scores based on mean scores?

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Related

Hot Network Questions