Human Resources and Skills Development Canada
Symbol of the Government of Canada

Human Resources and Skills Development Canada

www.hrsdc.gc.ca

Reading skills of young immigrants in Canada: the effects of duration of residency, home language exposure and schools - June 2008

6. School level variation using HLM

Schools play a crucial role in developing the skills of young people. Additionally for immigrants they are mechanisms for integration into the Canadian society. Reading skills in English and French are often developed in schools, and since many immigrants speak other languages at home, schools may be their only source of language training. Therefore, the choice of school and its characteristics could affect reading scores of not only immigrants, but also all other students.

The following analysis was aimed at investigating the school effects on reading skills of 15-year olds. A Hierarchical Linear Modeling (HLM) methodology was employed to study the reading score variation between schools with similar characteristics controlling for students' characteristics. The HLM methodology can estimate simultaneously the effects of student-level and school-level variables on an outcome measure (see Bryk and Raudenbush, 1992).

The students who participated in the PISA study came from 1,200 schools across Canada. However, since not all schools had students who were either immigrants or first generation, the sample of schools used in the estimation is lower. The following results measure the level of variation between schools simultaneously controlling for some key student level characteristics. Five separate models were estimated and the results are provided in Table 2.

Table 2 Results from HLM multilevel analyses

Model 1 is the null model which does not include any explanatory variables at either the student or school level and is designed to partition the total variance in the dependent variable, reading scores, into a between and a within-school component. Consistent with previous research using PISA data in Canada, the majority of variation is at the student level (82%) and only 18% of the variation in PISA reading scores is between schools (see Table 3).

Table 3 Analyses of variance of HLM models
    Null Model Model 2 (random effects) Model 3 (Level 1 model) Model 4 (province) Model 5 (province, SES)
Between school Intercept (Native) 1,603 (18%) 1,632 1,058 1,056 1,056
First generation difference   1,120 1,131 1,095 1,026
Immigrant difference   2,583 2,321 2,335 2,418
SES     219 219 218
BOYS     556 558 561
Within School Level 1 7,400 (82%) 7,131 6,123 6,123 6,124

Model 2 is designed to examine if the differences in reading scores between first generation and immigrants vary at the student as well as the school level. The dummy variables are allowed to randomly vary across schools, with variance all statistically significant. The results indicate that though the difference between first generation and Canadian born students is not significantly different from zero at the student level, however, there are significant variations across schools. Similarly, the immigrant differential of 20 points also statistically varies between schools. It should be noted that these are differences when no controls are introduced at either of the two levels of analysis. Overall, the results indicate that these differences randomly vary across schools and therefore can be specified as a function of independent variables. Model 2 is also run with a different set of dummy variables to understand the differences between the three groups. The results indicate that the average scores of first generation and immigrants tend to have a higher level of variation between schools compared to the scores of Canadian born.

Model 3 introduces key individual characteristics associated with reading at the student level, including family characteristics, socioeconomic status (SES) and gender, some of which are also treated as randomly varying at the school level. Even after controlling for student specific characteristics, the reading differences for the two groups remain the same and vary significantly across schools in Canada.

Model 4 attempts to model the average differences in first generation and immigrants at the school level by including the regional dummy variables in the school level equation. All the comparisons are relative to Ontario which is set as the default group. First, the difference in reading scores between immigrants and Canadian born (-20 points) does vary across schools but is not significantly different across regions in Canada. Second, the same is not true for the first generation group. Though the difference in reading scores between first generation and Canadian born is statistically insignificant in Ontario, but it does vary across regions in Canada. The results indicate that first generation students perform better than their Canadian born counterparts in British Columbia and Eastern Canada relative to Ontario. These results are statistically significant. A similar trend is apparent for Western Canada (Alberta, Saskatchewan and Manitoba) but it is not statistically significant at 10% level of significance (only at 13% level).

Model 5 extends the model further by including a control for the average socio economic status of the school in explaining the reading differences at the school level. As found in previous work done by Doug Willms (2003), average school SES is statistically significant in explaining variation in differences in reading scores between schools. In addition, the difference between first generation and Canadian born is 24 points and is statistically significant for Ontario. All the previous results hold including the difference for the Western Canada is now statistically significant.

To understand this further, the comprehensive model (model 5) was run with a different specification of dummy variables for the first generation and immigrant groups. Instead of modeling the differences, this model focused on estimating the intercepts for all three groups at the school level. Therefore, the variation explained by the model is still the same but now the interpretation of the reading scores of all three groups can be made separately.

The effect of average SES of school on reading scores of the three groups is large, and much higher for the first generation and immigrant groups (60 points versus 45 points for Canadian born).

All three intercepts (one for each Canadian born, first generation and immigrants) significantly vary between schools and differ significantly across provinces. Among Canadian born, the average reading scores is significantly higher in BC, Western Canada and Quebec relative to Ontario. For the first generation immigrants, the average scores are higher in all provinces/regions compared to Ontario. Students in the Atlantic region, perform significantly higher than first generation students in schools in Ontario. However, among immigrant students, region does not play a similar significant role. There are indications that immigrant students in Quebec do better than students in Ontario, but none of the other regional coefficients are statistically significant.

The overall results can also be discerned by examining the pattern of changes in the variance and its components as the intercepts are allowed to randomly vary between schools. The following discussion is based on the analyses of variance presented in Table 3. The total variation in the reading scores of 15 year old students in Canada is 9,000, and it is largely (82%) within schools compared to between schools. With the addition of variables at the student level that can account for some of the variation in reading scores at this level, it is expected that the within school variation will decrease. In model 2, the introduction of variables to control for the country of origin of students, the variation declines from 7,400 to 7,131. In model 3, with the introduction of SES, gender, family characteristics, the variation is further reduced to 6,123. This indicates that these variables account for significant variation at the student level. This does not change in model 4 or 5 as there is no further addition of explanatory variable at the student level.

Secondly, as is evident from Table 3, the variation between schools is largest for immigrant differences (2,583 in model 2). The introduction of province/region and average school SES does help to explain the variation in average reading scores between schools and to some extent the differences between first generation and Canadian born students. This is evidenced by the decline in variation between model 2 and model 5 for the reading scores and difference dummies. However, the model does not help explain differences for immigrants between schools, with not significant reduction noticed for the immigrant difference dummy.

Within schools, first generation students had reading scores similar to their native-born counterparts. The immigrant students' reading performance was almost 19 points below that of the reference group. The attempt to model between school differences did not yield many significant results. Only two regions were found to be statistically significant; Atlantic and British Columbia for the first generation students. On average within the school, first generation students attending schools in the Atlantic region and in British Columbia obtained reading scores 38 and 15 points higher than their peers from Ontario. No such differences were found for the immigrant students, indicating that regional factors do not affect the accumulation of reading skills of young immigrants within the school. In addition, even with the presence of control variables at the school level, the amount of variation in reading due to the choice of school was still much lower than the variation caused by individual characteristics of the students. For first generation immigrants, almost six times more variation was related to the students themselves as opposed to their school; for the immigrant born students it differed by a factor of almost three.

Finally, since average socio-economic status of the school could have an independent effect, it was introduced in model five. Within schools, immigrant born students as well as their first generation peers performed significantly lower than the reference group. At the school level, for both first-generation and immigrant born students, mean SES of the school was highly correlated with reading scores. The correlation was higher for immigrant born students than for first generation students (-40 vs. -23.6). In addition, for first generation students, their average school level reading performance in the regions of Atlantic, British Columbia and Quebec were higher than that of their Ontario peers attending schools with similar socio-economic characteristics (43, 19 and 18 score points, respectively). There were no significant regional differences in reading performance at the school level for immigrant born students. Individual level variations in reading scores were still higher than variations caused by schools.

Footer

Date Modified:
2008-06-11