An Estimation of Variances and Factors in Mathematics Attainment in Upper-Primary Schools

This study aimed at identifying variances related to school and pupil levels, and the performance indicators measuring the learning progress in mathematics. A longitudinal design was carried out and the stratified random sampling was used to obtain a sample of 5118 pupils from 60 primary schools. In each school three upper grades were selected, including their 180 class teachers and 60 headteachers. The data collection was carried out through questionnaires and school archives. Using multilevel analysis, the results showed that the proportion of total variance in pupil attainment for mathematics at the school level ranged from 20 to 29 per cent. When intake characteristics were taken into account as the baseline, the variance proportions were less than the prior attainments. Occupation and education of parents, besides reading facilities at home were significant explanatory variables for children progress in mathematics achievement. The outcomes have important consequences for school effectiveness research in determining the baseline and performance indicators.


Introduction
Schools exerted a more powerful influence on achievement than pupils' family background factors.It was the conclusion of most studies in developing countries up to mid-1980.This led to the inference that developing countries, schools were more effective than schools in well developed countries, both in raising achievement and in producing a route for social mobility [1,2].This conclusion had often been justified by the claim that in many developing countries, operating in communities [3] where written literacy and numeracy was a recent phenomena, and that as a result, schools with a few material inputs and only a moderate quality nonetheless might have a significance influence on academic attainment.
It was known from the survey that family background and school factors on pupils' achievement in developing and developed countries [1] as follows.The variation in school factors explained only small portions of variance in achievement in the developed countries.However, in developing countries the block of school factors explained significant portions of the variance in achievement.They also found a significant correlation between a nation's wealth and the amount of variance explained by the school factors and concluded that the lower the income of the country, the weaker the influence of pupils' social status on achievement.On this basis the conclusion was that a different paradigm of school effectiveness existed in the developing countries where schools had a greater impact on achievement than home background factors.
The Indonesian Primary Education Quality Improvement Project (PEQIP) published internationally [4,5,6] might be recognized as the first Indonesian research using multilevel approach.This study examined the PEQIP project in two provinces (from six provinces), Aceh and North Sulawesi.Each province included 72 to 78 schools with 9 clusters (each one nuclear school and six to nine satellite schools).The project was focused in teacher professional development, educational management (including community participation), books and learning materials, and evaluation and monitoring.Using two-level analysis they found that 46 per cent variance between schools for mathematics, 39 per cent for Indonesian Language, and 40 per cent for science in primary schools.In North Sulawesi where the PEQIP had a statistically significant influence, the pupil, classroom, and school variables were found for the three subjects as the following.By and large, PEQIP had positive effects on mathematics attainment and science where satellite schools made more progress than nuclear schools.The teacher professional development comprised innovative teaching and homework frequency were positively significant.The influence of evaluation and monitoring were not significant.
This current study search further by using multilevel techniques for primary schools in East Java, particularly in the City of Malang -Indonesia.In reality, the typical social data have multilevel characteristics [7].Due to only a handful study in developing countries using the multilevel techniques analysis, this was an effort to figure out the right reality by splitting the student and class levels then compared different models in explaining the educational attainment.The main research problem was that how much the school and pupil variances and what part taken into account among class/schools?

Methods
This study was designed as a longitudinal survey.The collection of pre-test data was carried out in the beginning of the school year.Then post-test at the end of the school year.While the background data on family and on teacher were obtained before the post-test.All the data related to children were nested into classroom and school for the sake of multilevel modeling.Data on teachers were parts of class level variables.
The sample in this study, was based on 5 subdistricts in the City of Malang (Blimbing, Kedungkandang, Klojen, Lowokwaru, and Sukun), the Province of East Java -Indonesia.The sample consisted of 5118 pupils, and 180 teachers from 60 primary schools.Three upper grades (4-6) were selected in each school.As a longitudinal survey, data were collected through questionnaires and archives in schools.Taking pupil prior year attainment as baseline, multilevel statistical techniques were used on assessment at the end of the school year to examine factors related in pupil, classroom and school levels which were related to pupil attainment a year later, described later in Table 1.
The response variable was the mathematics scores obtained by the pupils at the end of the school year.During this school year children had moved one grade higher i.e. grade 4. Tests at the end of grade 3, before moving to grade 4, were used as the baseline tests.The frequency distributions of the raw outcome measures were examined and tested the skewness and kurtosis were conducted as well.In order to facilitate comparison of the different model results it was considered desirable to transform the outcome measures to a standard format.After the response variables were transformed to normal scores and centering the baselines around the average to improve model fit [7,8], the strategy consisted of 4 possible consecutive models were planned.Model 0, known as the null model, disaggregated the variance components of the two levels.In this model, the mathematics scores were regressed on the constant term (coded 1 for every student).In addition, the constant term was set at random at both the student and school levels.The aim of this analysis was to estimate the overall mean achievement at both intake and at the end of school year and also to see whether there were any school differences in mean achievement.
Then the next was exploring the following models at the fixed part.A model reduction procedure of omitting predictors that did not contribute to the variation in achievement would refine the model into a 'minimal' model.Levels of significance under the 95 per cent confidence interval were obtained if the estimate was equal to or greater than double standard error [9].Model 1 was a simple value-added analysis of pupil progress over one school year after controlling for prior attainment only.Language and numeracy were closely related each other as shown in many studies,.Therefore in this study, both pupils' scores in Indonesian Language and in Mathematics were included as baseline measures of prior attainment.The quadratic terms of each baseline were also tested in order to allow for any non-linearity in the relationships with the response variable.
The contextualized model, Model 2, included measures of child background factors such as individual characteristics and family/home background without prior attainment.Model 3 was a more complex longitudinal value-added analysis which includes both significant background factors and the prior attainment measures.

Results
Pupil samples at Grade 4 for every school ranged in size between 14-41, at Grade 5 between 17-45, and at Grade 6 between 16-44 pupils.These conditions of range satisfied the requirements of multilevel modeling [9].After performing the null model, the modeling stages dealing with explanatory variables were consecutively entered: initial scores as baseline, child background and characteristics, class and school variables.Then the variances attributable to school and pupil were checked.The null (empty) models of the two-level analysis, as in Table 2, demonstrated that a significant proportion of the variance of each outcome was related to between class/school differences (as expected in all cases the majority of the variance was found to lie between pupils).From the total in the empty models, the proportion of class/school variances ranged between 20.4 to 29.2 per cent.The proportion of pupil variances ranged between 70.8 to 79.6 per cent.An interesting difference interesting result was that the higher the grade the larger the variance attributed to the class/school level.
Before entering the prior attainment into model 1, the correlations between pre-and post-test scores were checked.The correlations at grade 4, 5, and 6 were 0.54, 0.55, and 0.54 significantly.Model 1 revealed that the prior scores were non-linear predictors as shown by the significance of the squared variables.The results still existed up to the last model as in Table 3.
Results from stepwise procedures, discovered in Model 2 that pupil gender, father's occupation, mother's education, and textbooks at home were significant predictors of Mathematics attainment.Girls outperformed boys, children whose mothers were in clerical and professional work attained higher scores than those whose mothers were in skilled-manual or unskilled manual or non-earning categories.The higher the educational level of mother, the better the child achieved in Mathematics achievement.The greater the provision of textbooks in pupils' homes, the better the progress made in Mathematics.The prior scores were better predictors as well.In total variance for Model 1 was only 12.8 per cent the reduction.Of the remaining unexplained variance 84.2 per cent was attributed to the pupil and 15.8 per cent to the class/school level.After controlling for measures of the personal and family background of pupils and prior attainment, in the model 2, the range of proportion attributed to school and class factors were between 24.1 to 39.1 per cent.
A more complex value-added analysis appeared in Model 3 by including the initial scores and background factors provided a better fit.In all 35 per cent of the total variance was accounted for.The unexplained variance 75.9 per cent was attributed to the pupil level and 24.1 per cent to the school level.Apart from pupil gender, all the other findings differed in comparison with Indonesian Language attainment for the same cohort.Similar to the previous analysis, the coefficients of exploratory variables in Model 3 became smaller after prior attainment was controlled for.We might say that the prior attainment measures showed a stronger relationship with later Mathematics results.

Discussion
Data on pupil attainments over a one year period (using a pre-and post-test framework) were analyzed using multilevel statistical techniques as discussed early.These provide the most appropriate basis for the analysis of school effects on mathematics, a cognitive outcome in basic skills.Although the use of basic skills as a measure of school performance had been criticized, they can be defended for two reasons [10,11].Firstly, the historical view of effectiveness concerns the improvement of the basic skills of disadvantaged children.Secondly, basic skills provide the foundation for further learning and therefore can be seen as lying at the core of subsequent educational outcomes and progress.
From the empty model, the proportion of class/school variance was 20.4% for mathematics.The findings were in line with those found by other researchers working in primary schools in developed countries in terms of the effect size of the subject.Researchers using multilevel analysis in developing countries have found that the school variances tend to be considerably higher although many of the studies suffered from limitations in designing [12].For this Indonesian study the smaller variance results were likely to reflect the particular nature of the sample in which only urban state schools were included.In interpreting the results this particular context must be remembered and it was likely that if schools in other areas of Indonesia were included in the study the level 2 (described as a combination of class/school effects which could not be separated as only one class per grade were sampled) variance estimates would be expected to be greater.
After controlling for measures of the personal and family background of pupils and prior attainment, shown in Model 4, the proportion attributed to school and class factors was 24.1%.The proportion appeared to be different from those in developed countries i.e. 12-18% [13].Few exceptions were, for examples, a study England [14] where in the lower grades of British primary schools, 40% was attributed to the school level variation and a study in Scotland for Mathematics controlling for prior attainment 32% in the primary study [15].
It should be noted in interpreting this study's findings that the percentages had a role in setting the boundaries for the potential impact of factors in different levels only [3,16].To discover more about the explanatory factors in each level, further analyses were needed.
The relationship between outcome measures and pupils' prior attainment was assessed in order to examine the question of linearity.Non-linear relationships were found between grade 4 mathematics' and Indonesian language's prior with post scores.By controlling for these non-linear relationships, more appropriate results were achieved in terms of model fit [7,9].After discovering the typical relationships, the prior scores were treated as baselines for examining the impact of other significant predictors.
The child background measures i.e. gender, parental occupation and education, and books at home were found to be statistically significant predictors for mathematics.Girls were performing better than boys.Children whose fathers had skilled manual and unskilled manual jobs had poorer achievement than those with fathers in clerical and professional works.Mother's education was the stronger predictor for child's attainment.The availability of textbooks at home affected the mathematics achievement as well.These factors were confirmed in the previous studies [17][18][19].

Conclusions
The variances of children attainment at upper grades in primary schools were much higher at the individual level than the class level.Even low, the classes variances expressed how important the process of the teaching.The findings had important consequences for school effectiveness research in determining the baseline and performance indicators.Of course, the implication of this study was on the policy for educational quality improvement.The educational quality could be misleading if someone ignoring the possible variances in terms of controlling the predictors and the hierarchical aspects of schools.The present study was designed to assess a set of performance indicators for schools by distinguishing the characteristics of levels inside the schools that might be of use to subsequent educational policy development.

Table 1 .
Description of Pupil and School Variables

Table 2 .
Proportions of Unexplained Variance at Pupil and School Level

Table 3 .
Regression estimates and standard errors of explanatory variables per grade (the significant ones from model 3)