Quality and equality: Verifying “My reading experience” questionnaire for a sustainable literacy education framework

This paper reports one aspect of a larger project that set out to narrow the literacy gap among Malaysia’s rural and urban children in terms of their literacy achievement. Using Perak as a case-state, this overall project scrutinises why despite the Education Ministry being the biggest recipient of the recent national budget 2020, with an estimated allocation of RM64.1billion, there are still children especially in rural schools who are unable to master the ability to read and write, even in their own mother tongue language. Through this project’s on-going work which attempts to connect the theories of literacy with actual-on-the-ground issues of children’s reading experience especially in rural schools, important matters were flagged up. This paper will highlight these matters as they are uncovered vis-à-vis the verification of “My reading experience” questionnaire which was one of the main research tools that was used for this project. Mainly, these matters were located along three aspects of literacy i.e. context, definition and language as they relate to how the questionnaire was designed. This has important implications towards how a sustainable literacy education framework can be shaped.


Introduction
UNESCO's 17 Sustainable Development Goals (SDGs) is considered to be a roadmap for the global community to right some wrongs. In the last two to three decades of the 20th century, it became obvious that developed and developing nations were practising a consumeristic lifestyle that had begun to negatively impact earth and its community. The lifestyle was considered to be unsustainable and for that, world citizens were urged to return to basics.
Among the 17 SDGs, the 4th SDG focuses on Quality Education. This goal is a continuous global aim at narrowing the gap between the haves and have-nots, having been brought forward from the Education For All (EFA) 1990 agreement in Jomtien and the succeeding efforts in Malaysia and the rest of the developing world. In this time, the narrowing of social gap is aimed at bringing about sustainable transformation through quality living and equality sharing of opportunities across world populations.
In this light however, the rural-urban, socioeconomic and now, digital divide has resulted in the prediction of unequal life chances for children across the world. Although some measure of success was achieved in the final few decades of the 20 th century with respect to literacy and education gap especially for marginalised girls, the fact that the gap remains is in itself, worrying ( (Duncan & Murnane, 2015;Ministry of Education Malaysia, 2012). Therefore, we argue that in any education system of high quality, basic literacy must be considered the pre-requisite to any other cognitive and intellectual developmental process. Yet, this problem of unequal literacy attainment receives far less attention than concerns regarding the digital divide. The heavy emphasis on computer and computer-based infrastructure in schools is gravely misplaced because any effort at addressing its problems will be futile if the problem of marginalised children's basic literacy is not tackled. Therefore, it is critical that the nation's poorest and marginalised groups are empowered through overcoming low levels of literacy and reading practice. We argue that once we can understand the experience from seeing it through the children's perspectives, we may be better positioned to appropriate the vehicles of technology.

Problem statement
Yet, in the effort to arrive at the real situation in schools across the country, we found that there is a lack of a comprehensively designed survey tool which could gauge the reading experience and preferences of primary school children in the national and vernacular schools. Our search for literature regarding this continually showed up dated information that was no longer relevant (Abidin, Pour-Mohammadi, & Jesmin, 2011;Atan Long, 1984;Pandian, 2000;Small, 1996) or questionnaires that could not accommodate the various languages being used in Malaysian schools, both the vernacular and national. The complexities of the different languages will have important bearings to how the questionnaire will be represented.

Social inequity in reading practice and attainment
In Malaysia, this social inequity is clearly present. However of late, there is a tendency to address the social inequity as if it were exclusively a problem of digital equipment and broadband connectivity (Hong & Koh, 2002;Mohamed, Judi, Nor, & Yusof, 2012). As such, a major portion of the national budget is dedicated to equipping rural schools with the latest in technology (Ministry of Finance Malaysia, 2016, 2018 and the setting up of infrastructure for classrooms seems to be a yardstick to measure educational progress (Borneo Post Online, 2018). Yet, language educators continue to highlight that literacy in both English and Malay language is still underachieved for the bottom quartile of school students (Ong, Roselan, Anwardeen, & Mohd Mustapa, 2015;Yamaguchi & Deterding, 2016). Thus, current initiatives to narrow the digital divide will fail if fundamental problems associated with literacy and reading are not addressed.
The main implication of this initiative is how children who enter primary schools are assumed to be school-ready, particularly in terms of their emergent literacy abilities. Yet, underprivileged children in Malaysian rural schools continuously lag behind in literacy achievements (Kaur, 2017;Ministry of Education Malaysia, 2008, 2012, 2015. In a research which set out to understand literacy in terms of materiality, Chong and Renganathan (2016) found that approximately half of a class of Primary 2 children in a rural school still struggled with basic literacy even in the language that is native to them. Like many national schools up and down the country, the issue of literacy in this particular school was largely tackled from the academic perspective. For example, LINUS 2.0 was, at that time, counted on as a means to address the problem of low levels of literacy. In these situations, remedial help is supposed to be given to those who struggle with decoding. Chong and Renganathan (2016) also noted another important finding in their study. That finding was related to how the children's sociocultural spaces were less explored and virtually untapped. Not only that, it seemed that the children's underprivileged backgrounds were a disadvantage that had to be accepted. As a result, these children were expected to learn to be literate despite their difficult backgrounds. Yet, it has become critical to understand the socioeconomic and sociocultural contexts of underprivileged, rural-school children who are at the cusp of learning literacy skills so as to help them optimize their contexts for them to have a fighting chance at doing school.

Theories of language and new literacies
In the area of language learning, the research focus is necessarily pedagogical. Therefore, central to research in this area is the 'technical', classroom aspects of language teaching, learning, motivation, performance and testing (Dornyei & Ushioda, 2009;Evans, 2013). In this line of thinking, teachers are assumed to be paramount in imparting knowledge to students who are sometimes regarded as 'not knowledgeable' or even 'deficient'. As such, the responsibility of language learning lies squarely on the teaching and to a lesser extent, the larger sociocultural environment which is inclusive of family background and community-based factors.
In New Literacies, Kalantzis, Cope, Chan, and Dalley-Trim (2016) demonstrate that there is value in broadening the notion of literacy as being beyond the written word and beyond linguistic parameters. "We believe that we need to recalibrate our approaches to literacy teaching to align with contemporary conditions for meaning-making …in the wide range of social and cultural contexts in our daily lives" (Kalantzis et al., 2016, p. 73 This has implications to the way we design our data collection tool because more often than not, the considerations of sociocultural factors fail to be represented in the tool especially when the quantitative paradigm underpins the tool.

Methodology and methods
Overall, the mixed methodological design was adopted for this study in that the idea of the phenomenon of being literate was assumed to be somewhat measurable (i.e. via the assessment component of the questionnaire) whilst also giving liberty to cater for unexpected interpretations from the children and teachers. In Phase 1, quantitatively-informed paradigm underpinned the research design. The method of survey was used to collect broad, generalisable data. In Phase 2, qualitatively-informed paradigm underpinned the research design. The methods of in-depth interviews of participants, as well as site and field observations were used. In this paper, the method of questionnaire utilised in Phase 1 will be discussed. Particularly, the verification of the questionnaire design will be examined.
In terms of the sampling group, Perak serves as a suitable case-state because out of a total of 1097 registered primary and secondary schools in Perak state, approximately 78% or 845 schools are primary schools while only 22% or 252 are secondary schools (Jabatan Pendidikan Negeri Perak, 2015). Out of 845 primary schools in Perak, 75% or 636 schools are defined as rural schools while 25% or 209 schools are urban (Ministry of Education Malaysia, 2017). In contrast with some states (e.g. Penang and Selangor where the ruralurban divide is very small), ¾ of Perak's schools are most likely to be disadvantaged. For this reason, Perak can become a useful case-state from which the findings can serve to address the rural-urban gap.

Considerations for questionnaire design
The researchers designed a survey questionnaire that captured information regarding the students' background and experience of reading. This paper will report findings drawn from an analysis of how the survey tool i.e. the questionnaire was developed, particularly in how the questionnaire's validity was considered. The following sections delineate the three major considerations for how the questionnaire was designed. It is important to note that we were guided by the theoretical constructs of literacy as a social practice in our deliberation of the questionnaire design.

Validity of questionnaire
The measure of validity in a questionnaire is understood to be the extent to which a measurement actually measures what it aims to measure (Creswell, 2014). For example, if a questionnaire sets out to measure the construct of reading experience, the experts who design the questionnaire must be able to operationalise this construct via suitably-worded items in the questionnaire, carefully constructed layout and settings of the questionnaire. Particularly, the experts must be able to translate theoretical constructs into operational statements fit for the respondents (Bolarinwa, 2015). Thus, in the following sections, we discuss three considerations that we as the experienced designers of the questionnaire undertook to balance its validity, namely, the context of literacy, the definition of literacy and the language of literacy.

Context of literacy
Before we venture further into a discussion about our idea of the context of literacy, it is important to begin by clarifying the credibility of the researchers. In this project, we as the main researchers bring to bear our vast experience and expertise in examining matters pertaining to literacy and reading. Our collective experience is drawn from our combined years of international academic training (PhD theses) in literacy and education, awards in the form of national and international grant projects and publications in a wide array of journals, newspapers, books and book chapters (Chong, 2018(Chong, , 2019Janan & Wray, 2014;Lim, 2018aLim, , 2018bLim, , 2019Renganathan & Kral, 2018 ;Wray & Janan, 2013).
Based on our experience drawn from international understandings of literacy, we took into consideration the different contexts within which literacy questionnaires in other parts of the world have been designed. Particularly, our decision was guided by our previous research efforts in identifying well-designed questionnaires that focus on reading. From here, we narrowed down our selection to four sources for the following reasons. First, Malaysia remains the most important source as the context is immediately relevant to our study. Within this context, the national survey carried out by the National Library with published and publicly available findings is still considered to be the most comprehensive (Small, 1996). Second, a recent survey report representing Singapore's teenagers' reading habits was considered to be current and potentially useful (Loh & Sun, 2018). Third, United Kingdom's National Literacy Trust is known to have a long-standing record in reporting literacy practices of UK's students (C. Clark & Foster, 2005; C. C. Clark & Rumbold, 2006). Finally, a team of researchers in the United States also have a long-standing history of designing and verifying the Motivation for Reading Questionnaire (Wigfield, Guthrie, Tonks, & Perencevich, 2004). When comparing and contrasting these sources, we found some portions of each of them to be compatible while other aspects to be incompatible. Below, in Table 1, is a summary of each of the questionnaire's compatibility with our study's aim and the action we took to incorporate into our questionnaire. The stable questionnaire items on importance of reading and reader's attitude towards reading were considered From the considerations, the constructs that make up the Malaysian child's reading experience were decided to be the following: their self-reported reading ability, the importance of reading, their parents'/family support on their reading and their school/public support for reading and their access to multi-media (e.g. mobile phone, computer).

Definition of literacy
As previously mentioned, our project was conceived based on the theoretical understanding that reading is more than decoding a language. Thus, we adopted the sociocultural perspective for literacy practice (Gee, 2008;The New London Group, 2000). However, we were also cognizant that the term 'literacy' is not widely used in the Malaysian context. Reading is still understood by the masses, as making sense of linguistic systems. For this reason, we retained the word 'reading' and used that as the base for its translation into Malay, Mandarin and Tamil.

Language of literacy
In terms of the language of literacy, we acknowledged the multilingual nature of the student respondents in this research. This meant that the language of literacy both of the questionnaire and of other related reading materials were critically understood to be represented via multiple linguistic systems. This brought up the realisation that although we had to consider how some of the students could be struggling with reading and writing in a fundamental way, there could also be students who were highly-literate in one language but poorly-literate in another. This thinking was underscored by our application of the theories of literacy as a social practice because we acknowledged that these students brought different social and cultural practice from home to school (Heath, 2012;Street, 1995).
The base questionnaire was initially designed in English. This is because the original sources of the other questionnaires were in English. However, a fair amount of effort was spent in translating and testing the questionnaire items in Malay, Mandarin and Tamil. Questionnaires in Mandarin and Tamil were also provided with their equivalent Malay translations. Besides adhering to Ministry of Education's requirement of using bilingual sentences where the original were not in Malay, we were also interested to accommodate students who may be differently proficient in two languages. This would provide a sense of equality for the different backgrounds Malaysian students come from.
Related to language is also the consideration of appropriate use of symbols. This affected the questionnaire in two ways. First, we considered respondents who may struggle with reading long sentences. Thus in reducing them, we used emojis (the scale of four faces denoting 'very happy' to 'very sad') to represent the likert scale so that the respondents could access meaning through non-alphabetic/non-linguistic means. Second, where Romanised language was used for Malay questionnaires, the UD Digi Kyokasho NK-R font was used. This was to ensure that the letter 'a' that was used in the questionnaire was consistent with the letter 'a' that was learnt by the students via handwritten form. The letter 'a' can only be found in UD Digi Kyokasho NK-R font.

Closing remarks
The way in which our questionnaire was verified points to how our research project contributes to quality education through accounting for the challenges our respondents may face in their participation of our research. Indeed, the fundamental consideration of any research must be located in the way the respondents are anticipated to lend their perspective.