Assessing elementary school pupils’ narrative skills. A longitudinal study of the impact of current Greek language teaching practices

The aim of the present paper is to discuss the findings of a research study focusing on the narrative writing skills of Greek elementary school 6th grade pupils, comparing the pupils’ performance before the current language teaching material was introduced, and both one and six years after its implementation. An effort was also made to explore whether the pupils’ skills are influenced by gender, social background and teachers’ practices regarding written discourse production. The research participants come from 10 public elementary schools. The findings of the study indicate that, already after the first year, the pupils’ narrative skills display a remarkable improvement, which is even further increased after six years. It also suggests that their skills relate to their gender, social background, and current teachers’ practices. Despite a general enhancement, the pupils of the sample overall display a mediocre performance.


Introduction
The theoretical framework of the present study relates to the concept of literacy, a social practice that is constantly redefined in terms of the socio-cultural environment in which it takes place. Over the past thirty years, the debate on the concept of literacy created the framework and principles of literacy pedagogy (see Sydney school, i.e. [1,2]). According to genre based literacy pedagogy, the main unit of literacy is the genre, as shaped by the respective socio-cultural reality [3,4]. School literacy [2] is mainly achieved through the elaboration and production of -mostly written -genres considered important for defining and transmitting knowledge in various fields, aiming to develop critical language awareness [5]. It is recommended to conduct communicational and interactive activities (between students and their peers or their teachers), which include practices that help students realise the characteristics of each genre, and to allocate time for the production, reviewing and editing of written discourse by the students themselves. Such educational programmes have already been successfully implemented worldwide over the last thirty years [6,7].
In Greece, this debate led to the 2003 National Curriculum for the Greek language in the Elementary School [8] and the 2006 school textbooks and teacher manuals for the language arts class [9,10]. This researcher served as a member of the board of designers of the 2003 National Curriculum for Language a e-mail: afterniati@upatras.gr

SHS Web of Conferences
Arts in the Greek Primary School and as a member of the board of editors of the specific teaching material (textbooks and manuals) for elementary education Language Arts, which were composed (2004-06) by groups of expert authors (academics, researchers and practitioners, not including the editors of the board), after a competition.
Prior to the implementation of this material, several Greek studies had been conducted [11][12][13][14][15][16] on the previous teaching material and, more generally, on the quality of language teaching in primary education and on important points revealed by the pupils' written text production. These studies focused both on the common problems faced by children of different social background when producing written texts and on socially determined differences in language use.
The results were disappointing. They demonstrated that the majority of students displayed mediocre or low written discourse performance, while their success depended mostly on their social background. The above results were attributed to the nature of the teaching material, which was based on the structural approach in language teaching [17] and relevant teaching practices, as well as to a lack of adequate educator training.
These specific results were also attributed, according to previous research on the quality of written discourse teaching in Greece [16,18], to educators' dominant views and practices on the teaching of written discourse. It is claimed that before the 2003 curriculum and the 2006 teaching material were implemented, written discourse production was ignored at school as a dynamic cognitive process. This refers to a process that includes the elaboration of various versions of a text before the final product, under interactive circumstances, and taking into account both the communicational objective and the intended recipient/audience. According to the above research, written discourse practices were limited to 15-minute written text production exercises, with neither previous planning nor later elaboration, and without any self-or peer-evaluation, which were often assigned as homework.
As a result of the above, a new curriculum and teaching material was introduced to elementary school language arts. It seems to promote communicational and interactive activities, and the allocation of sufficient time to the production and editing of written discourse by the pupils themselves and their peers at school [19,20].
The research conducted so far in order to evaluate the above, in terms of meeting the stated objectives, focus on the teaching material [21][22][23]. They address whether the textbooks implement the main teaching practices adopted worldwide. They display divided views on the degree to which the textbooks' theoretical and practical choices implement the contemporary principles on the teaching of modern languages.
For this reason, it was considered necessary to conduct research not focusing on the teaching material, but evaluating the results of the implementation of the specific teaching material and respective teaching practices (provided they were implemented by the educators), in terms of the pupils' literacy skills, as revealed by the pupils' written production. The results of such a study could demonstrate whether the current Greek elementary school language arts practices are effective, in relation to the improvement of the pupils' literacy skills.

Research objective and design
The aim of the above mentioned study b is to examine and discuss the findings of a research study focusing on the writing skills of Greek elementary school 6 th grade pupils producing narrative, b Many thanks to Professor Yannis Katsillis, of the Department of Elementary Education, University of Patras, for his valuable help in designing the methodology of this research.

IAIMTE 2013
descriptive and argumentative text before the current material was introduced in 2006-07, one year after its implementation, and six years after its implementation.
The research lasted six years (2006)(2007)(2008)(2009)(2010)(2011)(2012), so that the pupils who attended the first grade in 2006, when the current books were first introduced, would complete their primary education (6 grades) with this teaching material. This is a presentation of the comparison of the 1 st and 6 th year data of the pupils' narrative skills c . The study investigates the differences on the level of the pupils' narrative skills before (pre-test) and after the first year (post-test) and six years after the implementation of the current teaching material, at the same schools, which operate in areas of different social background.
An effort was also made to explore whether the pupils' skills are influenced by parameters such as gender, social background, and teachers' practices regarding written discourse production.

Sample
The research implemented stratified random sampling techniques and took place in three phases (two the 1 st year -pre-and post-test respectively-and one more phase six years later) at the sixth grade of ten public elementary schools. The schools operated in urban, suburban, semi-urban and mountain rural areas, and were chosen so as to include pupils displaying widely differentiated social backgrounds. The population of the study consisted of 151 pupils (78 boys and 73 girls) for the first year, and 142 pupils (69 boys and 73 girls) for the sixth year.

Tools, data collection and procedure
The study used 2 questionnaires and the IEA test [24], revised and adapted to the Greek language and educational context.
In order to investigate the pupils' narrative skills, the research utilised the test only in terms of the criteria referring to the main organisational characteristics of the narrative genre: 1. the narrative pattern [25][26][27], that includes three of the main elements of the narrative genre: orientation, complicating action and coda / resolution 2. the evaluation of the narration 3. the cohesion [28] 4. the coherence [28] 5. the texts' grammaticality and semantic acceptability.
The above was analysed with twenty criteria and each criterion was evaluated using a five-point scale. The total score of the five categories, in case of the highest performance, would be 100. For more details on the theoretical framework and for examples on the method of the pupils' text analysis and writing tasks, see the study of Gorman et al. [29].
Apart from the analytic scoring, the texts were also holistically scored. Both a six-point and a tenpoint scale were used, so as to ensure greater objectivity [30].
Let us now analyse the criteria for each of the 5 categories: the 1 st category includes 5 criteria, the 2 nd category 2 criteria, the 3 rd category 6 criteria, the 4 th category 2 criteria and the 5 th category 5 criteria, for a total of 20 criteria.
c The findings about the pupils' descriptive and argumentative skills have already been announced at the International Conference on Greek Linguistics, held in Rhodes, 26-29 September 2013, with a special focus on Language and Education, and the announcement is to be included in the conference proceedings, when published.

SHS Web of Conferences
Regarding the 1 st category, narration of a complete episode, the study investigated whether the texts developed (5 criteria): a) orientation successfully: whether they provide sufficient information on the characters/heroes, the place, the time frame, and the initial situation in general, so that the reader can be orientated and successfully introduced to the narration. b) complicating action: first whether they: Include the event that upset the initial state, sufficient plot/action development (internal and external action), the climax and the end of the episode. c) complicating action: second whether they: Develop the heroes' characters according to the facts. d) complicating action: third whether they: Follow a clear sequence of events. e) a successful coda/resolution, that is a conclusive statement that provides a sense of closure.
The texts were then tested for an evaluation (2 nd category) of the narration by the pupil, that is whether the text presents the pupil's judgment as a narrator, on the meaning of the story, the point of the facts, and the narrator's attitude and feelings. The research tested for (2 criteria): a) Commentary on actions or situations (e.g. with adverbs that define the heroes' actions, with explanatory, final, concessive clauses, or other phrases/utterances expressing an event's cause or consequence). b) Commentary on the characters (e.g. clauses (mostly relative), adjectives, participles, or other phrases/utterances that state perception, judgment, will, feelings and constitute the narrator's comments on the heroes' behaviour and state of mind).
The 3 rd category tested the pupils' texts for cohesion. The study examined how the pupils structured meaning intratextually. The term refers to various linguistic means (grammar, vocabulary) that link sentences to form larger units and comprises the functional use of grammar and syntax structures. Starting from the original work of Halliday and Hasan [28], cohesion is considered to be the main quality that distinguishes a text from a random series of sentences. Cohesion is established by linking successive utterances with specific elements (connectors, pronouns, zero reference, etc.). In a narrative, events are semantically linked with time and causal relationships, that is the ways in which a situation or an event affects the circumstances of another situation or event [25]. The verbs are in past tenses.
Six criteria were used: a) The number and successful use of time indicators (conjunctions, adverbs, various determiners, participles, etc.). b) The number and successful use of causal indicators (conjunctions, adverbs, various determiners, participles, etc.) and other connectors. c) Textual cohesion, established through reference to people and places. d) Textual cohesion, established through lack of reference to people and places. e) The variety of past tenses. f) The correct use of past tenses.
The term coherence (4 th category), according to Halliday and Hasan [28], refers to the sequence of meanings, which makes a piece of discourse understood as a text. It refers to the suitability of the text content relative to the situational context. Coherence is established when the text is suitable, in part and in whole, for the reader and the purpose for which the text was written. The study examined the extent to which (2 criteria): a) The pupils understand the text's purpose and their texts are suitable for the specific situational context.

IAIMTE 2013
b) The texts are focused. All details are organised in a distinct pattern. The story is developed sufficiently, clearly, without digressions, with logical connections throughout the parts of the narration (including the paragraphs). It includes all necessary information (no more and no less).
The category (5th) on the texts' grammaticality and acceptability, tested for (5 criteria): a) The texts' grammar and syntax. b) Conceptual and semantic correctness, and the accuracy and compatibility with its linguistic choices for written discourse. c) Errors, and the extent to which they influence its understandability. d) Visual presentation and readability. e) The study also examined whether the vocabulary was suitable for the occasion and the text's readability (presentation -image -appearance).
As mentioned above, the texts were scored both analytically and holistically (two scales: a ten-point and a six-point). The study examined whether the plot is clear, complete, interesting, and sufficiently developed, and whether the narrative is vivid and reveals personal style. Each text was evaluated for all the analysis categories by two examiners. On the rare occasion that the two scorings displayed a difference larger than two points, a third examiner was called in d .
The data collected from the pupils' texts were complemented by two questionnaires, as mentioned above.
The first questionnaire was administered to the parents of the participant pupils, and referred to the parents' educational level, which was measured in a ten point scale, from not completing elementary school (1) to obtaining a doctorate degree (10). The questionnaire aimed at relating parental education to pupil achievement.
The second questionnaire was administered to the participant pupils' teachers, and referred to written production in class, in terms of allocated time, how it is conducted, and whether self/peer assessment is implemented. The aim was to reveal whether the allocation of more time for the editing of the pupils' texts by themselves with interactive practices enhances their performance.
All data were statistically elaborated using the SPSS software. Table 1 presents the performance mean of all pupils who participated in 2006-2007 and the difference between pre and post-test, per analysis category. e This section first examines the two holistic scorings, and then one by one the five categories that, as seen above, contribute to a clear, interesting and coherent narration. For each category, the tables present the mean of the sum of the criteria involved, while the end of the table presents Narrative Competency, the mean of the sums of all twenty criteria of the five categories (excluding the initial two holistic scorings). The total score of the five categories in the case of the highest performance would be 100.

Data analysis
All cases demonstrate enhanced performance from pre to post-test, as all mean differences are statistically significant. The most marked increase is demonstrated by cohesion, followed by   grammaticality-acceptability. They are followed by episode narration and its evaluation. The least marked increase is demonstrated by coherence. Table 2 presents the difference between performance mean of 2007 and 2012 per analysis category. In all cases, pupil performance displays an improvement from 2007 to 2012. The greatest increase was displayed by evaluation, which is the only category displaying a statistical significance, and then by grammaticality-acceptability. These were followed by cohesion and narration (episode development). Coherence again displayed the least increase. Table 3 presents the 2006-07 pupils' performance mean per school location, and the difference between pre-and post-test per analysis category.
As shown in the table, all cases demonstrate enhanced performance from pre to post-test, as all mean differences are statistically significant. Three area types show enhanced performance. The most marked increase is demonstrated in schools operating in wealthy urban areas, followed by schools operating in a semi-urban location, an affluent town, and in poor mountain rural villages. The least marked increase is demonstrated in schools operating in suburban areas, in deprived neighbourhoods, inhabited mostly by working class population and with more non-native speakers of the Greek language than the other areas.   An increase can be observed in most cases. Three of the mean scores are statistically significant. Compared to the results of Table 3, there is an important difference: this time, the greatest increase is not displayed in wealthy urban schools, as was previously the case, but mostly in schools that operate in semi-urban and poor mountain rural areas, and less so in urban schools. The least increase is again displayed by schools operating in underprivileged suburban areas.

IAIMTE 2013
The above could be explained by the Table 5 data, presenting the educational level of the pupils' parents per school location per research year on a ten-point scale.
This confirms that performance depends on social background. As could be expected, parental education levels are high in wealthy urban schools. It is an area of high social background, as most inhabitants belong to the University teaching and administrative staff. In terms of parental education level, the next area is the semi-urban location, which is actually an affluent small town. This is followed by parental education level in the suburban location, which is an underprivileged area, characterised mostly by working class population and more non-native speakers of the Greek language than the other areas under study. This location demonstrated the least marked increase in pupil performance. The lowest parental education level is displayed by small mountain rural villages, where the pupils display the lowest performance in the beginning and the highest performance increase from pre to post-test, almost equal to the urban and semi-urban locations after the first year of implementation, and an even more marked increase after six years. The area consists of poor mountain villages which are populated mostly by farmers, and have few non-native speakers of the Greek language.
The relationship between pupil achievement and social background has been demonstrated by numerous research worldwide, for several decades [reported in [31]], and has been interpreted in various ways [indicatively, see [32,33]]. A common point in the interpretation is that the process of mastering language as an instrument of communication can only be understood in relation to the social environment in which it takes place.
The above are clearly reflected in the findings of the present research, as indicated by the result of the correlation between the parents' educational level and the pupils' performance (Tables 6 and 7).
In Tables 6 and 7, the result of the correlation between the parents' educational level and the pupils' performance demonstrates that they are proportional; the higher the educational level of the parents, the higher the pupils' performance in written expression.
However, these correlations are stronger for the 2006-7 pupils than for the 2012 pupils. This finding indicates that six years after the implementation of the teaching material, pupil performance is correlated IAIMTE 2013   to the pupils' social background to a lesser extent, which might be attributed to the impact of the teaching material and relevant teaching practices, provided they were implemented by the educators. Tables 8 and 9 show a difference between performance mean for boys and girls, with girls displaying higher performance.
However, the boys show a more marked increase compared to the girls, both after the first and after the sixth year of implementation. The mean difference between boys and girls in the pre-test is higher (6,02) than both in the post-test (4,10) and the 2012 research (4,11).
This can probably be explained by the fact that all recommended activities to utilise the teaching material serve as a motive for action and sociability [34] and boys, being more active than girls, focus more easily on such activities.
Overall, the data demonstrate a considerable improvement in pupil performance between the preand post-test of the first year, as well as between the latter and the 2012 research. Therefore, the research indicates that the teaching material and the respective teaching practices that were adopted in some cases could have had a positive impact on most analysis categories under study for the 1st and 6 th year of implementation. It should also be noted that, after the 6 th year, the greatest increase was not displayed in wealthy urban schools, as was the case the 1st year, but mostly in schools that operate in semi-urban and poor mountain rural areas. Moreover, the correlation between the parents' educational level and pupil achievement demonstrates that six years after the implementation of the teaching material, pupil achievement is correlated to the pupils' social background to a lesser extent.
However, given that the total score for the highest performance would be 100, as can be seen in Tables 1 and 2, despite their improvement, pupil performance remains at a level that cannot be considered high.
The next three tables (10)(11)(12) examine the relationship between pupil performance and the way in which written discourse is produced.    As can be seen above, performance is higher when written discourse is produced exclusively at school and considerably lower when it is sometimes produced at home (Table 10).
It also seems that performance is higher when the pupils' texts are reviewed and edited using communicational and interactive practices and when the pupils themselves are involved in the process (Table 11).
It can be observed that pupil performance is higher when more time is allocated to producing and editing written discourse (Table 12). A positive fact that should be noted is that in 2012 no school in the sample allocates less than 20 minutes for the production of written discourse.
Finally, the next three Tables (13)(14)(15) investigate the extent to which the teaching practices promoted by the current material were implemented on the two pupil samples of the research. The tables display the percentages of the pupil sample per location, way of editing and time allocation, in terms of written discourse production. Table 13 demonstrates that in 2012 a larger percentage of the pupil sample engages in written discourse production at the school, compared to 2007. Also, the use of self-and peer-editing is greatly increased in 2012 (Table 14). Moreover, at least one hour is now allocated to the production and editing of written discourse (no school in the sample allocates only twenty minutes to written discourse production) (Table 15).
It seems that the teaching practices that correspond to the current material (communicational and interactive activities, allocation of time for the pupils to produce and edit written discourse at school) in many instances were not implemented during the first year of implementation. These instances are severely decreased after six years of implementation.
Consequently, it can be assumed that part of the improvement of the pupil performance is due to the improvement of the educators' teaching practices regarding written discourse.

Discussion and conclusions
The findings of the study indicate that after the first year of implementing the current teaching material and the respective teaching practices, that were adopted in some cases, the pupils' performance displays a substantial improvement, since their scores display a considerable difference from pre to post-test in the first year and after six years. Moreover, pupil achievement relates to different teaching practices implemented by the educators regarding written discourse production, as well as to the pupils' social background and gender f .
Particularly regarding the relationship between performance and gender, although the boys' performance before the implementation is often remarkably lower than the girls', after the implementation the boys demonstrate a more marked increase in their performance than the girls. As mentioned above, this fact could possibly be attributed to the fact that all recommended activities to utilise the teaching material serve as a motive for action and sociability [34].
Furthermore, it seems that pupils display an improved performance to a greater or lesser degree, whether coming from privileged or from less privileged social backgrounds (see Tables 3 and 4), as has been illustrated by research that implemented programmes with similar principles not only abroad [indicatively, see [6]], but also in Greece [11]. Sociolinguistics have established that children from non f It would be interesting to conduct further research in order to explore how each of the three factors mentioned above affects performance, which exceeds the -already extensive-scope of the present article. privileged backgrounds have less access to formal language, particularly to written discourse, which represents its most challenging form. It is most important that these children are introduced to this challenging process through original texts from the social sphere, which are more familiar, and through their participation in different interactive activities, where they assume various roles. It seems that this practice, promoted to a certain extent by the teaching material and teacher's guide [23], motivates the pupils to elaborate different genres and expand their discourse competence. The above may explain the fact that, after the 6 th year of implementation, the greatest increase is not displayed in wealthy urban schools, as was the case the 1 st year, but mostly in schools that operate in semi-urban and poor mountain rural areas.
Despite a general enhancement however, the data demonstrated that the pupils of the sample overall display a mediocre performance in terms of narrative competency. For most pupils, the scores are mediocre and their performance greatly depends on their social background, although this tendency decreases after a longer implementation time.
To a great extent, the findings of the present research confirm previous research mentioned above [11][12][13][14][15] on pupil written discourse performance in the Greek elementary school (see Introduction).
They also relate to the findings of research conducted in the USA and other countries [35][36][37][38]. In the above studies, the analysis of a large number of texts of various genres, written by students of various socio-cultural backgrounds, demonstrated that the text performance scores of most students are low or medium, and their success is directly related to their social background; the more privileged their background the higher the students' performance in written expression.
According to previous research on the quality of written discourse teaching in Greece [16,18], these results are partly due to certain outdated views and practices on teaching written discourse. This refers to a process that ignores written discourse production at school as a dynamic cognitive process (elaboration of various versions of a text, allocating time for interactive activities, including self-or peer-evaluation) [see also Introduction].
To a certain extent, these outdated views apparently still survive in primary education, as can be seen from the teachers' questionnaire answers.
For instance, in some cases written discourse production is assigned as homework. When conducted in school, in some cases it is neither allocated sufficient time nor characterised by interactive practices and self-and peer-assessment techniques, although the overall situation is enhanced after six years of implementation.
It can be assumed that the above may partly explain the pupils' mediocre performance in terms of their narrative competency, both before and after the implementation of the current teaching material, despite a remarkable increase after the implementation of the material.
This happens because it does not suffice to simply design and implement teaching material based on contemporary teaching principles. It is also necessary to properly and sufficiently train all educators involved, so that the teaching material can be implemented properly and yield maximum benefits.