Analysis of Students Use of Multimodal Representations in a Science Formative Assessment (Assessing Pupils’ Progress, APP) Task in the UK

The research questions guiding this study are the following:

APP is the government’s Assessment for Learning Strategy²³ and overseen by the Qualifications and Curriculum Authority (QCA) and Department for Children, Schools and Families). The DSCF claimed that the use of APP would promote Assessment for Learning (AfL) and lead to improvements in teachers’ curriculum planning and through this a raising of standards. APP in key stage 3 science was introduced in January 2009 in the UK. According to Rowe (2010),²⁴ the APP approach particularly supports periodic assessment, when teachers sum up progress over the medium term and adjust their curriculum planning. It also aims to provide support for day-to-day assessment and at transitional points between year groups, schools and phases. APP should be based on a wide range of evidence, be personalized and integrated into teaching and learning, and should yield both formative and summative information which feeds into curriculum planning and personalized learning. APP aimed to provide a better balance between content and scientific processes. APP is assessment for students undertaken by teachers and provides information to be used as feedback to modify the teaching activity and teachers’ planning. APP uses pupils’ day-to-day work that is already in pupils’ exercise books, portfolios and presentations on which some of their spoken contributions in class.²³ In this study, we consider that APP task gives students an opportunity to explain what they understand, after each module was finished. Therefore, APP task was one of the ways in which multimodal representations used by the students.

Multimodal representations are defined as various modes of language and forms in order to present and represent science data, ideas, knowledge and concepts and support a variety of learning activities, cognitive processes and deeper understanding of the science.²¹

Referring to three modes, Gilbert (2004)²⁵ defines, ‘verbal mode' as the reality and relationship with realities in writing that students use, ‘symbolic mode’ as made up by chemical symbols, chemical formulas, and mathematical expressions, and ‘visual mode’ as graphic, graphs and pictures. Our work in relation to multimodal representations has been refined through experiences in the science classroom of one of the researchers. In this study, the multimodal representations appearing in the pupils’ presentations of APP task are classified into three modes as mentioned above.

The study was conducted in the Year 7 of coeducational and catchment school located in Nottingham, in the UK. Students’ age range in the school was 11-18 and student roll numbers was 1359. The school had been converted from Foundation School to Academy at Aug. 31, 2010. This academy school was independent, nonselective, state-funded schools that fall outside the control of local authorities, and were managed by a private team of independent cosponsors. The curriculum in this school was decided without recommendation of government. Most of schools of the UK were KS3 consists of Year 7, 8 and 9. Instead, Year 7 and 8 belonged to KS3 in this school and Year 9 starts GCSE curriculum.

The participants were 54 Year 7 students. The experimental group consisted of one class of 27 students and the control group consisted of another class of 27 students. The experimental group received instruction encouraging using multimodal representations as evidences to support claims. On the other hand, the control group received instruction without encouraging multimodal representations. Students in two groups learned the same topic and the same content on the same module in Year 7 science course.

The teacher participating in this study had five years of teaching experience at the secondary level and majoring in biology. She taught both the experimental group and control group.

The authors developed the instruction encouraging multimodal representations based on prior research²⁶ and one of the authors participated in this study. She explained details of the research and the lesson to the teacher. She acted as an assistant teacher of two groups to answer fairly to the questions of the students and helped students prepare for presentation without mentioning about multimodal representations. She had 10 years of teaching experience majoring in chemical engineering and chemistry education. She was placed at the local school in the UK by ‘Overseas Expansion Program of Excelling Teachers’ for one year. She was involved in instruction encouraging multimodal representations as an assistant teacher.

Science modules of Year 7 consisted of 10 topics of module for three semesters in chemistry, physics and biology subjects. Topics and contents of science modules in the experimental and the control groups were shown in Table 1.

The topic selected for this study was ‘Life story’ in biology subject and also chose ‘Sexual reproduction’ content among five contents. The lessons were composed of four periods having 60 minutes for each period. During the first and second periods of the lessons, students of both groups learned the same contents as flower dissection and pollination. At the third period of the lessons, both group of students planned and prepared for APP task. They were divided into small groups of two to five persons. The group discussion took place in terms of how to discuss and organize their explanations and share their ideas. The last period of the lessons, students of both groups made their own presentation as APP task to whole class.

Lesson plans for the experimental and the control groups were shown as Table 2. The instruction of the experimental group designed to encourage the use of multimodal representations. Experimental group students started with explanation using a variety of representations as evidence to support their claims. Then they were asked to recall an importance of evidences and multimodal representations to persuade other people at the third period of the lessons. The instruction was to encourage the students to discuss how multimodal representations were used and whether these representations were useful in helping others understanding the ideas presented in their presentations.

APP task supported periodic assessment at the end of the lessons. During this assessment, students in both groups participated in a small group discussion and whole class presentations.

The intent of APP task was to encourage the students to discuss how scientific concepts were expressed and whether these were useful in helping them understand the ideas presented in their presentations. During planning and preparing for APP task, the students of both groups have an opportunity of discussion in a group. APP task was asked to produce students own explanation and present to the whole class. Students constructed a model or leaflet, created a PowerPoint, drew pictures, wrote scripts and practiced a role-play for their presentations. These presentations should reflect on students’ understanding and how sharing their ideas help students understand science concepts. They were video-recorded and collected by author, and then transcribed for analysis.

CAT was an assessment of a range of reasoning skills. CAT was sets of tests which assess a student’s ability to reason with and manipulate the three different types of symbols that played a substantial role in human thinking: words, numbers and shapes or figures, i.e. verbal, quantitative and non-verbal reasoning as thinking with shape and space.²⁷ CAT scores were utilized in order to acquire the students who were progressed form primary school. The scores indicate general transferable abilities, such as the ability to recognize similarities, analogies, patterns and relationships, all fundamental to understanding and using new information. They were designed to minimize the role of prior learning and could therefore provide an indication of potential.

Students who were score higher than above average 127+ was the ‘Very high’ category of CAT scores as shown Table 3. CAT test was conducted at the beginning of the new academic year for experimental and control group.

Module Tests were conducted after the lessons of each module as written test for judgments of academic achievement of experimental and control group. In this study we used test results of 2 modules before implementing target module for the study.

In the APP task, students were asked to explain to their friends what they understood about the topic given. Experimental and control group students completed this presentation task in the last period of the lessons. The APP task was scored using an analysis framework of multimodal representations. The analytical framework focused on two aspects of student’s presentation; an assessment of the use of multimodal representations and an assessment of embeddedness of multimodal representations.

Firstly, the assessment rubric focused on using modes of representations, such as verbal mode, symbolic mode, and visual mode. Verbal mode was representation for transferring concepts and ideas from evidence to support students' claims.^26,28 Mode was composed causal relationship, enumeration and experience of everyday life related to scientific phenomenon. Symbolic mode was the ability for the conceptual understanding in chemistry. There were the chemical symbols, unit of dimension and mathematical formulas and in sub-modes of symbolic mode. Visual mode was made explicit information and knowledge.²⁸ Mode consisted of picture and graph. ‘Picture’ aroused students' interest,^29,30 and brought positive relationship between problem-solving skills and students' visual representation.³¹⁻³³ ‘Graph’ was visual displays that depict the relationships between continuous variables in pictorial forms.³⁴ One point was added according to the frequency appeared in each sub-mode of multimodal representations, and the sum of each sub-mode score was the score of the three mode.

Secondly, the embeddedness of multimodal representations was accurately and coherently use of the representations in the context of the student to communicate with others. It meant the qualitative analysis of multimodal representations. It consisted of integration, accuracy of claim and/or evidence and emphasis. Integration was linking among the different modes representing science information.^26,35 Adequacy was a logical relationship between the representation as evidence and the claims. The prerequisite for adequacy was made up of the correct scientific concepts.²⁶ Emphasis was the way to help others understanding such as captions for figure and grape, placed next to the text and referred to the next with the associated visual mode, and key terms underlined or highlighted.^26,35

The analytical framework of understanding of scientific concept was developed for this study based on the analysis framework which Nam et al. (2008)³⁶ developed. Understanding of scientific concept consisted of two parts: big idea and describing scientific concepts.

Big idea meant fundamental elements of scientific concepts. There were two big ideas of this particular plant lesson. Describing scientific concepts included accuracy of scientific concepts, text characteristics match intended writing type and language appreciate for audience (Table 4).

Data were analyzed using SPSS 18.0. CAT score, Average APP level and Module Test percent were tested for homogeneity of the experimental group compared with the control group using independent t-tests. Differences for encouraging multimodal representations were examined for all students of both groups using a t-test.

There was no significant difference in the mode of multimodal representations between the experimental group and the control group (p <.05). In the two sub-modes of multimodal representations such as verbal and visual mode, we confronted with data showing no statistically significant difference between two groups. The symbolic mode was not shown at all in the experimental group and the control group (Table 6).

There was a significant difference in the embeddedness of multimodal representations between the experimental group and the control group (p <.05). In an accuracy of claims and evidence as a sub-element of embeddedness, the experimental group gained significantly higher than the control group (p <.05). In integration and emphasis in the sub-element of embeddedness, the experimental group was higher than the control group, but no significant difference (Table 7).

The results indicated that students involved in the multimodal representation instruction were significantly better in embeddedness of multimodal representation.

There was a significant difference in understanding of scientific concepts in a science APP task between the experimental group and the control group (p <.05). All sub-elements of understanding of scientific concepts in the experimental group was significantly higher than the control group (p <.05) (Table 8). From these results, the multimodal representations encouraging lessons had an effect on students' understanding of scientific concepts. Students organized their ideas by using a scientific language.