Projects are often listed among alternative tools for assessment, especially when talking about teenagers. The reason for this seems to be clear, as project work is potentially motivating, stimulating and challenging, allowing learners to show what they can do. Besides, it generates a lot of data, and is therefore a rich tool to gather evidence of learners’ production and of learning.
However, assigning a grade to projects is not that easy, and being fair to all learners when translating the data gathered into a grade may be challenging. One of the reasons for this difficulty is the fact that projects involve both individual and group work. When we assess tasks that involve meaningful interactions among peers, “it inevitably involves confounding factors that might compromise its construct validity – that is, its ability to measure what it is intended to measure” (Jang, 2014: 127).
Thinking about using projects as an assessment tool means deciding how it is going to be done. Some questions to be asked in this process include:
· What kind of assessment is going to be favoured?
o Formative, summative, or a combination of both?
o Formal, informal, or a combination of both?
· Is it necessary to generate traditional grading?
· What are parents’ expectations and beliefs in terms of assessments?
· What is the influence of the teaching context?
· Is it possible to incorporate projects into the assessment system of the school/ language institute where I teach?
Ways of approaching the assessment of projects
As we can see, assessing projects can be a complex matter and, as such, it can be dealt with in different ways. Maybe the least complicated way of approaching projects as an assessment tool is to opt for formative assessment, because no formal grade is given to the project. Thus, teachers give general feedback to learners and parents, and use the data generated to inform further work done in class.
Another alternative is to use projects to encourage collaborative assessment, where the responsibility of selecting assessment criteria is shared between teacher and learners. Harris & Bell (1996) highlight that “the collaborative method of assessment easily enables groups goals to include process”. This, however, depends on the group’s dynamic and the relationship between teacher and learners. When dealing with teenagers, collaborative assessment is potentially engaging, as learners feel more empowered and part of the process.
There are also other ways of involving learners in the assessment process. This can be done via self- and peer-assessment. As McKay (2005) explains, learners “can be encouraged to be active participants in the assessment process if they are guided to think about their own performance, and the performance of peers. A way of dealing with it is to prompt reflection with questions such as “What have I learned?”, “Which problems did I encounter?”, and “How did I solve the problem?”.
Besides the kind of assessment chosen, the design of the project itself may make assessment easier. One way of doing so is to think of the project as a series of steps. Being complex activities, projects naturally involve different steps (McKay, 2005). Breaking them down in this way can favour assessment in so far as it may be easier to isolate the different skills involved in the completion of the project: writing, speaking, reading, listening, information gathering, selection of information, etc. However, as McKay explains (2005) explains, it is important that language is involved not only in the process of carrying out the project, but also in its outcome or final presentation. This can happen “either through an accompanying description (a written piece, an oral presentation) or within the exhibit itself (a role-play, a simulation, a video of a language event in which learners were involved)”.
Once the parts or steps that are going to be assessed are clear, it is time to focus on how the assessment will take place. If the idea is to assign a grade to the project, Coombe et al (2012) suggest that “projects should be assessed in the same way that writing samples are assessed”. Thus, it can be done holistically or based on an analytical scale. If the idea is to have as much reliability as possible, analytical scales tend to be a better choice, but this will depend on the teaching context.
The scales suggested by Coombe et al (2012) to access projects encompass linguistic and non-linguistics aspects of the project, as well as elements related to both the production process and the final outcome. This is a good starting point when devising the assessment criteria for projects, but it is also important to remember that the scales and the criteria used have to be well communicated to students, so that they can understand the grade assigned to their work.
Although these ideas represent different ways of dealing with the assessment of projects, one does not necessarily exclude the other. As Harris & Bell (1996) point out, the assessment of projects can be done combining different methods. This will depend on the teaching context and the students’ needs.
Implementation of project work and assessment issues
Being responsible for the implementation of a course for teenagers which involved extensive work on projects, one of the most recurrent issues I faced regarded its assessment. Questions about how to assess the projects came mostly from novice teachers, but some experienced teachers showed concerns related to the validity of the assessment. How to use projects as evidence of learning and data for formative assessment was also a recurrent concern.
This course was implemented in a large language institute in Brazil and impacted more than 8,000 lower secondary learners per year and about 400 teachers in more than 30 different cities. Therefore, reliability was an important aspect to be considered, and all the questions raised by teachers made it clear that they were not sure of what they were supposed to do.
Investing in informal formative assessment would not fit the context of this language institute, because of the value placed on formal grading. Besides that, parents also value formal summative assessment, and tend to use grades as evidence of learning. In a survey carried out with 300 parents of teenage learners (12-16 years old), the most frequent answer (43%) to a question about how parents follow up their children’s learning process was ‘their grades’. Only 16% mentioned formative tools such as teachers’ feedback. As Jang (2014) explains, parents’ beliefs are important and may influence adolescents’ learning and assessment experience. According to the latter, when they tried to use the project as an informal tool of formative assessment, most learners asked them what their grade was, even after receiving comprehensive feedback on their performance. This kind of attitude shows that grading projects formally is important in this context, as it guides the perception of learning.
Teaching context and associated needs
With the context above in mind, I devised an assessment framework, which is now in implementation. The main aim of this framework is to help teachers use the projects proposed in the course as assessment tools, while addressing institutional demands, learners’ needs, and parents’ beliefs.
Thus, this framework is expected to cater for:
· the main characteristics of projects, which include assessing the multiple skills involved in the making of the project and the interaction among learners
· parents' beliefs and learners’ needs, which means that it should result in a formal grade and, at the same time, provide genuine information about learners’ performance
· teachers’ needs, providing them with clear pointer on what to assess and what criteria to use
· institutional needs, which can be translated in terms of the balance between validity and reliability.
The framework
The conclusion reached after examining the teaching context and its needs was that projects should be graded. In order to systematise the assessment of different elements of projects and, thus, help teachers grade the projects, these should be organised into a series of steps. The parts of each project may vary depending on what is being developed, but it should involve at least a written component, ongoing spoken production, and an outcome.
With organisation into steps established, assessing the different elements of the project can be done separately. The elements to be assessed are:
· written production, including preparation for the final outcome or presentation
· spoken production, including the interaction among learners during the process of preparation for the final outcome or presentation
· the final product of the project, which can be a written outcome or an oral presentation
· research carried out by learners
· information / content selected and included in the project.
In order to foster reliability, analytical scales are to be used to help teachers award grades. As teachers are already used to employing analytical scales to assess learners’ written and spoken production in other courses, this facilitates their use for assessment of skills used in the preparation of projects. An additional scale (Figure 1) is being introduced to assess the final presentation or outcome, the research carried out by learners and the content of the project.
Figure 1 - new analytical scales introduced to assess some elements of project work. Adapted from: Coombe, C., Purmensky, K. & Davidson, P. Alternative Assessment in Language Education. In: Coombe, C., Davidson, P., O’Sullivan, B. and Stoynoff, S. (2012) The Cambridge Guide to Second Language Assessment. Cambridge University Press: 151.
The existing scales for speaking and writing result in a total of 10 points each. Thus, in order to balance the weight given to presentation, research and content, the 30 points are also converted to 10. With this adjustment, project work assessment results in a total of 30 points (Figure 2), which is to be combined with other assessment tools, such as tests, to give the total mark for the semester.
Figure 2 - components of the project work grade
References
• Coombe, C., Purmensky, K. & Davidson, P. (2012) Alternative Assessment in Language Education. In Coombe, C., Davidson, P., O’Sullivan, B. and Stoynoff, S. (Ed.) The Cambridge Guide to Second Language Assessment (pp. 147-155). Cambridge: Cambridge University Press.
• Harris, D. & Bell, C. (1996) Evaluating and Assessing for Learning. London: Kogan Page
• Jang, E. E. (2014) Focus on Assessment. Oxford: Oxford University Press.
• McKay, P. (2005) Assessing Young Language Learners. Cambridge: Cambridge University Press.
Biodata
Leticia Moraes has been involved in ELT for more than 15 years. During this time, she has worked with young learners, teenagers and adults, but in the last 4 years she has been dealing especially with secondary learners, developing courses and tests for students of this age and also training teachers to deal with them. She has also published articles about project work with teenagers, and delivered talks and workshops on this subject in conferences and online events in Brazil and abroad. Her experience also involves materials writing, and setting up and organizing training initiatives for teachers. She is currently one of the Events Coordinators of the IATEFL YLT SIG and a partner at Troika.
*This article was originally published in the IATEFL TEASIG Newsletter, Issue 64, 2018: The newsletter of the Testing, Evaluation and Assessment Special Interest Group of the IATEFL. Source: Troika.Br
Seja um(a) colaborador(a) do Blog SBS! Se você é profissional da área da Educação e deseja colaborar com o nosso blog, envie um de seus artigos para nós através do e-mail mkt.estrategicosbs@sbs.com.br com o assunto "Artigo Blog" e publicaremos aqui