[MUSIC] Welcome to session two of this week on human judgement scoring. In this session we're going to look at tools that guide our judgements, that is both the holistic and the analytic rubric, how we design them and how we use them. Remember in our map, we're looking at interactive informal assessment processes, in which we're using our human judgement to more formally guide and evaluate student work. A rubric is a scoring guide, and most scoring guides have to have a statement of characteristics associated with different levels or grades of performance. So, we have a beginner level, an intermediate level, an advanced level, and a super advanced level, perhaps, and with each level of quality or progress, we have certain statements that describe that. So, that when we look at student work, we can say, this work looks like this level of progression. Lots of terms for rubrics: marking schemes, progress indicators, progress maps, matrices. All of these terms refer to the same thing, a set of rules to guide your judgment as to the quality of work. Now, the reason we do this is so that we have a clear understanding of what the learning intentions are, and what the stages of progress are. If the teachers have this clear in their minds as to what progress looks like, it's much more likely that students will also understand that. In fact, research has shown that when students are involved in developing rubrics, statements of progression, they come up with statements and criteria that are very similar to what teachers and experts would come up with. Rubrics allow us how we score lot's of different types of assessments: essays, portfolios, performances, behavioral observations, and next week we're going to talk about involving students in student self and peer assessment. All of these assessment techniques require judgements, and judgements are guided with rubrics. Now, let's not kid ourselves, this is not a precise science, it's an art form, it's a complex judgement made, hopefully by competent experts - remember you're the teacher, you're the expert in the room. And what we need as we make these judgements, is evidence that validates our evaluation. If we think it's level three, if we think it's a B+, then hopefully another marker, or another type of learning process, assessment process would tell us, yes, this is consistent. Certainly if another teacher in your school is teaching the same grade, doesn't give a similar grade to yours, you know there's a problem. There are two styles of rubric, the holistic and the analytic. The holistic rubric tries to take everything into account all at once to guide your impression. These rubrics tend to be very sketchy, not too detailed, but allow you to make an over-all evaluation. Analytic rubrics on the other hand, allowed detailed description of different dimensions of a piece of work. And these different dimensions are really useful for guiding teaching and learning. But if you're in a hurry and you need a grade by the end of the day, or if there's no more teaching and we've reached the end of the course, then a holistic rubric will probably be sufficient. One of my favorite foods, which is bad for me I know, is the chocolate chip cookie. I love chocolate chip cookies, because chocolate is wonderful. And when I go out and buy chocolate chip cookies and I'm tasting-- if there's an offer to taste a chocolate chip cookie at the supermarket and I taste it, basically I'm going to score it in a holistic fashion. "This is really horrible", "This is okay but I wouldn't buy it", "This is amazing, what do you mean I can only have one?", and "Show me where they are, I'll buy lots of them". This overall holistic judgement is sufficient. And you can see this smiley face scale ranging from the most horrible to the most amazing. In terms of an overall snapshot judgment, this kind of scale is all we really need to make a simple decision. Unfortunately, most decisions that we need to make in education are much more complicated. On screen now is an analytic rubric that breaks the qualities of a chocolate chip cookie, my own preferences of course, into one, two, three, four, five different categories or dimensions. And on each dimension, there's a description of what I think makes a chocolate chip cookie delicious, good, fair, or poor. So, we've got five dimensions and four scales, and twenty little cells. So, I need to make five different decisions about each chocolate chip cookie to use this analytic rubric. Clearly this is much more time consuming. But actually if I'm a chef, and I'm making chocolate chip cookies and I want to make the best chocolate chip cookie in the world so that everybody buys my chocolate chip cookies, this is the kind of information I want from my potential customers. I want detailed analysis of what they like and don't like about my cookies. So, which one should you use? Well, that's a purpose driven decision. Holistic rubrics are very useful when it's summative. You've reached the end of the course. You can't do anything more about it. You're just trying to make an overall rating, or when you don't have much time to make a decision. Or if you don't have much content expertise, and you just want an impression of quality. This is especially the kind of rubric I think students could be using if you involve them in making ratings. Analytic rubrics, on the other hand, are very formative because it tells you the relative strengths and weaknesses of a performance. But you've have to have enough time to make multiple judgements with each piece of work. And of course, it requires expertise to make the decisions - so many different decisions about so many different aspects of a piece of work, that may only slightly differ one from the other. This is probably not the kind of decision making process you want a student to engage in. But I would certainly show the analytic rubric to the students, so they know what their teacher is going to be looking for. So, we've covered the two basic kinds of rubrics that you need to be able to come up with or look for from your curriculum system. Next, we're going to talk about how to use such a thing in evaluating essays, an important part of learning in most subject areas. [MUSIC]
