Thorston is giving us an interesting question, a topical interesting question, do rubrics guide student behavior? So to what extent do you recommend to guide the students' learning behavior using rubrics. So I guess it's sort of saying using them as a predictive tool for how students will respond. >> Well I'm not so sure that it's a predictive tool, but it can definitely be a tool for self assessment and to self regulate one's own learning. So, if you have a rubric, where you have standards and you have different criteria associated with those standards, then I mean, one of the best practices is provide opportunities for students to see the rubric early on so they can know what is expected but also to ask them to self assess where they are at because then they can see if they are at the more proficient level, for example or if they are already at the advanced level. How they can move towards the advanced level and therefore modify their learning. >> Just for a different point of view. >> Yep. [LAUGH] >> Just to be controversial. I agree rubrics can be a really good framework to get an idea of what's expected and how to actually achieve those different criteria or criterion in the actual assessment. But sometimes, do you think they go too far and they can actually be quite restrictive? I mean, I am from a creative background. I work in an art design faculty. So obviously sometimes, you can't fit a solution into a grid. So is there a balance between having a really heavily detailed rubric clearly outlined down to the nth degree of what the student should do to get a certain mark or is there some sort of room to use rubrics as more of a suggestive guide for ways to start thinking. >> Holistic approach. >> Exactly. >> Well, there are two different types of rubrics. There are holistic rubrics that actually guide you to do that more of that holistic looking at where someone is at and allows for more of that creativity I suppose and then there's more analytical rubrics which are those that are saying, they will be deducted two marks or three marks. The rubrics that we've used in the MOOC even though they're not structured as a grid per se because of >> That means the same information's there. It's just not laid out in a grid format. >> Yeah, it's still a rubric in a sense because there are standards and there's criteria associated with them. They're just drop down menus because that's how Coursera structures it. They're more analytical right? So they are the ones where if you select your advance you get a four for example, if you haven't done a particular thing you get a zero and it's very analytical into that T. While in some of the other courses I teach I actually use the rubric to give feedback to students but I don't actually use it in a very detailed analytical way. It might guide how you might give the overall mark. Obviously if you're always at the just barely meeting the criteria, you might not get a A. >> Yeah. >> Or high distinction. But it guides you but it's not necessarily to the point numerically. >> And I think what we see in the MOOC is that there are hundreds of different types of learners here. >> Yeah. >> With different types of background and different types of experience. >> Yeah, absolutely. >> And the thing with the MOOC is that we have to find a balance between providing as much guidance and analytical framework as we can, while also realizing that not everyone has the same level of experience there. So I mean, for me- >> And they have different levels of experience using rubrics as well. >> Exactly. >> Yeah. >> So I mean, I think we're using them in a way to actually provide as much of that framework as we can, but also as a jumping off point for more analytical sort of thought processes or broader thought processes. So given the scope of the MOOC, sometimes it's hard to pinpoint. But it really is about what's coming back and the experience of going through this than the actual precise numbering a lot of the time but. >> And, also a way to provide feedback as well. >> Exactly. >> I mean that's what I love about rubrics is that it does help to give feedback a little bit more efficiently, because then you don't have to write down why someone has met a particular criteria or not to the T. And also for that self assessment, I think it's really important. But, I mean it has its limitations, actually. >> Yeah. So what, so wrapping up, what are we saying with the rubric. Can they, should they guide student behavior, kind of yes or no? Yes and no, really. >> I think they can for sure if they're used as a self assessment instrument or to self regulate someone's learning and help them improve. Absolutely they can. >> Yeah, but also sometimes it's good to not lock yourself in so rigidly. >> Yeah. >> Depending on the context. >> Exactly.