On Marking

Serendipitously (via pedablogue, a site which I originally visited for its witty name) I came across this interesting if lightweight article discussing how flexible one might be in marking (or grading in the US), and its comments.

I hate marking. I put it off as long as possible; but the students were asking me about it last Monday, so I have to do it this weekend come what may.

Why do I hate it? Two reasons, really. First, I have no problem with providing formative feedback; indeed on the Master's course where the submission of draft work was encouraged, I tackled it with enthusiasm at the first opportunity. My problem is really with summative assessment—actually closing down the debate and discussion with a unilateral mark. I am not so confident with my own authority as to do that comfortably. It is not made any easier by the fact that in deference to my seniority (perhaps) second-marking colleagues rarely disagree with me. OK, there are some (most, I suppose) clear cases, but...

The second reason is of course that the work students submit is the rawest form of evaluation of the teaching. They have done the best they can, I assume. I have no problem with castigating them for lack of referencing, for grammatical solecisms, for poor structure. But when they show that they have not really understood a concept—that says more about my teaching than their performance. And if several of them make the same errors; that says more about me than about them.

In the States, "evaluation" is often used for what we in the UK call "assessment"; we routinely (and without much thought) refer to "assessment" of student learning, and "evaluation" of teaching. But we do make a big mistake in assuming that "assessment" is purely about students; it is also about us. It is a cruel mirror to the teacher's effectiveness.

In part, of course. Oh, and see http://www.learningandteaching.info/teaching/marking.htm for ways of removing personal bias.

Today our session included a discussion of what really works in teaching. See http://www.learningandteaching.info/teaching/what_works.htm What really works is, in Hattie's phrase, "dollops of feedback". Marking really matters, and detailed constructive marking feedback matters even more. Going back to the link at the top of this post, the "grade" is a really crude instrument. Your scrawled comments in the margin are for more important for actual learning. (Assuming you can get the students to read them, of course.)

On preparation

My email is working again, so ignore the earlier post.

As I wrote to a correspondent a few minutes ago:

• " It's 11.25 pm, and I am wrestling with my lecture for tomorrow. How did I ever teach for two hours on "motivation and learning"? (I go through this every year--perhaps for the last time?) I have reviewed my PowerPoint, which is rubbish, and the exercise, which even I don't understand... And I have just picked up a new text (pub 2006--my first of the year) on this with 400 pages of text and 70 pages of references (which the keen ones will expect me to have read already). So I need to re-do the whole thing. Not that I will use it, of course, it's merely a safety net in case the discourse does not emerge. It's a con-trick, really. We are supposed to teach the students that if you are properly organised this kind of situation does not arise. Bullshit. I've been doing this for 40 years and it still happens. And I would not have it any other way. I'll miss it! "

(Note to students; you're not supposed to know that!)

On being off-line

For some reason my main email account (to which other email addresses get re-directed) has been corrupted. My ISP says that they will fix it in 48 hours, but that I may lose anything still in the in-box on the server. So my apologies to anyone who has tried to contact me over the past day or so; the gmail address still works, though. Try that one or try again in a couple of days. Thanks.

On being (over) prepared (2)

OK; I did the session. It went "quite well" as we report to our colleagues who bother to enquire. I managed to avoid imposing all my slides on the poor students (the follow-up web page with the presentation material I actually used can be seen here), but the evidence of the closing stages was that they were mainly baffled, if a little intrigued.

We started with two student mini-presentations; these were pre-arranged as part of the overal scheme of work, and consisted of two five-minute presentations; one was on Dewey, and the other on Freire. Both of them provided quite useful links with associated material for the main session, so that was helpful. I followed up the Freire presentation with some more material (which I just happened to have prepared earlier) on "banking education" and his overall project from The Pedagogy of the Oppressed.

As we moved on to the main topic, I asked the students about what they had written about as "critical incidents" in adult learning. The first student to volunteer was spot on; she spoke about her decision to return to full-time education, saying something like;

• "A number of things changed in my life, and that set me thinking about what I was going to do with it. So I decided to get off my butt and get a proper education..."

As I commented, you can't get much of a better definition of "transformative learning" than that! Other students reported on; how studying literature at university was different from doing it at school, a dance module she was taking which she hated because it was so unstructured, and the experience of becoming a mother for the first time. I was able to pose some questions in response to each of these which helped to focus the idea of transformative learning (at least, I thought so).

In introducing the input I made explicit reference to the Kolb learning cycle, pointing out that they had had concrete experiences, the critical incident exercise had helped them to reflect on them, and that we were now moving on to theoretical/abstract conceptualisation of that reflection.

We were now half-an-hour into the class (nominally a two-hour session, or a one-hour lecture followed by a one-hour seminar according to the bureaucrats). The students (10/14) needed prompting to contribute, but they became more forthcoming as we went on, and they found that I was keen to pick up on their ideas and relate them to the taught material. (Until the end.)

So to my presentation. (You can see the PowerPoint material at http://www.dmupce.org.uk/transformative/Transformative_Learning_files/v3_document.htm)
I followed through the eight "mainstream" slides as planned, and introduced some anecdotes and comment as I went along. At "A view of its significance" we had a brief discussion of how some adult education practitioners emphasised this transformation (particularly in term of building self-confidence) at the expense of the actual "stuff" to be learned; this elicited some protest—as indeed it should—from the mature students who had come in to university via "Access" courses (a non-standard route for mature students without the usual entry qualifications). This could have gone further, but on reflection my reaction probably came across as defensive, and thereby alienated those students in some measure. Damn!

There was some protest at the "Mezirow's stages" slide that I was not leaving it there for long enough for them to make notes; I responded by promising that it would appear on the web, but also questioning why they wanted to make notes of the detail. The fact that we fell back to that level of conversation suggests we were back at "transmission" learning, in the terms of this session.

On "Perspective transformation" I raised the issue of the emphasised assertion of values. I enquired whether anyone had done a mini-presentation about Gramsci. One student had! I asked about "organic intellectuals"; she searched her notes and came up with a definition (which was slightly unfocussed), but it did give me the cue to discuss the issue of how current educational policy "creams off" into formal education and the middle classes, those people who might otherwise have become "community leaders". I'm not sure that this meant much to them, and at this point I began to feel that my own awareness of the links between this notion of transformative learning, and a host of other ideas in adult education was beginning to lose them. The relationships make sense to me, and I want the students to pick them up, but I should have been more focused in the direct presentation.

So to the questions posed at the end of the main sequence. There was little spontaneous response. Sometimes you get the feeling that if they do say something, it is chiefly to make you feel that your efforts have not gone unappreciated. It was like that this time; but the student who had made the original brilliant comment on her "critical learning incident" did ask how transformative learning could be anything other than a Good Thing. This was my cue to talk about the less value-laden notion of crisis intervention theory, and how the stages described by Mezirow could go wrong. I did not, for some reason, relate this to my own research.

By this time they had had enough (after about 95 minutes); there were no further responses to my request for questions or discussion points, and so I wound the session up.

From moment to moment, it all went pretty well according to plan. I managed to contain myself about Perry, Belenky and Gilligan (who they? it doesn't matter). But it could have been better, and the "tipping point" at which I lost them was my over-stated assertion about Access courses, and my subsequent defensiveness. The mature students in this group probably identified their Access courses as their occasions of transformative learning, and I managed to undermine and devalue that. Damn! again.

That was not the problem I thought I was going to have. But I did; it was not fatal, and all being well they will find enough material on the web-page to encourage them to explore further, but it could have been better.

Sorry! Didn't mean to write so much, but that is what happens when you start doing disciplined reflection!

On being (over) prepared

One thing about being semi-retired is that you have more time. Or at least you do if you are stuck in the house waiting all day for a delivery (of joinery supplies, of course) which never came.

So I spent the day preparing for a session tomorrow, standing in for a colleague, with final year undergraduates whom I have not yet met. It's on "transformative learning". It's a fascinating topic, which reaches out in all directions, and so it was not difficult to spend the day looking things up in books and journals and the net, and putting together the presentation. Then I had to impose some order on all the stuff I wanted to convey...

Uh-oh! Dangerous! Having spent the best part of forty years never having enough time to prepare, I know that this could well be the kiss of death for this session. What I want to convey? That's the way novice teachers think; and it is ironic that given ample time to prepare a two-hour session, I fall back into that trap. 44 slides? No. It is a recipe for disaster.

I have at least learned something about how PowerPoint can prepare a "summary slide", but still requires you to enter hyperlinks manually, so I am free from its dreaded inexorable bullet points (it's an oldie, but see Edward Tufte on this) but will I feel compelled to use the stuff because I have it, regardless of whether the students can/want to take it in?

I'm posting this now as a hostage to fortune. I could edit it tomorrow, but I promise not to; but I will report on how I actually handled the session.

On the teacher as a variable

Like most real-world disciplines, with the exception of best practice in medical research, it is rarely that those of us in education get to witness anything resembling a controlled experiment. In this case, all I mean by that is a situation in which just one variable changes between two teaching situations, so that we are privileged to get a clue about the significance of that variable.

I won't go into detail, partly to keep this anonymous, but partly for other reasons. However, I recently did two teaching observations, of different teachers, immediately after each other. They were teaching full-time mature students on intensive language courses (the language was one I do not understand and can't even read—which is always interesting anyway!) What I had not realised was that they were both going to teach the same group of students, in the same room, with just a 15 minute comfort break between the sessions. Both teachers were confident and fluent (but not native) speakers of the language, and indeed were graduates of a very similar course.

There was one other minor variable which changed, of course; they were not teaching exactly the same material. However they were both teaching points of grammar which followed one another according to the syllabus.

The first teacher was the more experienced, and the more confident; he had had some input to writing the course. His approach was very conventional; he introduced the topic, worked through some examples with extensive student questioning, assisting students with step-by-step prompts as they struggled with the grammatical construction, and then set a written exercise from the course book. When they had done it, he went through their answers, going round the class with targeted questions about it. Apart from the book, he used only questioning and the whiteboard (probably appropriately). Not exciting, and raising interesting questions about how to teach people to manage a complex rule primarily through individual examples, but sound stuff.

The second teacher was much newer to whole-class teaching, having previously mainly worked on individual tutorial (or "oral practice") sessions. Although familiar with the material, he was not as familiar with the prescribed text-book, and frequently had to look down at it, open on his desk, to check that he was following the recommended structure. The session followed broadly the same pattern. As far as I could gather, though, all this teacher's examples (bar one) were straight out of the book. He showed his nervousness by banging the cap back on to his marker pen after each major point he had to make. He asked questions, but not as frequently as his more experienced colleague, and he did not wait as long for the answers, or prompt students to construct their stumbling answers. Perhaps most important, he did not look at the students much. Like any beginning teacher (and in case he reads this, he'll get there, with more practice and experience) he was pre-occupied with himself and his performance, rather than the students' reaction and learning. He fitted with Dreyfus' and Dreyfus' definition of a "novice".

What did not really occur to me until I reflected on the trip back from the observation, was the significance of what I had seen of the students' behaviour. I usually have to infer this from a position at the back of the room, but on these occasions, because of the fixed layout, I had to sit at one side between the front row of students and the teacher's dais (yes, there was one). So, opportunistically, I got to see them from the front.

It would have been good to have a video record. In the first session, the students sat more upright, spent more time looking at the teacher (who in turn was looking at them; it's called "eye-contact" in the jargon, which sounds portentous, but just means "looking at each other") than at their books or notes (although some annotated their books, in neither condition did they seem to make many spontaneous notes). In the second condition, they appeared more "slumped" in their positions, and looked down at their desks for more of the time, only raising their eyes occasionally to the board as the teacher wrote up his examples, (which he actually did rather better than the first teacher, making use of different coloured pens to good effect.)

There was some difference, too, in the students' responses to questioning. With the first teacher, more hands went up to volunteer responses; the students seemed more confident, and more prepared to take risks. They fairly readily suggested their own examples of the grammatical construction, and worked their way through the translations with an apparent sense of achievement.

The second teacher more frequently (I wish I had been counting, but hey... this was real-world grounded theory non-participant observation) had to use the "pose—pause—pounce" tactic to get students to work on the example set in the book. Their tone of voice also seemed more tentative and they were more inclined to talk to their desks than to the teacher (bear in mind that I may be over-stating the case, as I reflect and reconstruct my memory).

At the end of the session, the first teacher got more questions from the students than the second (five as opposed to two, as I recall), which he fielded more confidently. The second teacher got a googly, in the form of a question about a rather obscure special case. He responded, rightly, with "I'm afraid I don't know the answer to that," but omitted to undertake to find out and report back. I speculate that part of the motivation for the googly (no relation to Google, unless it is the derivation of that brand name; it's a term from cricket where it refers to a notoriously unpredictable style of bowling) might have been "testing-out" the teacher, in a very under-stated way as might be expected in the particular situation.

Conclusions? Please comment!

On interesting new links

Thanks to an indirect recommendation from one of our librarians, I have discovered "The Irascible Professor" site: if you like my "Heterodoxy" pages, you will like this. It is US-based but its opinion pieces and polemics are not only entertaining but often thought-provoking.

More conventionally, but perhaps more solidly rewarding, those of you in adult education should be sure to bookmark Roger Hiemstra's web-site, with full on-line versions of whole books to which he has contributed, as well as a great list of links (I would say that; he has just been kind enough to add mine!)

On experience versus theory

I've just had an email from a student who teaches in a rather unusual setting. S/he is just starting our PGCE course and coping with her (OK, she is female—no great breach of confidentiality, but I can't hack all this gender-neutral stuff in practice) first major assessed work. She has been trying to reconcile her experience and practice with the theory, and it hasn't been working; she realises that she has simply been parroting (how many 't's in 'parroting'? OK, the spell-chequer accepts it!) the received wisdom in the texts, but it has not touched her practice (let alone changed it).

This was my reply, in case any other readers are in the same boat. It starts from the fact that she wrote and explained the problem [necessary commentary in italics]:

You may not realise it, but your note has actually been a really good start to your submission [our course does not have "assigned" assessments, but invites students to "submit" their ideas for credit]. And it clearly shows the strategy which will pay off when you start writing.

In short, trust your experience! If it doesn't match with the theory, that's the theory's problem, not yours. You can address the outcomes by showing how conventional approaches break down in non-standard situations like yours; you demonstrate that you are trying to link the theory and the practice, as required, but we make no assumption that they will always link up seamlessly. Not only that, but PGCE encourages a critical approach; so if you read something in Reece and Walker [one of the recommended texts] and think "chance would be a fine thing", then write about it and explain the issue.

Having said that, we have had some people working in non-standard situations who used that as a standing excuse for not drawing on established good practice; I remember one guy working in the local prison who maintained that nothing on the course applied to him. However, there had been students before (and indeed after) him working there who showed the contrary; I know, because I observed their teaching. The theory is not a matter of faith, to be accepted or rejected wholesale; it's pragmatic, and you are free to test it and apply or reject as befits your practice.

You have already immersed yourself in the outcomes [the course requirements are expressed in term of 'outcomes'; "by the time you have completed this module, you should be able to..."]. So forget them. Write about what you want to say in the first place, and then go back and check off those you have already addressed (probably more than you think). Then go back and deal with those you have not mentioned; you may be able to integrate them with your own narrative, but if you can't, it's OK to settle for second best and write about them separately.

Actually, the "submission proposal" system is intended to enable you to pick up on and discuss these issues at an early stage, and get your tutor's agreement in advance to your strategy. ["Submission Proposal" is course jargon for a learning contract] So it's a good idea to be quite bold with the proposal in future; you may have to negotiate a bit, but if your tutor signs it off, he/she is agreeing that if you deliver what you have promised, it will be awarded credit; and then you will be free of the uncertainty which has been dogging you for a couple of months.

If you trust your experience (and talk about it in class—other people may well benefit from your testing of the ideas, and have their own angles to offer) I'm sure you will find the course much more stimulating and enjoyable. After all, if we are not offering what you need, that's our problem, not yours.

Hope this helps, and all the best with the submission!

Where's the reflection? Actually, it goes back to the bookshelves I have been building. Again. I designed them and worked out the dimensions. I measured twice and cut once. All by the book. But I have learned from prior experience that when it comes to actually erecting them, in a real room, you always have to leave some margin for reality. So I have put them up with a spirit-level and a tri-square, not a ruler. I just could not measure all the bumps and troughs in the real room, so when it came to actual construction I deferred to pragmatism; I pre-cut everything for the first four bays, but the fifth (up to the opposite wall)? I built that on the basis of measurement in situ. OK, this goes back to the experience of the wardrobe a few days ago, but it also relates to my correspondent's concerns; in professional practice (not that my joinery skills aspire to professional standards; even I wouldn't pay me for it) the theory takes us so far, but no further. It is always subject to the vagaries of circumstance. The mark of a professional, however, is whether you have the tools (intellectual as well as practical) to deal with that last, idiosyncratic, bit.