Category Archives: primary

What is a “particular weakness” anyway?

In DfE terms, it’s early days for being able to make decisions about KS2 Writing outcomes. After all, it wasn’t so long ago that we were reaching February without any exemplification at all, so for the STA to have released its “particular weakness” scenarios as early as mid-January is progress!

However, publishing the materials is one thing. Providing the clarity that a high stakes statutory assessment process dearly needs is quite another. The example scenarios offer some insight into the thinking at the STA about this new ‘flexibility’, but seem to have deliberately skirted round the key issues that keep coming up, such as dyslexia!

In an effort to get a sense of the interpretations out there, I put together some very brief scenarios of my own, and asked Y6 teachers to say whether or not they thought such pupils would be awarded the expected standard. And as I feared, there is a real lack of clarity about. The six example scenarios follow, accompanied by the pie charts showing decisions. In each case, the blue represents those who would award EXS (based on a sample of 668 responses)

Scenario 1

graph1

77% award EXS

Edith has shown herself to be a fluent and confident writer. She adapts her writing for a variety of purposes, and in many cases has evidence of elements of working at Greater Depth. However, there are no examples of the passive voice used in any of her writing, except through planned tasks.

Scenario 2

graph2

67% award EXS

Beowulf is a good writer, who meets almost all of the requirements for EXS. However, he has been identified as being at high risk of dyslexia. In his writing he has shown that he can use some of the Y5/6 words accurately. However, he struggles with some of the regular spelling patterns from the curriculum, and his work contains several errors, particularly for the more complex patterns.

Scenario 3

graph3

36% award EXS

Ethelred writes effectively for a range of audiences and purposes, with sound grammatical accuracy. He uses inverted commas correctly to mark speech, but does not yet consistently include punctuation within the inverted commas.

Scenario 4

graph4

71% award EXS

Boudicca writes well, showing an interesting range of language, sentence type and punctuation. However, she has developed a largely un-joined style of writing, which although clearly legible does not include the usual diagonal or horizontal strokes.

Scenario 5

graph5

55% award EXS

Cleopatra is a confident writer, who shows good grasp of technical aspects and a beautiful joined style of writing. She enjoys writing fiction and can develop good plot, with writing that flows well. However, in non-fiction texts she is not always able to use the cohesive devices that enable cohesion between paragraphs. There are some examples of stock phrases used (On the other hand, Another reason, etc.) when writing in a formal style, but these are not consistent across the non-fiction texts she writes

Scenario 6

 

graph6

92% award EXS

Englebert is a technically sound writer. He is able to adapt writing for fiction and non-fiction purposes and uses a variety of language and punctuation techniques. His spelling of common patterns is generally good. However, there are a number of examples of words from the Y5/6 lists which are mis-spelt in his writing generally. His teacher has shown that he could spell these words correctly when tested in the context of dictated sentences throughout the year.

 

Notably, all but one of the results were within 5 percentage points of the figures above when looking only at those who said they had had some training provided on this topic. The biggest difference came for scenario 4 (handwriting) where only 61% of those who said they’d been trained would award EXS compared to 71% of the full sample.

 

It’s hard to say what I expected when I set up these little scenarios. I certainly don’t know what any “correct” responses might be. I think I imagined that some would be fairly evenly split – as with the case of Cleopatra’s weak use of cohesive devices.

Scenario 6 has genuinely surprised me. I don’t know what a moderator would say, but my fear about dictated sentences would be that children could easily be tested on a handful of words each week, learned for Friday’s test, and then quickly forgotten. Is that sufficient to say they can spell at the Expected Standard? Who knows? (That’s not to say that I think ‘no’ is the correct answer either; I’m not persuaded that the importance of spelling those particular words is as great as the system might suggest).

I’m equally surprised at scenario 3. Is it really right that speech punctuation is so so important that 2/3 of teachers would deny a pupil an EXS judgement on this alone – even when so many are happy to overlook spelling or handwriting failures?

As I say – I don’t have any answers. If any moderator – or perhaps an STA representative would like to give a definitive response, I’d be glad of it. I suspect that as close as we’d get to an official answer is that a moderator would have more evidence upon which to make a decision. Which is all well and good. For the 3-4% of pupils whose work gets moderated. For everyone else, we have to hope that teachers have got it right. And judging by these results, that’s not that easy!

Advertisements

Writing Moderation materials

Just a quick post to share the moderation support materials that were shared by the STA today. For some reason, they have only been shared via the password-protected NCA Tools website. However, there is no indication that they should be maintained under any conditions of secrecy, and no indication that they are not covered by the usual Crown Copyright rules… so here they are:

KS1_standardisation_training_presentation_1

KS1_teacher_assessment_moderation_training_pack_1

KS2_standardisation_training_presentation_1

KS2_teacher_assessment_moderation_training_pack_1

The presentations include clarifications about some of the criteria included in the assessment frameworks.

The moderation training packs include the examples that are meant to help illustrate what counts as an exception when you want to overlook one of the criteria.

See if you find it at all helpful…

National Curriculum test videos

With the introduction of the new style National Curriculum tests in 2016, I made some short informative videos for parents about each set of tests. Since then, I’ve updated then each year to reflect changes such as this year’s timetable changes at KS2. The videos last around 5 minutes and are ideal for sharing on school websites, twitter feeds, facebook pages, etc.

To help schools use them most effectively, I have provided links below in each of the main formats so they can easily be shared. Please feel free to share or download the videos and use them for your school:

Key Stage 2 tests

youtube facebookicon Twit mp4icon
YouTube Facebook Twitter MP4 download

Key Stage 1 tests – including Grammar, Punctuation & Spelling

 

youtube facebookicon Twit mp4icon
YouTube Facebook Twitter MP4 download

Key Stage 1 tests – without GPS

youtube facebookicon Twit mp4icon
YouTube Facebook Twitter MP4 download

Primary Assessment changes… again!

First of all, let me say that I’m pleased that primary assessment is changing again, because it’s been a disaster in so many ways. So here is a summary of the changes at each key stage – with my thoughts about each.

Early Years Foundation Stage Profile

  • The EYFS Profile will stay, but will be updated to bring it into line with the new national curriculum and take account of current knowledge & research. I’ve never been a huge fan of the profile, but I know most EY practitioners have been, so that seems a sensible move.
  • A proposed change to reduce the number of reported Early Learning Goals to focus on prime areas and Literacy/Maths
  • The ’emerging’ band may be divided to offer greater clarity of information particularly for lower-attaining pupils.
  • An advisory panel will be set up to advise on changes to the profile and ELGs. Membership of that could be contentious

Reception Baseline

  • New Reception baseline to be introduced from 2020 (with proper trialling beforehand this time, one presumes!) to take place in the first 6 weeks of school.
  • Won’t be a ‘test’, but also won’t be observational over time. Suspect something more like the current CEM model, perhaps?
  • Will focus on literacy & numeracy, and potentially a ‘self-regulation’ element, as good predictors for attainment in KS2
  • Data won’t be used for any judgements about Reception, but will be used at cohort level to judge progress by the end of KS2.
  • The intention is for the assessment to provide some narrative formative information about children’s next steps.

Key Stage 1

  • The KS1 Grammar, Punctuation & Spelling test will remain optional.
  • Statutory Assessment will remain until at least 2023 (to allow for a year of overlap with the first cohort to be assessed using Reception baseline).
  • A new framework for Teacher Assessment of Writing has been published for this year only. Exemplification will follow this term.
  • DfE will continue to make assessments available (perhaps through an assessment bank if that ever gets off the ground!) after 2023, to help schools to benchmark attainment.
  • After 2023, tests and statutory teacher assessment will become optional for through primary schools.
  • There is more work to be done to find a system which works well for infant/junior and first/middle schools. This will be done with those in the sectors.

Key Stage 2

  • A multiplication check will be introduced at the end of Year 4. (Although, of course, whether the end means July or May remains to be seen).
  • School-level data on the multiplication check won’t be published.
  • This will be the last year that teachers have to make Teacher Assessment judgements for Reading and Maths
  • A new framework for Teacher Assessment of Writing has been published for this year only. Exemplification will follow this term.
  • DfE will continue to evaluate other options for the future, but not really committing to anything yet.
  • Small trials of peer-to-peer moderation will take place this summer.
  • Science Teacher Assessment frameworks will be updated next year.
  • The Reading test will not be timetabled for Monday of SATs week any more (hurrah!)
  • The DfE aims to link the reading content of the tests more closely to the curriculum to ensure children are drawing on their knowledge.

My thoughts

Overall, I’m pleased. Most of these changes are to be welcomed. The Reception baseline is a sensible idea (just a shame it was so badly implemented the first time round), as is scrapping KS1 assessments. The Early Years changes seem reasonable given the popularity of the current setup. The improvements to the KS2 Reading test are positive, as is the removal of pointless Teacher Assessment judgements.

On Writing, I fear we haven’t gone far enough. The current system is a joke, and it seems like the interim solution we’ll have to replace the old interim solution will just aim to make it less awful without really fixing the problem. It’s a shame that there is no obvious answer on the horizon. Perhaps the department has had its fingers burnt by rushing into quick fixes in the past and is prepared to bide its time.

In the interim, the updated expectations for Writing seem more manageable both in terms of achieving and assessing them. Of course, the devil is in the detail. If we get another exemplification book that breaks down single statements into several tick-boxes then we may be back at square one. Equally, of course, we can expect proportions of pupils meeting the expected standard to rise again substantially this year. Surely we have to be honest now and say that we really cannot use this data for accountability purposes? Mind you, perhaps it won’t matter – if we’re all getting 90% in Writing, it’ll only be the tested subjects that will make a difference to the accountability!

There are some other changes I would have liked to have seen. I really don’t think the “expected standard” label is helpful, particularly in subjects where scaled scores are used; it’s a shame we’ve not seen the back of that.

We’re not out of the woods yet. But we’re heading in the right direction, and credit is due to those at the department for listening. Let’s just hope they keep listening until we all get it right.

Stop moaning about tests!

Today marked the end of 4 short days of testing. For Year 6 pupils everywhere, they’ll have spent less than 5 hours on tests – probably not for the first time this year – and later in the year we’ll find out how they did.

Now, I’m the first to complain when assessment isn’t working, and there are lots of problems with KS2 assessment. Statutory Teacher Assessment is a joke; the stakes for schools – and especially headteachers – are ridiculously high; the grammar test is unnecessary for accountability and unnecessarily prescriptive. I certainly couldn’t be seen as an apologist for the DfE. And yet…

For some reason it appears that many primary teachers (particularly in Facebook groups, it seems) are cross that some of the tests contained hard questions. I’ve genuinely seen someone complain that their low-ability children can’t reach the expected standard. Surely that’s the very reason they’re defining them as low ability?

Plenty of people seem annoyed that some of the questions on the maths test were very challenging. Except, of course, we know that some children will score 100% each year, so the level of challenge seems fair. There were also plenty of easier, more accessible questions that allowed those less confident mathematicians to show what they can do. It’s worth remembering that to reach the expected standard last year, just 55% of marks were needed.

But the thing that annoys me most is the number of people seemingly complaining that the contexts for problem-solving questions make the questions too difficult. Of course they do, that’s the point: real maths doesn’t come in lists of questions on a page that follow a straightforward pattern. What makes it all the more irritating is that many of those bemoaning the contexts of problems are exactly the same sort who moan about a tables test, complaining that knowing facts isn’t worthwhile unless you can apply them.

Well guess what: kids need both. Arithmetic knowledge and skills need to be secure to allow children to focus their energies on tackling those more complex mathematical problems. You can’t campaign against the former, and then complain about the latter.

The tests need to – as much as possible – allow children across the ability range to demonstrate their skill, while differentiating between those who are more and less confident. That’s where last year’s reading test fell down: too few accessible elements and too many which almost no children could access. This year’s tests were fair and did a broadly good job of catering for that spread. For those complaining about the level of literacy required, it’s worth remembering that questions can be read to children, and indeed many will have had a 1:1 reader throughout.

No test will be perfect, and there are plenty of reasons to be aggrieved about the chaos that is primary assessment at the moment, but blaming tests because not all children can answer all questions is a nonsense, and we’d do well to pick our battles more carefully!

KS2 Writing: Moderated & Unmoderated Results

After the chaos of last year’s writing assessment arrangements, there have been many questions hanging over the results, one of which has been the difference between the results of schools which had their judgements moderated, and those which did not.

When the question was first raised, I was doubtful that it would show much difference. Indeed, back in July when questioned about it, I said as much:

At the time, I was of the view that LAs each trained teachers in their own authorities about how to apply the interim frameworks, and so most teachers within an LA would be working to the same expectations. As a result, while variations between LAs were to be expected (and clearly emerged), the variation within each authority should be less.

At a national level, it seems that the difference is relatively small. Having submitted Freedom of Information Requests to 151 Local Authorities in England, I now have responses from all but one of them. Among those results, the differences are around 3-4 percentage points:

moderated

Now, these results are not negligible, but it is worth bearing in mind that Local Authorities deliberately select schools for moderation based on their knowledge of them, so it may be reasonable to presume that a larger number of lower-attaining schools might form part of the moderated group.

The detail that has surprised me is the variation between authorities in the consistency of their results. Some Local Authority areas have substantial differences between the moderated and unmoderated schools. As Helen Ward has reported in her TES article this week, the large majority of authorities have results which were lower in moderated schools. Indeed, in 11 authorities, the difference is 10 or more percentage points for pupils working at the Expected Standard. By contrast, in a small number, it seems that moderated schools have ended up with higher results than their unmoderated neighbours.

What can we learn from this? Probably not a great deal that we didn’t already know. It’s hard to blame the Local Authorities: they can’t be responsible for the judgements made in schools they haven’t visited, and nor is it their fault that we were all left with such an unclear and unhelpful assessment system. All this data highlights is the chaos we all suffered – and may well suffer again in 2017.

To see how your Local Authority results compare, view the full table* of data here. It shows the proportions of pupils across the LA who were judged as working at the Expected and Greater Depth Standards in both moderated and unmoderated schools.


*Liverpool local authority claimed a right not to release their data on the grounds of commercial sensitivity, which I am appealing. I fully expect this to be released in due course and for it to be added here.

Some clarity on KS2 Writing moderation … but not a lot

Not for the first time, the Department has decided to issue some clarification about the writing assessment framework at Key Stage 2 (and its moderation!). For some inexplicable reason, rather than sharing this clarity in writing, it has been produced as a slowly-worded video – as if it were us that were stupid!

Here’s my take on what it says:

Some Clarity – especially on punctuation

  • For Greater Depth, the long-winded bullet point about shifts in formality has to be seen in several pieces of work, with more than one shift within each of those pieces.
  • For Expected Standard, it is acceptable to have evidence of colons and semi-colons for introducing, and within, lists (i.e. not between clauses)
  • For Expected Standard, any of either brackets, dashes or commas are acceptable to show parenthesis. There is no need to show all three.
  • Bullet points are punctuation, but the DfE is pretending they’re not, so there’s no need to have evidence of them as part of the “full range” of punctuation needed for Greater Depth.
  • Three full stops to mark ellipsis are also punctuation, but again, the DfE has managed to redefine ellipsis in such a way that they’re not… so again, not needed for Greater Depth.

A bit of guidance on spelling

This was quite clear: if a teacher indicates that a spelling needs correcting by writing a comment in the margin on the relevant line, then the correction of that spelling cannot be counted as independent. If the comment to correct spellings comes at the end of a paragraph or whole piece, without specifying what to correct, then it can still count as independent.

No clarity whatsoever on ‘independence’

Believe me, I’ve re-watched this several times – and not all of them at double-speed – and I’m still bemused that they think this clarifies things. The whole debacle is still reliant on phrases like “over-scaffolding” and “over-detailed”. Of course, if things are over-detailed then there is too much detail. What isn’t any clearer is how much detail is too much detail. The video tells us that:

“success criteria would be considered over-detailed where the advice given directly shapes what pupils write by directing them to include specific words or phrases”

So we know specifying particular words is too much, but is it okay to use success criteria which include:

  • Use a varied range of sentence structures

Is it too specific to include this?

  • Use a varied range of sentence openers

What about…?

  • Use adverbs as sentence openers

There’s a wide gulf between the three examples above. Which of these is acceptable? Because if it’s the latter, then schools relying on the first will find themselves under-valuing work – and vice versa, of course. That’s before you even begin to consider the impossibility of telling what success criteria and other supporting examples are available in classrooms at the time of writing.

The video tries to help by adding:

“success criteria must not specifically direct pupils as to what to include or where to include something in their writing”

But all of those examples are telling children what to include – that’s the whole point of success criteria.

If I’ve understood correctly, I think all three of those examples are acceptable. But it shouldn’t matter what I think: if the whole system depends on what each of us thinks the guidance means, then the consistency necessary for fair and useful assessment is non-existent.

The whole issue remains a farce. Doubtless this year Writing results will rise, probably pushing them even higher above the results for the externally tested subjects. Doubtless results will vary widely across the country, with little or no relationship to success in the tested subjects. And doubtless moderation will be a haphazard affair with professionals doing their best to work within an incomprehensible framework.

And to think that people will lose their jobs over data that results from this nonsense!


The full video in all its 11-minute glory can be found at: https://www.youtube.com/watch?v=BQ-73l71hqQ

 

National Curriculum Test videos

I’ve updated the videos I made last year to explain the KS1 and KS2 tests to parents. As there is an option about using the Grammar, Punctuation & Spelling tests in primary schools, there are now two versions of the video for KS1 (one with, one without the GPS tests).

Please feel free to use these videos on your school’s website or social media channels, or in parent meetings, etc. There are MP4 versions available to download.

Key Stage 2

Facebook shareable version:

Re-tweetable version:

Downloadable MP4 file: https://goo.gl/B1fwiP

Key Stage 1 – version that includes the GPS tests

Downloadable MP4 file: https://goo.gl/jo18qk

Facebook shareable version:

Key Stage 1 – version for schools not using the GPS tests

Re-tweetable version:

Facebook shareable version:

Downloadable MP4 file:  https://goo.gl/xMDFSJ

The impossibility of Teacher Assessment

I’ve said for a fair while now that I’d like to see the end of statutory Teacher Assessment. It’s becoming a less unpopular thing to say, but I still don’t think it’s quite reached the point of popularity yet. But let me try, once again, to persuade you.

The current focus of my ire is the KS2 Writing assessment, partly because it’s the one I am most directly involved in (doing as a teacher, not designing the monstrosity!), and partly because it is the one with the highest stakes. But the issues are the same at KS1.

Firstly, let me be frank about this year’s KS2 Writing results: they’re nonsense! Almost to a man we all agreed last year that the expectations were too high; that the threshold was something closer to a Level 5 than a 4b; that the requirements for excessive grammatical features would lead to a negative impact on the quality of writing. And then somehow we ended up with 74% of children at the expected standard, more than in any other subject. It’s poppycock.

Some of that will be a result of intensive drilling, which won’t have improved writing that much. Some of it will be a result of a poor understanding of the frameworks, or accidental misuse of them. Some of it will be because of cheating. The real worry is that we hardly know which is which. And guidance released this year which is meant to make things clearer barely helps.

I carried out a poll over the last week asking people to consider various sets of success criteria and to decide whether they would be permitted under the new rules which state that

independent

So we need to decide what constitutes “over-aiding” pupils. At either end of the scale, that seems quite simple.Just short of 90% of responses (of 824) said that the following broad guidance would be fine:

1.png

Simplest criteria

Similarly, at the other extreme, 92% felt that the following ‘slow-writing’ type model would not fit within the definition of ‘independent’:

8

Slow writing approach

This is all very well, but in reality, few of us would use such criteria for assessed work. The grey area in the middle is where it becomes problematic. Take the following example:

5

The disputed middle ground

In this case results are a long way from agreement. 45% of responses said that it would be acceptable, 55% not. If half of schools eschew this level of detail and it is actually permitted, then their outcomes are bound to suffer. By contrast, if nearly half use it but it ought not be allowed, then perhaps their results will be inflated. Of course, a quarter of those schools maybe moderated which could lead to even those schools with over-generous interpretations of the rules suffering. There is no consistency here at all.

The STA will do their best to temper these issues, but I really think they are insurmountable. At last week’s Rising Stars conference on the tests, John McRoberts of the STA was quoted as explaining where the line should be drawn:

That advice does appear to clarify things (such that it seems the 45% were probably right in the example above), but it is far from solving the problem. For the guidance is full of such vague statements. It’s clear that I ought not to be telling children to use the word “anxiously”, but is it okay to tell them to open with an adverb while also having a display on the wall listing appropriate adverbs – including anxiously? After all, the guidance does say that:

guidance.png

Would that count as independent? What if my classroom display contained useful phrases for opening sentences for the particular genre we were writing? Would that still be independent?

The same problems apply in many contexts. For spelling children are meant to be able to spell words from the Y5/6 list. Is it still okay if they have the list permanently printed on their desks? If they’re trained to use the words in every piece?

What about peer-editing, which is also permitted? Is it okay if I send my brightest speller around the room to edit children’s work with them. Is that ‘independent’?

For an assessment to be a fair comparison of pupils across the country, the conditions under which work is produced must be as close to identical as possible, yet this is clearly impossible in this case.

Moderation isn’t a solution

The temptation is to say that Teacher Assessment can be robust if combined with moderation. But again, the flaws are too obvious. For a start, the cost of moderating all schools is likely to be prohibitive. But even if it were possible, it’s clear that a moderator cannot tell everything about how a piece of work was produced. Of course moderators will be able to see if all pupils use the same structure or sentence openers. But they won’t know what was on my classroom displays while the children were writing the work. They won’t know how much time was spent on peer-editing work before it made the final book version. They won’t be able to see whether or not teachers have pointed out the need for corrections, or whether each child had been given their own key phrases to learn by heart. Moderation is only any good at comparing judgements of the work in front of you, not of the conditions in which it was produced.

That’s not to imply that cheating is widespread. Far from it: I’ve already demonstrated that a good proportion of people will be wrong in their interpretations of the guidance in good faith. The system is almost impossible to be any other way.

The stakes are too high now. Too much rests on those few precious numbers. And while in an ideal world that wouldn’t be the case, we cannot expect teachers to provide accurate, meaningful and fair comparisons, while also judging them and their schools on the numbers they produce in the process.

Surely it’s madness to think otherwise?


For the results of all eight samples of success criteria, see this document.

 

You’re not still teaching that are you?

This has become something of a recurring refrain over my teaching career, and it always – always – frustrates me.

Nobody ever says it about Science: “Oh, you’re not still teaching solids, liquids and gases, are you?”. Or music: “Oh, you’re not still teaching standard notation, are you?” And yet for some reason it seems to abound in other areas – especially English.(Even maths seemed to go through a phase where the standard basics were frowned upon!) But such decisions are often distinctly personal.

The first time I read Holes by Louis Sachar, I couldn’t wait to get planning for it, and was desperate to start teaching it. Now, having taught it too many times for my own liking, I’m tired of it. I suspect that this will be my last year of tackling it because I’ve lost my love for it. But for my class this year, it was their first time of approaching it. It was fresh for them. The only reason to abandon it is that my waning love for it risks coming through in the teaching.

But that won’t stop somebody somewhere from saying “Oh, but you’re not still teaching Holes, are you?”

It happens too often.

Tonight I’ve seen the same said of both The Highwayman and the animation The Piano. Now for sure they’ve both had more than their fair share of glory, but there was a reason why they were chosen in the first place. I’m all in favour of people moving away from them, finding better alternatives, mixing things up a bit. But they don’t cease to be excellent texts just because they’ve been done before. Every Year 5 child who comes to them does so for the first time.

I’ve heard the same said before of The Lighthouse Keeper’s Lunch at KS1 -as though somehow the fact that a topic has worked brilliantly in the past should be ignored simply because a consultant is over-familiar with it.

Of course, there are reasons to ditch texts. Sometimes they become outdated. Sometimes they cease to match the curriculum. Sometimes the ability of the children demands more stretch. Sometimes something much better comes along. Sometimes you’re just sick of them.

I’ve never cared for Street Child even though it’s wildly popular. I’ve always found Morpurgo’s work irritating. But if others find them thrilling, and get great results with their classes, then so be it. Who am I to prevent them teaching them?

As somebody also responded on Twitter this evening: the best “hook” is the teacher. If a teacher feels passionately about a poem, a book, or a topic, then it can be a great vehicle for the teaching that surrounds it. And if we make them all ditch those popular classics merely because they’re popular, then you’d better have a damned good replacement lined up to offer them!