Some thoughts on KS2 Progress

Caveats first: these conclusions, such as they are, are drawn from a small sample of a little over 50 schools. That sample of schools isn’t representative: indeed, it has slightly higher attainment than the national picture, both in terms of KS2 outcomes, and in KS1 starting points. However, with over 2000 pupils’ data, it shows some interesting initial patterns – particularly when comparing the three subject areas.

Firstly, on Maths – the least controversial of the three subjects. It seems that – in this sample – pupils who achieved Level 2c at KS1 had an approximately 40% chance of reaching the new expected standard (i.e. a scaled score of 100+). That leaps to around 66% for those achieving L2b at KS1 (i.e. just short of the national average)

mathslevels

The orange bar shows the average of this sample, which is slightly higher than the national average of 70%

It’s important to note, though, that progress measures will not be based on subject levels, but on the combined APS score at Key Stage 1. The graph for these comparisons follows a similar pattern, as you’d expect:

mathsaps

Where fewer than 10 pupils’ data was available for any given APS score, these have been omitted.

There is an interesting step here between pupils in this sample on APS of 13 (or less) who have a chance of 40% or less of reaching the expected standard, while those scoring 13.5 or more have a greater than 50% chance of achieving the standard. (The dip at 12.5 APS points relates to pupils who scored Level 2s in Maths and one English subject, but a level 1 in the other, highlighting the importance of good literacy for achievement in KS2 Maths)

For Reading, the graphs look broadly similar in shape

readinglevels

Blue bar shows average of this sample at 67%, which is slightly higher than national average of 66%

Interestingly here the level 2c children scorers still have only 40% chance of meeting the expected standard, but those achieving 2b have a lower chance than in maths of reaching the expected standard (58% compared to 66% for Maths).

When looking at the APS starting points, there is something of a plateau at the right-hand end of the graph. The numbers of pupils involved here are relatively few here (as few as 31 pupils in some columns). Interestingly, the dip at 18.5 APS points represents the smallest sample group shown, made up of pupils who scored 2a/2b in the two English subjects, but a Level 3 in Maths at Ks1. This will be of comfort to teachers who have been concerned about the negative effect of such patterns on progress measures: it seems likely that we will still be comparing like with like in this respect.

readingaps

It is in Writing that the differences become more notable – perhaps an artefact of the unusual use of Teacher Assessment to measure progress. Compared to just 40% of pupils attaining L2c in Reading or Maths achieving the new expected standard, some 50% of those in Writing managed to make the conversion, and this against a backdrop of teachers concerned that the expected standard was too high in English. Similarly, over 3/4 of those achieving Level 2b managed to reach the standard (cf 58% Reading, 66% Maths)

writinglevels

In contrast to the other subjects, attainment in this sample appears notably lower in Writing than the national average (at 70% compared to 74% nationally)

With the APS comparisons, there are again slight dips at certain APS points, including 18.5 and 19.5 points. In the latter case, this reflects the groups of pupils who achieved Level 3s in both Reading and Maths, but only a 2b in Writing at KS1, suggesting again that the progress measure does a good job of separating out different abilities, even using combined APS scores.

writingaps

Of course, this is all of interest (if you’re interested in such things), but the real progress measures will be based on the average score of each pupil with each KS1 APS score. I’d really like to collect some more data to try to get a more reliable estimate of those figures, so if you would be willing to contribute your school’s KS1 and KS2 data, please see my previous blog here.


Spread of data

Following a request in the comments, below, I’ve also attached a table showing the proportions of pupils achieving each scaled score for the two tests. This is now based on around 2800-2900 pupils, and again it’s important to note that this is not a representative sample.

proportions

28 thoughts on “Some thoughts on KS2 Progress

  1. Mark Williams 10 July 2016 at 8:14 pm Reply

    I’m sure you did, but have you allowed for the fact that maths will count for double in calculating ks1 apps?

    • Michael Tidd 10 July 2016 at 8:23 pm Reply

      Yes – although inadvertently counted Writing twice at first!🙂

  2. Martin Said 11 July 2016 at 10:04 pm Reply

    Hi Michael, for secondaries looking to get an indication (and I know it would only be an indication) of the spread of results, would it be possible to post a histogram showing percentages of students that achieved across the range from 80-120 for reading and maths please?

    Keep up the great work!

    • Michael Tidd 11 July 2016 at 10:18 pm Reply

      Not sure how clear a chart would be, but happy to share the data that’s drawn from my sample of around 2800-2900 pupils. Will add it to the bottom of the article.

      • Martin Said 11 July 2016 at 10:31 pm Reply

        Brilliant, thanks.

  3. oldprimarytimer 12 July 2016 at 5:42 pm Reply

    Thanks for this. Please could we also have the average scaled score for each sub level eg 2B readers on average got 99.4 in reading? I think this might be more useful than threshold measures.

  4. Amyhoppy 14 July 2016 at 9:10 pm Reply

    To repeat oldprimarytimer, please could you publish the average scaled score for each sub level that you have gathered so far? Would be really interested to see that. Thanks.

  5. Karen 19 July 2016 at 11:03 pm Reply

    New….but not so new new ‘interim’ frameworks out for 2016 – 17 as they remain unchanged ……so here we go again for yet another year. No sign of long awaited Rochford report on P levels and in the meantime we are left with interim pre Key Stage standards that don’t help sufficiently support SEN attainment and progress in being able to bridge the gap between P levels and NC expected standards.

  6. Tim 22 July 2016 at 7:24 pm Reply

    Michael. How will pupils disapplied from both KS1 and 2 SATs affect a school’s progress measures this year?

  7. Tim 22 July 2016 at 7:27 pm Reply

    ….and if calculating a class’s average scaled score, would you 1. include a disapplied child and 2. give them a score of 0 or 80?

  8. Anon 23 July 2016 at 8:20 am Reply

    Thank you for these, really useful. I have looked at the last table and come up with some rough deciles. A child getting a mark of 98 in Maths would be in the lowest 20-30% of children taking the test. A child with a score of 108 in English would be around the 80th percentile – scoring higher than about 80% of children taking the test.

    Maths English
    lowest 10% 93-94 89-90
    20% 96-97 94-95
    30% 99-100 98-99
    40% 101-102 100
    50% 103-104 102-103
    60% 104-105 104-105
    70% 106-107 106-107
    80% 108-109 108-109
    90% 112-113 112-113

  9. Matt 1 September 2016 at 2:11 pm Reply

    So today the tables checking exercise allowed me to find out my Unvalidated Progress scores at the end of KS2.
    6.2 Reading (2.8 – 9.6 confidence band)
    7.0 Writing (3.6 to 10.4)
    4.5 Maths (1.6 to 7.4)

    Still no national comparison so what is the median progress score in these data sets?
    Any idea?

    • Michael Tidd 1 September 2016 at 2:16 pm Reply

      Median progress is no longer the issue. For floor standards you need to have scores which are above -5 for Reading/Maths and above -7 for Writing. For the coasting standards, we don’t yet know the thresholds.
      Overall, the national average score is 0 for each, as that is what it’s based on (so your progress figures look very healthy!)

      • Ruth O'Reilly 3 September 2016 at 11:17 am Reply

        Any guess on where they might put the coasting standard thresholds?

  10. Matt 1 September 2016 at 3:24 pm Reply

    Does that mean that 0 is the average? Are you aware of any data that tells us what percentage of schools achieve progress scores in each band? I presume a lot will fall between -3 and +3.
    I guess this data is currently not available but it must exist.

  11. Elastictrickery 3 September 2016 at 4:11 pm Reply

    Having ‘explored’ the data in the spreadsheet from the tables checking exercise for our (small) cohort, there are some worrying anomolies created by relying on the KS1 APS as an indicator of general ability in a given subject….

    Maths, for example…

    Child 1 – KS1 R=3; W=3; M=2A – target for KS2 maths = 107.54 (and achieved 110, ‘progress’ of 2.46)

    Child 2: KS1 R=2B; W=2B; M=2A – target for KS2 maths = 103.58 (and achieved 110, ‘progress’ of 6.42).

    Problem 1: The decimal in the target score is disingenuous for a start – there are no decimal scaled scores and there are some pupils who will have missed out on their target by an unattainable decimal margin.

    Problem 2: The margin of difference in the scaled scores looks so small… there’s ‘only four points in it’ you might say…. but the difference between those two target scores in real terms was HUGE.

    A mathematics scaled score of 104 (to exceed Child 2’s decimal target) equates to a minimum of 78 points in the test whilst a score of 108 (to exceed Child 1’s decimal target) required 93 points. So child A required 15 marks more than Child B from the outset to ‘make progress’.

    Similarly, a rough estimate makes that scaled score of 104 equivalent to an old 4A, whilst 107 would have been a 5B. Their actual score of 110, by my reckoning, is a 5A in old money.

    Problem 3: Despite these two pupils having the same ‘starting point’ in KS1 and subsequently achieving the same end score in KS2 Child 1 is deemed to have made ‘more progress’. One was targeted at 6 sub-levels of progress under the old system, the other at 8 sub-levels.

    In reality, they both effectively went from a 2A to a 5A, and each made the equivalent of 9 sub-levels of progress.

    In real terms, they both did equally brilliantly. The progress data suggests that one didn’t do quite so well.

    To compound this issue, another child who got a 2A in maths at KS1 wasn’t quite as capable at reading and writing. As a result his APS was lower and his maths target was just 101.5 (equivalent to a minimum of 70 marks)

    He attained 105, giving him a progress measure of 3.5. Effectively, despite ‘only’ attaining the equivalent to a 4A/5C borderline, he is deemed to have made more progress than a child who went from 2A to 5A!

    To recap, here’s the implied ‘target conversation’:

    “Well children, you all did a little bit better than average at KS1 in maths this year. We’re going to set you progress targets for Year 6 based on your levels of 2A. Child 1 – you have to score at least 78 marks. Child 2 – you need 93. Child 3 – you’re good at maths but can’t read and write very well – you just try to get 70 marks in maths in Year 6, good lad…’

    And the ‘system’ for calculating progress in writing?

    HAH!

  12. Michael Tidd 3 September 2016 at 4:23 pm Reply

    Although all these things look like nonsense, I do think that the new measure is better than it first appears. Instinctively we want to compare children who had the same level in a given subject at KS1 (as your 2a maths children), but actually the DfE stats show that reading & writing ability at KS1 *do* have an impact on attainment at KS2, and that as a result, the estimated outcomes should be affected by them.
    They’ve presumably crunched the numbers to conclude that the combined figures are fairer overall. It will never be perfect, but just because the system *looks* more unfair, doesn’t necessarily mean that it is.

    • Elastictrickery 3 September 2016 at 5:16 pm Reply

      Once the current year 3’s reach year six and are being compared against the same numerical scoring system, it’ll probably work ‘better’. But the current variation in targets from the ‘same’ starting point isn’t ….

      It’s hard to see just how wide that variation is with just 11 kids, but comparing those in the same ‘KS1 Attainment band’ which is subsequently converted to a ‘prior attainment band for progress measure’ shows that this is primarily allocated according to the APS score.

      As such it disregards the actual levels attained in each subject, as various configurations will generate the same APS.

      Example:

      We have two children in ‘KS 1 band 2’ which becomes ‘progress group 14’.

      Their KS1 results were:

      Child A – R-2A; W-2C; M: 2A
      Child B – R – 2B; W -2B ; M-2A

      Both had a KS1 APS of 16. As far as I can tell, it is this alone that dictates which band they go into and which generates their targets.

      Despite the differences in their starting levels, their KS2 targets are identical for each subject.

      There could easily have been a child who had R-3 W-2B M-2C which would also have been an APS of 16 and who would have had the same KS2 targets.

      In maths that would have meant converting their 2C to an equivalent of 4A… against a backdrop of increasingly difficult written calculations as well as ‘fractions, decimals and percentages’.

      That conversion isn’t entirely beyond the realms of possibility, but it’s fairly challenging: whilst for child A and B, it would be relatively simple.

      It just doesn’t sit right…

      • Michael Tidd 3 September 2016 at 5:32 pm Reply

        Actually, a child with Reading L3, Writing L2b and Maths L2c would have had an APS of 15.5 so been in a lower PA group.
        As I say, it’s not perfect, but I’m not persuaded that it’s any worse than the old system.

        • Matt 4 September 2016 at 7:11 pm

          I’m not sure it will work better when current year 3s get into year 6. Our current year sevens could have received a 3, 2A, 2B, 2C or a 1 at Ks2. (Mainly) but his years year two children were predominantly split on GDS EXS OR WTS. (Again mainly) so there may be a huge ability difference between child A who achieved EXS and child B who achieved the same grade. Although schools generated a scaled scores on year 2 Sats these were not reported.
          I wonder whether this will mean a push toward reporting test scores as scaled scores at KS1 rather than teacher assessment.

        • Michael Tidd 4 September 2016 at 7:32 pm

          I suspect that collecting scaled scores at KS1 is very much on the cards.
          As you rightly say, the teacher assessment data is fairly meaningless, so I suspect 2020 is another cock-up in the making!

  13. Elastictrickery 3 September 2016 at 4:43 pm Reply

    Actually, didn’t STA ask for suggested test questions last month?

    This makes a great one:

    In KS1, three boys took a maths test and got the same grade. Four years later they took another test.

    Jimmy scored 78 points. Billy scored 98 points. Timmy scored 99.

    Afterwards, the teacher said that Timmy had made the LEAST progress and Jimmy made the MOST.

    Explain….

  14. Irish Eileen 11 September 2016 at 11:24 am Reply

    Just wondered how secondary schools are going to use this data to set targets? any ideas???

    • Michael Tidd 11 September 2016 at 12:01 pm Reply

      Not really. They’ll only need to take Reading and Maths scores into account for the purposes of Progress 8, but like in primary, there no longer an easy measure like the old “three levels of progress”

  15. Irish Eileen 11 September 2016 at 5:36 pm Reply

    Thanks for this, am intrigued as to what other secondary schools are going to use to show progress from Year 7 to 11 and how they will track it.

  16. Sandor Beagle 11 September 2016 at 9:01 pm Reply

    what about children working below the level of the tests who come from P Scales. it appears that those coming from PAG 2 cannot achieve a positive progress score unless they get a minimum score of 84 points on the tests. the Pre key stage always gives them a negative score

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: