Chapter 2

National Assessment Program – Literacy and Numeracy (NAPLAN)

2.1        This chapter covers the background to NAPLAN, its purpose, uses of the results data and the issues raised during the inquiry regarding the administration of national standardised literacy and numeracy testing.

Background

2.2        Prior to the first NAPLAN tests in May 2008, each Australian state and territory managed its own literacy and numeracy testing regime, commencing in 1989 with the Basic Skills Test in New South Wales. Despite differences in the state and territory regimes, national comparative data was prepared on an annual basis from 1999 through a process called ‘equating'.F[1]F A similar process is still used today to enable comparisons between tests done in different years.F[2]F

2.3        In July 2003, the Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) agreed to work towards enhanced collection, consistency and reporting of literacy and numeracy performance data. The Schools Assistance (Learning Together – Achievement through Choice and Opportunity) Act 2004 prescribed the implementation of national tests by 1 January 2008. In July 2006, MCEETYA agreed that national literacy and numeracy testing for all students in years 3, 5, 7, and 9 would commence in 2008. The NAPLAN tests began to be administered across all states and territories with support from all education ministers.F[3]

What is NAPLAN?

2.4        The National Assessment Program – Literacy and Numeracy (NAPLAN) commenced in Australian schools in 2008. Each year, all students in years 3, 5, 7 and 9 are assessed on the same days using standardised national tests in Reading, Writing, Language Conventions (Spelling, Grammar and Punctuation) and Numeracy.F[4]

2.5        Parents receive individual reports for their children, showing how each child performed compared to the national average, and, in some states and territories, compared to the school average. Schools are provided with detailed student results at the same time as, or sometimes before, parents, depending on the timing of school holidays in individual states and territories. After that:

The results are released to the public in two stages. The first stage is the NAPLAN Summary Report, released in mid September, showing results at each year level and domain by state and territory and nationally. The second stage is the NAPLAN National Report that includes detailed results by sex, Indigenous status, language background other than English status, parental occupation, parental education, and geolocation (metropolitan, provincial, remote and very remote). The National Report is released at the end of the year of testing.F[5]

2.6        Testing at the national level is not a new concept and is conducted in countries such as Belgium, Canada, France, Germany, the Netherlands, South Korea, Sweden, the United Kingdom and the United States among others.F[6]F Methods, of course, vary. In Canada, for example, the Pan-Canadian Assessment Program tests sample student groups in reading, mathematics and science literacy. Results are used by jurisdictions to validate data from their own, separate jurisdiction-level assessments, as well as Canada's results in Programme for International Student Assessment (PISA) tests.F[7]

2.7        A report on assessment systems published by the United Kingdom Qualifications and Curriculum Development Agency in 2007 provides details on a number of national assessment systems, such of that of France, where full student cohort literacy and numeracy testing is in place for Year 3 and Year 6 students and results are published by the Ministry of National Education's Division for Assessment, Evaluation, Potential and Performance.F[8]

2.8        The committee majority notes the Australian Primary Principals Association's position on NAPLAN as being only one element of the education system:

...NAPLAN is only one source of information about student achievement and the primary curriculum is designed to promote the social and emotional development of children as well as their academic attainment across all areas of the curriculum.F[9]

Who manages NAPLAN?

2.9        Following agreement at a Council of Australian Governments (COAG) meeting in October 2008, a national education authority, the Australian Curriculum, Assessment and Reporting Authority (ACARA), was established in December 2008 by the Australian Curriculum, Assessment and Reporting Authority Act 2008. One of the authority's key tasks is to assess the literacy and numeracy capabilities of the student population, and to this end ACARA manages NAPLAN. It should be noted that NAPLAN is just one aspect of the broader National Assessment Program (NAP).F[10]

How are the tests developed?

2.10      ACARA explained that the process for developing the tests is comprehensive and involves input from experts providing services under contract, supported by expert review and state and territory officials. The process takes around 12 months and has five phases: test development; administration; marking; analysis; and reporting of results.F[11]

What is the purpose of NAPLAN?

2.11      ACARA notes that the main purpose of NAPLAN testing is:

to identify whether all students have the literacy and numeracy skills and knowledge that provide the critical foundation for other learning and for their productive and rewarding participation in the community.F[12]

2.12      The first step towards improvement is the identification of areas of need. As succinctly put in a highly regarded report on the world's top-performing school systems by McKinsey & Company:

All of the top-performing systems...recognise that they cannot improve what they do not measure.F[13]F

2.13      A large number of submissions considered NAPLAN tests to be useful diagnostic tools.F[14]F Others described NAPLAN testing as '...an important advance in addressing poor performance'.F[15]F The Association of Heads of Independent Schools of Australia (AHISA) stated that NAPLAN:

...can help in the development of targeted programs for the professional development of teachers and school improvement...[and]...has unique value in that it provides state/territory and national data that allows principals a broad brush comparative benchmarking of student achievement.F[16]

2.14      The ACT Council of Parents and Citizens Associations echoed these views:

Council strongly supports national NAPLAN testing which provides parents with an additional resource on the progress of their child's education and has the opportunity to strengthen partnerships in learning between parents, teachers and schools.F[17]F

2.15      The Independent Education Union of Australia submitted that:

...the primary purpose of assessment and reporting is to provide meaningful information so as to improve student learning. The reporting process must be an integral part of the teaching and learning process.F[18]

2.16      The committee majority also notes the submissions which did not consider NAPLAN tests to be necessary or beneficial, but instead, for example, found them to be a '...low-cost, broad-brush, rough assessment guide, not a fine-grained diagnostic tool'.F[19]

How test results are used

2.17      ACARA noted that uses of NAPLAN data include:

• Students and parents may use individual results to discuss achievements and progress with teachers.
• Teachers use results to help them better identify students who require greater challenges or additional support.
• Schools use results to identify strengths and weaknesses in teaching programs and to set goals in literacy and numeracy.
• School systems use results to review programs and support offered to schools.F[20]

2.18      The Australian Council for Educational Research (ACER) identified the following uses for the data:

• to identify individuals who are not meeting minimum literacy and/or numeracy expectations for their year level;
• to identify, at a school level, areas of the curriculum in need of further teaching and emphasis;
• to monitor the performance and progress of social inclusion priority groups such as Indigenous students;
• to set targets for improvement at school, regional, state or national levels; and
• to monitor changes in literacy and numeracy standards over time.F[21]

2.19      The committee majority notes that NAPLAN data enable a direct comparison of results from one year to the next. NAPLAN helps schools identify successful programs and identify areas in need of improvement. Importantly, it assists education systems and governments to identify schools performing well or poorlyF[22]F which informs the allocation of resources.

2.20      State and territory governments have welcomed NAPLAN testing and confirmed that results are used:

...for system level reporting, for school accountability and for strategic planning. This data has enabled jurisdictions to target their support in resourcing schools and students with the greatest need... Moreover, the real power of the data derived from NAPLAN testing is through jurisdictional analysis whereby schools, and individual teachers, have access to a thorough diagnostic analysis of the performance of each student on each test, provided on a question by question basis.F[23]

2.21      The government of South Australia reported that results also help education departments apply measures which assist in developing intervention plans for students who do not meet minimum standards.F[24]

2.22      Mrs Sharyn Lidster, Acting General Manager, Strategic Policy and Performance, Department of Education, Tasmania, explained how jurisdictions use test data collected:

The NAPLAN tests are used extensively to support schools. Our jurisdiction, and others, provides the information back to schools. A lot of analysis is done that supports the schools. Workshops are run to help the schools to interpret the results and use them effectively to support their teachers. We also conduct workshops to assist senior people within schools to interpret the information and, where appropriate, we provide additional support for schools in relation to teacher development. Also, funding is provided to support the additional programs to improve the outcomes for students, where they are identified as performing below where we would expect them to be.

In Tasmania we also use the NAPLAN results to link to the teachers' assessments... We link the results of NAPLAN to those assessments and we provide the information back to the teachers to give them an objective piece of information that says that your assessments are consistent with the students' performance on the actual national tests.F[25]

2.23      In addition, the data can be used by parents and caregivers to make informed decisions about the education of their child. Standardised tests allow parents to see how their children are performing compared to the national average, and are supported as an additional resource enabling parents to measure their children's educational progress.F[26]F As noted by Professor Geoff Masters et al:

Parents and caregivers require valid and reliable information to evaluate the quality of the education the children are receiving, to make informed decisions in the best interests of individual students, and to become active partners in their children's learning. They require dependable information about the progress individuals have made) the knowledge, skills and understandings developed through instruction), about teachers' plans for future learning, and about what they can do to assist. There is also considerable evidence that parents and caregivers want information about how their children are performing in comparison with other children of the same age. And, if they are to make judgements about the quality of the education their children are receiving, they require information that enables meaningful comparisons across schools.F[27]

2.24      The Australian Parents Council, putting aside its reservations about the way NAPLAN results are currently used, recognised that parents need and are entitled to information on their children's education and progress.F[28]F The Australian Education Union submitted that parents have a right to know about their children's progress, but stated that '...there is no inherent right to information concerning other children at the school'.F[29]F

2.25      Barrack Heights Public School members of the NSW Teachers Federation called for more safeguards around the use of test results in order to prevent profit‑based organisations, such as media outlets and real estate agencies, from manipulating data for financial gain.F[30]F

2.26      This tension between the rights and advantages of parents and teachers accessing information on student progress on the one hand, and how this information was being used on the other, was evident in other submissions too:

The results of testing should be used to inform the teacher on the progress of students, the effectiveness of their teaching strategies and to give feedback to students and parents. Test results should not be used to make odious comparisons between schools.F[31]

2.27      Dr Ben Jensen emphasised how test result data should be used:

Any measure of school performance should not be viewed as an end in itself; they should be a basis of action. NAPLAN and My School should trigger actions that help Australian students.

First, NAPLAN results could be used to trigger actions to help underperforming students. Actions should be taken once a student performs at or below minimum literacy levels in the NAPLAN assessments and a development program and perhaps special assistance introduced until they are performing at appropriate levels. This has been successful in numerous high-performing countries. Second, schools labelled as underperforming on My School should be placed on a development program until they are performing at adequate levels. We are failing the students in these schools and we all need to ensure that these problems are addressed.F[32]

10BCommittee majority view

2.28      The committee majority agrees with the fundamental importance of literacy and numeracy, as supported by educational research. It understands that NAPLAN tests, while not in the traditional sense designed as 'diagnostic' assessments, can identify strengths and weaknesses in literacy and numeracy, and recognises the importance of measuring literacy and numeracy.F[33]F The committee majority notes the many uses for NAPLAN data and agrees that standardised testing is a useful instrument for informing system-wide policy decisions such as the allocation of resources. It helps schools to identify strengths and weaknesses of programs, allows for the comparison of results each year to identify trends and enables parents to follow performance and make informed decisions about the education of their children.

2.29      The committee majority is particularly drawn to the concept of providing underperforming students with a development program, the provision of which would be directly triggered by low NAPLAN results.

Recommendation 1

2.30      The committee majority recommends that ACARA and MCEECDYA explore and report publicly on ways in which to use below-average NAPLAN test results as a trigger for immediate assistance aimed at helping individual schools and students perform at appropriate levels.

Issues raised during the inquiry

2.31      Many submissions referred to national testing as a useful diagnostic tool, and supported it purely in that capacity.F[34]F The main issues raised in submissions and at the public hearings related to how the results are subsequently published and used. These issues are addressed in Chapter 3. However, submissions did raise some concerns in regard to standardised testing itself, and these are discussed below.

Test methodology and data reliability

2.32      Professor David Andrich in his submission to this inquiry noted:

The benefits that can arise from NAPLAN are based on the assumption that the quality, administration, analysis and reporting of the assessments is of the highest quality. If it is not of the highest quality, then unfortunately the assessments can not only be of little use, but can even be counterproductive. Because Australia has substantial skills and resources in educational assessment, there is no reason that NAPLAN should be of anything but highest quality.F[35]

2.33      A number of questions regarding the reliability of NAPLAN test methodology and data were drawn to the committee's attention. Some submissions suggest the tests may be prone to error, unreliable, or for varying reasons and to varying degrees unsuitable in assessing student or teacher performance,F[36]F for example because they '...are not sufficiently long to produce data of sufficiently high reliability to enable individual intervention or clinical style decisions to be made'.F[37]F

2.34      Associate Professor Margaret Wu of Melbourne University had concerns regarding margins of error when using NAPLAN tests to measure student performance. Given that NAPLAN consists of only one 40-question test per subject area, Associate Professor Wu concluded that scores in fact contain large margins of error and as such '...do not provide sufficiently accurate information on student performance, student progress or school performance'.F[38]F This can be additionally problematic as student scores are used to measure school performance, when in fact

...the publication of NAPLAN results on the My School website should be deemed as providing false information to the public, as the red and green bars do not in any way show school performance as claimed by the government.F[39]

2.35      Associate Professor Wu concluded that it is 'educationally unsound' to ask parents to make judgements on schools on the basis of NAPLAN results, and expressed her view that scores should not be published or accepted by the public without awareness of the impact of random fluctuations on results. She advocated against accepting scores '...if the confidence level of the results is not revealed'.F[40]

2.36      The Australian Education Union (AEU) further illustrated the problem presented by the margin of error in NAPLAN tests:

If you are examining literacy, for example, there might be, for argument's sake, 1,000 things you expect a child to know at nine years of age. They may know 600 of them. They do not know 40 per cent of them and do know 60 per cent of them. Depending on how you pick the test items, they may be picked disproportionately from the 40 per cent they do not know or they may be picked disproportionately from the 60 per cent of facts they do know. When you take that into account, that is where the measurement error for a test arises from.

With a test of 40 items, which the NAPLAN tests are, the measurement error for a student is around 12 per cent. For example, a student whose parents are advised that they have achieved a score of 60 per cent in a literacy test in fact has a score somewhere between 72 and 48. How people can ascribe usefulness to the data or to the My School website in the way that they have is totally beyond belief.F[41]

2.37      Professor Geofferey Masters, Chief Executive Officer of the Australian Council for Educational Research (ACER), which develops NAPLAN tests under contract to ACARA and manages the Program for International Student Assessment (PISA) for the OECD, emphasised that the assessments are '...firmly grounded in 20 years experience through the state literacy and numeracy testing programs.'F[42]F Professor Masters explained that:

NAPLAN is also pretty firmly grounded in international best practice in tests of this kind. Of course, they are point-in-time tests, so they are limited in that sense. They only assess part of what is important in schools. There is inevitably a degree of imprecision, measurement error, around the estimates that they provide. But they do represent best practice internationally. Part of the reason that the Australian Council for Educational Research are managing the PISA tests for the OECD out of Melbourne is that we do have international expertise in the analysis and reporting of data, and we are applying that to the NAPLAN tests.F[43]

2.38      Dr Peter Hill, Chief Executive Officer of ACARA, stated that NAPLAN tests:

...are provided to give an overall snapshot, and for that reason, unlike what was said earlier, we do not provide a score to parents. There is no score provided. In fact, a sheet similar to the one I have here is what is provided. It gives no score at all. It is quite a large dot on a continuum to indicate the position of the students, recognising that there is, indeed, always imprecision in our measures. However, as we aggregate those measures up to a school level and to a system level, then the more reliable those data become.F[44]

2.39      Professor Masters made the important point that there is a distinction between measures of student, teacher and school performance:

Some people believe that it is possible to go fairly easily from measures of student performance to measures of teacher performance, school performance or system performance. I do not share that view, and I think most of my colleagues at ACER do not share that view, but it is a commonly held view... I think that at ACER and also at ACARA we have not believed that... [it]...is either educationally or statistically valid...to [as is the case in the UK] move quickly from test results to a number or measure for each school in the country, and that measure is supposed to represent the school's performance so every school can be lined up and ranked on the basis of this one number, based entirely on the test results.F[45]F

2.40      MCEECDYA met on 15 October 2010 to discuss enhancements to the My School website recommended by ACARA's My School Working Party. The enhancements, most of which will be implemented in December 2010, include depictions of margins of error. Indications of the range in which school average performance may be located, with 90 per cent confidence, will be displayed alongside result data.F[46]

Committee majority view

2.41      The committee majority supports NAPLAN tests based on the understanding that they are grounded in international best practice. The committee majority recognises the concerns raised about test quality, and acknowledges that NAPLAN tests are subject to the same limitations in precision which apply to all such assessments. The committee majority believes that the current structure and appearance of the government's My School website is leading users to draw unintended or mistaken conclusions about how much can be inferred about teacher and school performance from the student test result data presented.

2.42      The committee majority strongly supports the abovementioned MCEECDYA initiative on the premise that confidence levels will be displayed on the My School website with adequate prominence, thus providing the community with a greater awareness of the complexities of the test result data.

Testing smaller student cohorts

2.43      The committee was also made aware that using mean school scores to measure school performance is problematic where small cohorts are involved. In such cases the performance of just a handful of students, sometimes one or two, who achieve extremely high or low scores can have an excessive and over time erratic  effect on the overall school result.F[47]

2.44      The committee majority supports the Tasmanian Department of Education's belief that the median may be a better and '...more stable measure for schools with small student populations'.F[48]F

Recommendation 2

2.45      The committee majority recommends that ACARA assess and report publicly on the potential benefits of moving to a system that reports the median rather than the mean school performance.

Data for researchers

2.46      The Australian Primary Principals Association (APPA) suggested that MCEECDYA amend the guidelines and protocols covering access to NAPLAN data so that qualified researchers can obtain access to de-identified data.F[49]F The committee majority supports MCEECDYA investigating how this request can best be met.

Testing year 3 is too early

2.47      It was suggested in a small number of submissions that Year 3 NAPLAN tests are too difficult and 'developmentally inappropriate' for this age group, and will ultimately shift teachers' focus onto, for example, persuasive essay writing at a time when they would otherwise focus instead on more basic writing skills.F[50]

2.48      Professor Brian Caldwell posed the question:

...[A]re the problems facing Australia so serious that we require students as early as year 3 to complete 40 to 50 mostly multiple choice tests when the information that is furnished and the strategies that should be adopted have been known for at least a decade?F[51]

2.49      In its submission the Northern Territory Government also touched on the question of NAPLAN and Year 3 students, saying that the diagnostic needs of the highest and lowest student achievers, particularly those in Year 3, are not currently addressed by NAPLAN.F[52]F The submission suggests a 'rigorous interrogation' of contextual bias and reconsideration of '...the range of difficulty of test items...to maximise information gathering opportunities at the top and lower ends of student ability'.F[53]

NAPLAN and special needs students

2.50      Some submissions were concerned about low NAPLAN scores attained by special education students or students with learning difficulties causing a distorting effect on school performance.F[54]F The NSW Teachers Federation quotes a principal who received an apology from the parent of a child with severe learning disabilities because that child would impact negatively on the school results. This all points to '...a perverse incentive for schools to exclude students who are most in need of support'.F[55]

2.51      One suggestion to address this issue was vetting such results prior to reporting NAPLAN test scores on My School.F[56]F Following this suggestion would mean that special education students would still be tested and their teachers and parents given results, but that their scores would not impact on the overall school results.F[57]F

2.52      Another concern raised in submissions was that if schools fear underperformance they may discourage students with learning difficulties, those from non-English speaking backgrounds or those who are low achievers from sitting the tests.F[58]F There was some media reporting alleging that schools discouraged students from attending on test days if they knew their results were likely to pull the school's overall performance down.F[59]F

2.53      The ACT Council of Parents and Citizens Associations stated that some parents had reported having to provide schools with written requests before children with a disability could sit NAPLAN tests.F[60]F

2.54      On this point, ACARA clarified:

Every child is eligible and is encouraged to participate in the testing. In the circumstances of students with significant disabilities, a decision is made in consultation between the principal and the parents as to the impact that testing might have on the student from the perspective of either their ability to perform or their inability to perform – the effect that might have on their confidence and their self-esteem. The requirements for that process are clearly set out in the administrative handbook, which is provided to principals in every school.

I suspect what we are looking at is a situation where a parent has had a discussion with a principal about the participation of a student and the principal might as a matter of prudence have decided that it might be worth confirming in writing the parent's consent for the child to participate so that there is no misunderstanding of the circumstances.F[61]

2.55      The committee notes that the federal Department of Education, Employment and Workplace Relations (DEEWR) is working towards developing a nationally agreed definition of disability.F[62]

2.56      Following particular allegations in Victoria that some students who did not sit the tests did not meet criteria for exemption, such as having an intellectual disability, the Education Minister Bronwyn Pike MP reported that between 4.7 and 8.6 per cent of students were absent or withdrawn from school on the day of NAPLAN testing, which was particularly high. To address this, the Victorian government has requested that principals formally agree to ensure the highest possible level of participation.F[63]

2.57      At its October 2010 meeting MCEECDYA formally endorsed ACARA's proposal to include student participation data more prominently on the My School website, including absences, exclusions and withdrawals.F[64]

Committee majority view

2.58      The committee majority notes that the NAPLAN Frequently Asked Questions (FAQ) website advises that all Australian governments have committed to promoting maximum participation of students in the national assessment process.F[65]F Common national practices for providing students with special support, adjustments and accommodations for the administration of the NAPLAN tests have been agreed.F[66]

2.59      The committee notes the concerning evidence and allegations of schools attempting to manipulate test results by urging potential low achievers to stay home. Despite the relatively small number of such cases and the commitment from federal, state and territory governments to address the issue by tracking participation rates, the committee majority is deeply concerned that parents of children with a disability be given adequate opportunity to access information about their children's progress, to which they are entitled like any other parent.F[67]

Recommendation 3

2.60      The committee majority recommends that MCEECDYA and relevant jurisdictional test administration authorities look at and report publicly on ways to ensure that children with disabilities are not discriminated against and denied the right to participate in national testing.

NAPLAN and learners of the English language

2.61      The committee received submissions calling attention to possible inadequacies of the NAPLAN tests for some Indigenous students who may not speak Standard Australian English as their first language but are not treated as students with a Language Background Other Than English (LBOTE). Queensland Indigenous ESL and FNQ (Far North Queensland) Language Perspective stated:

The current measures of disaggregation according to Indigenous status, students with a language background other than English (LBOTE) and the Index of Community Socio-Educational Advantage provide a false and pernicious picture of Indigenous learner performance and are leading to inappropriate and wasteful measures of intervention.

...[W]ithout the requirement of second language assessment in schools for students being tested on NAPLAN, Indigenous learners are generally not placed within the LBOTE group.F[68]

2.62      NAPLAN may also have limitations in capturing the progress of students starting to learn English as a second language,F[69]F which again may affect not only children of recent migrants, but also some Indigenous students who speak non‑standard English dialects.F[70]

2.63      Submissions argued this is because NAPLAN tests do not collect adequate information about students who are learning English as an additional language or dialect, and may not recognise them as such before reporting their sub-standard results.F[71]F

2.64      Suggestions for addressing this included the collection of more specific data for students from a non-English speaking background.F[72]F Disaggregated data would enable specific support to these categories of students.F[73]F

Committee majority view

2.65      The committee majority supports the collection of more specific student data which would then be used to ensure appropriate support is available.

2.66      The committee majority notes that the My School website will in future display the percentages of students from a non-English speaking background.F[74]

Recommendation 4

2.67      The committee majority recommends that ACARA analyse and report publicly on how NAPLAN tests are serving different groups of Language Background Other Than English (LBOTE) students.

Indigenous student performance

2.68      The committee majority notes the situation in the Northern Territory, where Indigenous students comprise over 40 per cent of the student population, with particular concern. International testing shows that Indigenous students are overrepresented in the lowest performance categories for both literacy and numeracy, and underrepresented in the highest.F[75]F This trend is evident nationally as well, with Indigenous students considered the most educationally disadvantaged.F[76]F

2.69      The Council of Australian Governments (COAG) reports a marked difference in NAPLAN results in the Northern Territory, where only between 62.5 per cent and 77.0 per cent of students reach the national minimum standard in all domains and in all years. All other states and territories perform well against the national benchmark, with varying room for improvement. The committee notes that this difference between the Northern Territory and other states and territories reflects particularly negatively on educational outcomes for Indigenous students, who comprise 40.7 per cent of the school population in the Northern Territory. NAPLAN results for non-Indigenous students in the Northern Territory are comparable to other jurisdictions.F[77]

2.70      The committee is aware of evidence which suggests that students who begin to fall behind without being caught and helped to catch up will continue to fall further and further behind, making these achievement gaps increasingly difficult to close as time goes by.F[78]

2.71      Speaking before the committee, Dr Peter Hill emphasised the importance of addressing quality of teaching in areas of greatest student need:

I think that what you said earlier about the heart of the matter being teacher quality is absolutely true. All of the international surveys point to teacher quality – or quality of teaching, should I say – being the key in all of this. What we need to use My School for is to understand where the problems are and then to have strategies that can come in, address quality of teaching... I am thinking particularly of our Indigenous students; we cannot have any pride at all in our record of achievement there. We need to put in additional resources and do what we can to improve the quality of the teaching that those students experience.F[79]

2.72      The above sentiment was echoed by Professor Brian Caldwell, who stated that '...there is almost unanimity among researchers and policymakers that the most important resource of all is the quality of teaching'.F[80]

Committee majority view

2.73      The committee majority believes that, in order to produce a more accurate and detailed picture of students who are achieving below the benchmarks, ACARA should investigate ways to provide access to rich information on lower achievers so that targeted support and resources can be made available.

2.74      The committee majority also believes that more emphasis must be placed on nurturing and developing the professional skills of teachers, in particular those working to improve outcomes for lower student achievers.

Recommendation 5

2.75             The committee majority recommends that ACARA investigate and report to MCEECDYA on enhancing NAPLAN to support the diagnostic needs of higher and lower student achievers.

Expanding testing

2.76      Dr Ben Jensen, Director of the School Education Program at the Grattan Institute, explained that the best predictor of a student's likely performance on a given day is their performance in the previous year:

There has been a lot of work done on this in the United States. In some states where there is serious testing in a wide variety of subjects it is annual or more frequently than that. They actually do not bother using the socio-economic background characteristics of students because it just drops out of the model. It is not important; it does not matter. What is much more important is your progression, how much you have progressed and, of course, where that begins.F[81]

2.77      Dr Jensen, while not advocating increased frequency of testing, explained that the two-year gap between tests in Australia means that extrinsic factors such as socio‑economic background have an increased opportunity to influence individual student performance.F[82]F Australian students currently take NAPLAN tests only four times over the course of their schooling, in Years 3, 5, 7 and 9. More frequent testing would lessen the influence of extrinsic factors and provide a more accurate picture of student progress.

2.78      More frequent testing would also enable schools and policymakers to more accurately capture the progress of students who change schools between primary and high school. Currently, My School includes comparisons of Year 7-12 high schools and P-7 primary schools, despite the fact that in some jurisdictions Year 7 is the first year of high school and students may have been at a different school three months prior to the NAPLAN tests.F[83]F The absence of information on student performance in Years 6 and 8 makes it difficult to ascertain which school, the primary or the high school, is responsible for student performance in Year 7.

Committee majority view

2.79      The committee majority supports the collection and reporting of information about progress in schools and believes it should be expanded. In order to provide parents, teachers and government with the best possible record of student progress and immediately begin to address serious gaps between our highest and lowest achievers, a national test designed to measure improvement for students should be conducted every year. The committee majority believes that more frequent testing would help drive momentum for helping underperforming students.

Recommendation 6

2.80      The committee majority recommends that ACARA and MCEECDYA expand NAPLAN to include annual testing from years 3 to 10 in order to more accurately track student performance and give parents, teachers and policymakers a far better understanding of how students, teachers and schools, are progressing.

Timing of test administration

2.81      Submissions pointed out that students are assessed against their respective year standard before being taught most of the year's curriculum because tests are administered in the first half of the year (with results not available until the second half). It was argued that this could disadvantage students whose schools teach relevant material later in the year, and could lead schools to alter teaching plans in order to attain better NAPLAN results.F[84]F

2.82      The timing of the tests also means that by the time teachers and parents receive student results, in the second half of the school year, it is too late to incorporate any resulting teaching requirements into that year's teaching program. This may have led to some schools pressuring teachers of Year 1 and 2 students to prepare students for concepts tested in Year 3 NAPLAN tests.F[85]

2.83      Dr Peter Hill, Chief Executive Officer of ACARA, explained that:

The purpose [of national testing] has been to get a snapshot of student performance for reporting back at different levels: at the parent level, at the school level, at the jurisdiction sector level and the national level. That was the purpose from the beginning, and the purpose has never been diagnostic assessment.

Diagnostic assessment means that we look at the reasons why students are, perhaps, not performing. For that purpose we need an immediate feedback; these tests are broad in scope and would not be very useful for diagnostic purposes, particularly as the results come through very late.F[86]

Committee majority view

2.84      The committee majority believes that the government's poor communication of the intended purpose of NAPLAN tests has led to widespread community misunderstanding or confusion about the capacity of the tests to diagnose why a child is performing at a particular level. The committee majority notes that then-Education Minister the Hon. Julia Gillard MP may have helped perpetuate an erroneous perception of the purpose of NAPLAN tests by stating about test result data:

It's important to teachers; they do value this diagnostic information to work out what they need to do next for the children in their class.F[87]

2.85      The committee majority believes that a better communication strategy is needed to explain the true purpose of NAPLAN tests.

Security of the tests and allegations of cheating

2.86      The committee majority notes media reports of teachers and schools allegedly cheating to boost their NAPLAN results.F[88]F Save Our Schools pointed out that schools and teachers can cheat in various ways and argued that this calls into question the reliability of NAPLAN tests in measuring school performance.F[89]F

2.87      Submissions identified that increased accountability pressure may unintentionally increase the likelihood of cheating.F[90]F A number of suggestions were made to address this issue. The ACT Council of Parents and Citizens Associations recommended a review of the document that outlines how tests should be conducted.F[91]F The New South Wales Primary Principals' Association suggested publishing clear and uniform delivery protocols and highlighting the consequences of any breaches.F[92]

2.88      Responsibility for test material during development falls to ACARA, which '...prescribes security requirements for states and territories, schools and principals in a nationally agreed document – National Protocols for Test Administration.' State and territory jurisdictions are responsible for security during test administration. Test administration authorities in states and territories are responsible for investigating any allegations of security breaches.F[93]

2.89      The committee majority is aware that MCEECDYA has endorsed a range of measures aimed at enhancing test administration security. ACARA is now working with state and territory authorities to strengthen test security for 2011, and is:

...mounting a multi-level communication strategy in 2011 to further develop understanding of the required protocols for the management of test materials on the part of schools, principals and staff.F[94]

2.90      ACARA has informed the committee of plans to include annual statements on its website detailing all reports of security breaches, the status of reported cases and outcomes of any subsequent investigations. Schools and individuals will not be identified. Education ministers are currently considering a draft of the first statement.F[95]

Committee majority view

2.91      The committee majority believes that the community should be able to have confidence in the testing process and that uniform test administration guidelines should be developed and made publicly available as a matter of priority.

2.92      The committee majority notes that steps have been taken by state and territory education departments to investigate and address allegations of cheating, with more than 51 separate investigations under way.F[96]F The committee majority recognises that the government has been firm in its commitment to supporting investigations into all allegations of cheating.F[97]

2.93      The committee majority also notes that the Australian Primary Principals Association has developed and sent to the government for consideration a set of principles and a number of proposed safeguards aimed at mitigating a range of negative impacts and adverse effects of the test administration and reporting regime.F[98]F

Recommendation 7

2.94             The committee majority recommends that MCEECDYA explore ways for state and territory test administration authorities to more strongly enforce security protocols.

Conclusion

2.95      The committee majority believes that NAPLAN is an important foundation for measuring the performance of students but needs to be strengthened in a number of ways. It needs to provide a more accurate and detailed picture for all students, particularly those not meeting performance standards. Test developers need to gradually look at ways in which to reduce the margin of error in order to turn NAPLAN into a more accurate tool. Furthermore, to provide an even better understanding of student progress trajectories year on year, the committee majority believes that national testing should be conducted every year. This would be particularly beneficial for students who do not meet national benchmarks as it would help build a sense of urgency and reduce the delay in delivering targeted assistance. These enhancements will provide a more accurate and detailed picture of students' ability without the influence of extrinsic factors, and will provide policymakers and schools with an informed picture of which educational programs are working and which ones are not.

2.96      Building on the enhancements outlined above, the committee majority also believes that substantial work is required to address the significant issues raised during the committee's inquiry in relation to the reporting of NAPLAN data on the My School website. These matters are covered in the next chapter.

Navigation: Previous Page | Contents | Next Page