Chapter 3

The conduct of the postal survey


3.1        This chapter canvasses a number of issues raised during this inquiry including:

Precedents for surveys of this kind

3.2        At the public hearing on 17 August 2017, Mr Jonathan Palmer, Deputy Australian Statistician at the Australian Bureau of Statistics (ABS) described the scope of the Australian Marriage Law Postal Survey (postal survey):

A single question will be asked: should the law be changed to allow same-sex couples to marry? We will ask all eligible Australians—and they are the people who, as at the end of 24 August, would be entitled to vote as per the electoral roll—to provide a yes or no answer on a strictly voluntary basis. The point I would like to emphasise here about the process is that it's been designed to provide both secrecy and integrity. The vast bulk of Australians who are eligible for this survey will receive and return their survey form by post. They'll be sent a letter with instructions, a survey form and a reply-paid envelope. The barcode on the survey form will be used for marking purposes only. We're calling it a single-use, anonymous marking code. No person who sees or has any access to the completed forms will know both the name of the eligible Australian and the related single-use code. The survey form will not have any name or address on it. Only one response can be made by each eligible Australian. When the form is scanned, the barcode will be recorded in a separate file for marking purposes and the survey response will be recorded in another file. There will never be an electronic file containing both the survey response and the bar code. The form's encaptured images will be destroyed within 60 days of our publishing the statistics on 15 November.[1]

History of plebiscites and postal surveys in Australia

3.3        Plebiscites and postal surveys have rarely been used at a national level in Australia. In a 2011 research paper, the Parliamentary Library stated that there have been three national plebiscites:

3.4        Postal surveys are almost unprecedented, with only one occurring nationally. In an answer to a question on notice, the Department of Finance described the 1997 Constitutional Convention Election as 'a national voluntary postal process to elect delegates to the Constitutional Convention to determine whether Australia should become a republic'.[3]

3.5        However submitters gave evidence that the postal survey was fundamentally different from other endeavours undertaken by the ABS. The committee was told in a submission by an ABS staff member, in their private capacity, that the directives to the ABS were so prescriptive as to 'violate the guidelines and principles of official statistics'.[4]

3.6        The committee were told that the ABS has collected data on social issues for over forty years:

The ABS has been conducting large social surveys since the 1970s, and it's familiar with undertaking voluntary statistical surveys that seek views and opinions of Australians about issues such as their self-perceived wellbeing, social experiences and society in general. I thought I would share just two examples for you of instances where we've sought opinions...So, from a personal safety survey, we asked: how safe or unsafe do you feel walking alone in your local area after dark? And we published statistics on that question. From a survey on disability, ageing and carers, we asked: do you feel you need more support or an improvement in your situation to aid your role as a carer? And the answer to that was a yes or no question.[5]

Timetable of the survey mail-out

3.7        A timeline of key events during the postal survey is outlined below in Table 3.1. The committee understands that the ABS and other government departments and agencies have met each of these milestone dates.

Table 3.1—Timeline of key events during the postal survey[6]

24 August 2017 Commonwealth Electoral Roll closed.
12 September 2017 Mailing of forms and collection process began.
13 September 2017 Legislation providing additional safeguards during the marriage law survey passed the Parliament.[7]
25 September 2017 All survey packages should be delivered by this date.
20 October 2017 Final day to request or pick up replacement forms.
27 October 2017 Final day to post forms to ensure that they are included in the final survey results.
7 November 2017 Postal survey closed.
15 November 2017 Postal Survey results announced and published.

The Australian Bureau of Statistics' capability to deliver

3.8        When the ABS were announced as the lead agency to conduct the postal survey, many questions were raised about the ability of the ABS to manage this process in light of some of the difficulties associated with the 2016 Census.[8] Mr Palmer acknowledged these concerns:

Another issue I would like to touch on is concerns that have been expressed about our [ABS's] capacity to deliver on undertaking this survey. In response to that, we say, 'Yes, we can.' We have been given an adequate appropriation—$122 million—through an advance to the Finance Minister determination. The 2016 census online form outage did hurt the public reputation of the ABS. We acknowledge that. But we're seeing some rebuilding of that reputation following the release of a quality census dataset. I can assure you that we've reflected and learnt from the lessons of the 2016 census and we're applying those experiences and lessons to the conduct of this exercise. We've established a team of more than 40 skilled professionals, including staff seconded from the Australian Electoral Commission and other agencies. We have systems, processes, technical expertise, and we will partner with high-calibre suppliers and vendors. Undertakings as large and as complex as the Australian Marriage Law Postal Survey entail risks. We will manage these risks and issues well. And I would like to place on record my appreciation for the work done by the task force so far. We've done an enormous amount.[9]

3.9        This came at a cost to the ABS's other operations. The committee was informed by the ABS that it diverted staff from other projects, including the 2021 Census in order to be able to deliver the postal survey.[10]

3.10      Earlier in the report, the committee discussed the source of the funding for the postal survey—an Advance to the Finance Minister of $122 million. In announcing the result of the postal survey, the Australian Statistician, Mr David Kalisch noted that the postal survey was delivered under budget:

...while costs are still being tallied, the ABS is confident the final cost for the survey will be under $100 million, many million dollars less than the available budget.[11]

3.11      Throughout the inquiry, the committee has held government departments and agencies, the ABS in particular, to account to ensure that the postal survey was completed as intended. The committee has focused on a number of key areas which are described in more detail through the remainder of this chapter.

Accessibility and participation in the survey

3.12      The postal survey had a national participation rate of 79.5 per cent. Breaking down the results by state and territory, age and sex returned a participation rate range between 70 and 80 per cent. Notably, participation rates for males aged 20–24, 25–29 and 30–34 years all fell below 70 per cent.[12]

Challenges in remote areas

3.13      On 17 August 2017, Mr Palmer acknowledged that many people in remote parts of the NT did not have street addresses and that the postal survey would be a different process to the census in which ABS employees attend people's homes to collect information on census night in remote localities.[13]

3.14      Mr Palmer reassured the committee that the ABS was finalising an inclusion strategy to ensure a high participation rate in the postal survey:

I'm not yet able to tell you what our inclusion strategy will be, but we're looking at all the options for reaching those people and giving them an opportunity.[14]

3.15      At its committee hearing on 7 September 2017, Mr Kalisch noted that the ABS had finalised and published a 'comprehensive list of inclusion strategies' on its website, and acknowledged that:

The ABS recognises that Aboriginal and Torres Strait Islander communities in remote areas may experience particular challenges in participating in the Australian Marriage Law Postal Survey.[15]

3.16      The strategy for remote and rural locations included early despatch of survey materials for remote locations,[16] 'the use of pick-up locations, Department of Human Services' agents, access points and remote service centres'.[17] Mr Palmer elaborated on the remote strategy at the 15 September 2017 public hearing:

If remote Aboriginal and Torres Strait Islander people don't receive a form, there are still a number of options available to them over the four weeks from 25 September to 20 October. We will be providing the opportunity for forms to be collected in 27 different regional locations—places like Karratha, Mount Isa, Coober Pedy, to name some—and 200 other remote locations. These locations will be advertised on the ABS website and communicated directly to remote communities and promoted through local radio and other media. In addition, people with access to phone or the internet can respond using our paperless response options. Material will be translated into a number of different Aboriginal and Torres Strait Islander languages for radio advertising and to support the field work.

To reach those communities we are also working with the Department of Human Services to provide additional opportunities. We are going to leverage over 600 DHS agents' access points and remote service centres across Australia. Most of these locations can provide self-service computer access or telephones to freely access the survey information line on the ABS website and, indeed, those online response options of the online form and the telephony response. These facilities allow a person to request new materials from the ABS and to participate in the survey online, if that's their requirement, from 25 September.[18]

3.17      Notwithstanding the strong participation rates nationally and the ABS inclusion strategies, the overall participation rate in the Northern Territory (NT) was 58.4 per cent—significantly lower than the national participation rate of 79.5 per cent. When the NT is broken down into its two federal electorates, the statistics become even starker. The more remote electorate of Lingiari had a participation rate of 50.1 per cent. That is, nearly half of the people in the Lingiari electorate did not respond.[19] When broken down by age group, less than 40 per cent of eligible 20–24 year olds responded.[20] The Lingiari electorate is classified as rural with the majority of the enrolment being outside the major provincial cities[21] with 41.7 per cent identifying as Aboriginal and Torres Strait Islander and 24.5 per cent of households speaking a non-English language (with the top five responses being Aboriginal languages).[22]

3.18      The low participation rate for the electorate of Lingiari in the postal survey is consistent with the turnout history of the electorate at the 2016 federal election where Lingiari also recorded the lowest enrolled voter turnout of all federal elections.[23]

3.19      The difficulty of delivering the postal survey to eligible remote citizens was raised in correspondence tabled at the committee's public hearing on 17 August 2017 from the NT Chief Minister, Hon Michael Gunner to the Prime Minister, Hon Malcolm Turnbull MP. In this correspondence, the Chief Minister raised his concerns about 'Territorians, particularly remote Indigenous Territorians, being disenfranchised from voting in the postal plebiscite'.[24] Mr Gunner elaborated:

Only 82% of Territorians are presently enrolled meaning that more than 25 000 eligible voters are not enrolled. There is also a high level of mobility amongst Indigenous electors.

Significantly, for a very large number of our remote communities there are poor or non-existent postal services. For many communities, the nearest post office is in the next regional town which could be hours away.

Furthermore, there is a high proportion of Indigenous electors who do not speak English as a first language and I am convinced that there has been very little information about in-language material being made available for electors in the plebiscite...

I find it very difficult to comprehend how the Australian Government and the Australian Bureau of Statistics is reaching out to local communities to encourage people to enrol, to update their enrolment details and provide adequate and in-language material to encourage people to cast and return their vote for this important societal issue.[25]

3.20      In its submission, Tangentyere Council Aboriginal Corporation (Tangentyere) noted that 'Town Camp street addresses are not recognised as enrollable addresses by the [AEC]'. [26] This has a number of serious consequences for Town Camp residents:

Not having a street address means that with the exception of residents with a separate post office box no resident receives mail from the Australian/Northern Territory Electoral Commission. This in turn leads to reduced enrolments, obsolete enrolment details and poor rates of participation.[27]

3.21      Tangentyere shared its observations of low participation in Town Camps in the Alice Springs area:

The evidence suggests that very few Town Camp residents received the 'Marriage Equality Postal Survey' without the intervention of Tangentyere and the flexibility of the Australian Bureau of Statistics. Tangentyere canvassed residents from Ilperle Tyathe, Aper-Alwerrknge, Mount Nancy, Anthelk-Ewlpaye, Nyewente, Akngwertnarre, Ewyenper Atwatye, Yarrenyty Arltere (Larapinta Valley), Anthepe, Inarlenge, Ilyperenye, Ilparpa (New Ilparpa), Mpwetyerre, Karnte and Lhenpe Artnwe and was informed that other than those with [post office] boxes at Australia Post nobody received the postal survey.[28]

3.22      Furthermore, Tangentyere noted that this situation 'would be systemic across remote Aboriginal communities, outstations and Town Camps throughout the Northern Territory'.[29]

Updating of the electoral rolls

3.23      Mr Palmer acknowledged that the electoral roll presented risks to the postal survey process:

Another risk is that the electoral roll might not be an accurate reflection of all eligible Australians. We're mitigating that risk by undertaking a major communications campaign and working with the AEC [Australian Electoral Commission]—well, the AEC is undertaking an update of the electoral roll.[30]

3.24      The committee heard that where a person has not notified the AEC of a new address before the closure of the electoral roll that a new form could only be provided to that person when they bring it to the attention of the AEC. Mr Palmer explained what would happen if a survey form was sent to the wrong address:

They might have realised that their postal address might not be current, in which case they can contact our information line and ask for us to send a form to another address. We'll issue them with a new code and they can use that code. The code that was on the survey form that was sent to the wrong address, if you like, becomes invalid and won't be processed.[31]

3.25      Mr Andrew Brooks raised his concerns about his postal survey being despatched to a previous address and questioned whether an updated version of the electoral roll was being used:

When the postal survey was first sent out between Tuesday 12th September and 25th September, I did not receive mine at my enrolled addressed. It was not until my mother who was visiting my grandparents at my old registered address notified me that my postal survey was there. I have been enrolled to vote in the Parkes electorate since 2010 and even ran and voted for myself with no problems and had no issues voting in subsequent Federal Elections. My electoral roll details have not been registered in Blaxland since before 2010. I contacted the ABS and same sex survey hotline who passed off my concerns by trying to claim that I had enrolled too late after the 24th August deadline despite knowing I was enrolled in Parkes.

I subsequently verified on the AEC website that I am enrolled in Parkes and to make sure checked my old address in Blaxland in case there was a double up on the system. There was none.[32]

3.26      Despite these concerns, the committee heard that the AEC had processed 'nearly a million transactions in the lead-up to the close of the roll for this survey'.[33] To put this into perspective, 'this is approximately 250 000 more enrolment transactions than for the close of rolls period for the 2016 election'.[34]

Lost and stolen forms

3.27      At its 7 September 2017 public hearing, the committee sought assurances from Australia Post that despatched and returned forms would not be lost or stolen from the mail system and that they would be delivered on-time. Ms Christine Corbett, then Interim Chief Executive Officer of Australia Post stated that:

Our service performance standard is over 98 per cent of letters are delivered on time or early. This is independently audited. We have very strong confidence in our ability to deliver the mail, as we have done for centuries. If anyone is concerned that they have not received a postal survey, the ABS has robust processes in place for a survey to be then sent out to any individual...

You are looking at 1.3 per cent that are not on time. When you then look at a delay of a further day or so on top of that, you are very close to 100 per cent.[35]

3.28      About 10 days later, the committee became aware of the alleged theft of postal survey forms in Melbourne.[36]

3.29      Fairfax Media reported that:

When prep teacher Kerry Ford returned to her Brunswick home from a holiday on Monday, what she found behind her carport made her angry and upset.

Seventeen addressed and unopened same-sex marriage postal vote envelopes had been dumped in front of her children's cubby house.

The letters were addressed to houses in the surrounding streets.

"I think it happened last week, some of them have been eaten by snails," she said. "Our postal votes have also been stolen."[37]

3.30      Stolen forms were also reported in other locations including from Mona Vale on Sydney's northern beaches.[38]

3.31      The deficiencies in the complaints handling process are discussed in Chapter 4.

Overseas voters

3.32      The ABS also answered questions about the postal survey process for eligible overseas based voters. The committee were informed that overseas voters in the first instance would be provided with the option to complete their postal survey online. Mr Palmer explained:

...we have tried to make it even [easier] for those people by sending them a secure access code straight up rather than requiring them to contact us. So those people who had an overseas address on the roll will get a letter with a 16-digit secure access code and, from the 25th [of September 2017], they'll be able to access the online form or the telephone.[39]

Scrutiny of the survey

3.33      The survey was not subject to the level of scrutiny usually applied to an electoral process.

3.34      Mr Palmer described to the committee how the ABS would ensure the high quality and integrity of the statistics gathered during the postal survey and what elements that this would focus on:

We will have an overarching assurance process...It will ensure that everyone who was on the electoral roll was given an opportunity to participate; that every valid form returned was processed and counted; that no person was able to participate more than once; that controls were in place to reduce, detect and remove fraudulent activity; that appropriate security arrangements were maintained around the data throughout the survey; that there was a separation of the response from personal information which was maintained throughout the survey; that all physical and electronic data was destroyed at the end of the survey; and, finally, that there were opportunities provided to external observers who were able to assure around the interpretation of marks made on survey forms. So we've got that, if you like, end-to-end coverage of steps in the process to ensure integrity. Those controls will be reviewed and reported on by independent auditors and assessors. Their report on these controls will be included in our quality statement when we publish the statistics on 15 November.[40]

3.35      Notwithstanding these arrangements, Mr Kalisch made the point that the postal survey is different to an election in that 'there are not the traditional scrutineering processes you would be familiar with for an election'.[41]

3.36      In place of scrutineers, the ABS made allowance for observers. Observers had more limited access to survey materials than scrutineers would have in an election, and were subject to far stricter eligibility requirements:

The external observers are people who will be nominated by two parliamentary committees to participate in the survey process in a particular way. The role of an observer will be to witness the interpretation of survey responses in order to report on their observations of the integrity of this key aspect of the survey process. We're requiring that observers be over 18 years of age. They'll be required to undergo a police check and sign a deed of confidentiality.[42]

3.37      Mr Ian Brightwell, a former Chief Information Officer and Director of Information Technology at the NSW Electoral Commission commented on the integrity and quality control processes at the ABS. Mr Brightwell noted that the ABS published online its procedures around the use of observers during the postal survey. Interestingly, Mr Brightwell contacted the ABS call centre to discuss these procedures in mid-August and was 'told by operators on both occasions that they did not believe there were any procedures available and did not know when they would be available'.[43]

3.38      Furthermore, Mr Brightwell pointed out that 'observers would only be allowed to observe key steps in the survey process' rather than the entire process. He also noted that the independence of this scrutiny process would be undermined in that observers would report directly to the Australian Statistician rather than to an independent body such as a parliamentary committee.[44]

3.39      On 15 November 2017, the ABS published a summary of results of the external observers' observation program. The ABS described this process:

External observers provided feedback on the coding decision [yes/no] made by the [ABS] on 606 991 survey returns.

3.40      Of the 311 318 automatically coded Yes or No survey forms that were examined by the external observers, there was only one instance where both observers disagreed with the ABS coding decision equating to 0.0003 per cent of these forms.[45] Table 3.1 below summarises the results from the external observer program.

Table 3.1—Summary of results from postal survey observation program[46]

Table 3.1—Summary of results from postal survey observation program

3.41             At the same time, the ABS also released a quality and integrity statement. Importantly, this statement noted that the 'Australian Marriage Law Postal Survey (AMLPS) statistics meet the requirements of the Census and Statistics (Statistical Information) Direction 2017 issued by the Treasurer on 9 August 2017'.[47] The ABS noted that the 'collection and processing of survey responses was undertaken with rigour and the results accurately reflect the views of survey respondents'.[48] Furthermore:

An overall participation rate of 79.5%, with consistent distribution across age groups, gender and geography, is a strong indicator of quality. This high participation rate was achieved because of Australians' significant interest in this topic combined with the statistical collection design and process which promoted participation and ensured quality and integrity, in particular:

Collection and destruction of survey forms

3.42      The committee heard that all forms 'will be destroyed within 60 days of [the ABS] publishing the statistics on 15 November' and that 'there will never be an electronic file containing both the survey response and the barcode'. The committee understands that destruction of the forms occurred within 60 days.[50]

Navigation: Previous Page | Contents | Next Page