Changes to the accuracy of labour statistics

Monthly Statistical Bulletin Feature Articles

Feature Article Changes to the accuracy of labour statistics

Recently the Australian Bureau of Statistics (ABS) has made changes to the way it collects labour market data. These changes have reduced the accuracy of results published as the ABS labour force statistics.

Labour Force Survey

It is important to decision makers to receive timely and accurate social and economic indicators of what is happening in the economy. Among the key indicators are labour market data. The ABS has been producing labour force data on a monthly basis for more than 30 years.

It is not practical to ask every person in Australia their labour market status on a monthly basis. Hence the ABS conducts a sample survey, the Labour Force Survey (LFS). The LFS survey randomly selects a number of households from the population and questions the occupants on their labour force status and experience. There are about 30 000 households surveyed each month.

All people aged 15 years and over in these households are the sample; in June 2008 this was 54 900 people. This sample is used to calculate all LFS estimates including such important and widely-reported statistics as the number of people employed, the number unemployed, the unemployment rate, and the labour force participation rate.

Sample size and accuracy

Ultimately the size of the sample dictates the accuracy of the statistics calculated from that sample. In general terms a small sample has lower statistical accuracy, a large sample has higher accuracy. High accuracy is clearly desirable. However, surveying a large number of people costs more than surveying a few people. So, the sample size must be chosen to achieve an accurate outcome while at the same time minimising costs.

Whenever a statistic such as the level of unemployment is calculated from a single random sample we do not know whether the statistic is high or low or just about right. This is because there are a very large number of samples which we could have taken and we only have one of them. Each sample will produce a slightly different result so that some samples will produce a low estimate of the statistic being estimated and other samples will produce high values.

Statisticians measure accuracy—how close the statistic is to the true value—using a statistic called the standard error of the sample. There is a 68 per cent chance that the true value lies within one standard error of the sample statistic and a 95 per cent chance that the true value is within two standard errors of the sample statistic.

This goes to the heart of the changes now made to the way that the ABS collects data to produce a picture of the labour force. According to the ABS, its tight 2008–09 budget has forced reductions in its statistical work program. As a result the ABS has decided to reduce the LFS sample size across Australia by 24 per cent, i.e. from 54 900 in June 2008 to around 41 900 in July 2008. With this has come an inevitable reduction in accuracy which shows up as an increase in the standard errors associated with each statistic.

This loss of accuracy can be shown in two ways. One is to calculate what is called the relative standard error (RSE) which is the ratio of the standard error to the value of the statistic. Thus between June and July 2008, for the level of unemployment, the relative standard error increased from 2.7 per cent to 3.3 per cent. The other way to show the loss of accuracy is to simply use the standard errors which are conceptually easier to grasp than RSEs and are published by the ABS.

Effect on estimates

Every statistic calculated from a survey such as the LFS has an associated standard error. For the level of unemployment, for example, the standard error was 12 400 in June 2008. This meant that in June 2008, we could be 68 per cent confident that the true value for unemployment lay in the range 12 400 less than the reported statistic to 12 400 more than the reported statistic. Thus at the 68 per cent level, we could say that the true value lay in a range that was two times 12 400 wide, viz. 24 800 people wide. At the 95 per cent level the true value lay in a range from twice 12 400 less than the reported statistic to twice 12 400 more than the reported statistic; this made the range 49 600 people wide at the 95 per cent level.

In July 2008, the published standard error had grown by 2200 from the 12 400 of June 2008 to 14 600. This means that the range had grown to 29 200 people at the 68 per cent level and 58 400 at the 95 per cent level.

Not only is it possible to calculate standard errors for the levels of the various LFS statistics, it is also possible to calculate them for changes in levels, i.e. in movements. Movements are often quoted in the media as, for example, the level of unemployment rises and falls. Hence in July 2008 the published standard error on the changes of level of unemployment was 15 300 which is an increase of 2000 from that for June 2008 which was 13 300. This makes the range for one standard deviation 4000 people wider and for two standard deviations 8000 people wider.

The bottom line is that the accuracy of the estimates produced by the LFS has declined. Depending on the certainty needed, the width of the confidence intervals on levels and on movements have grown, in the case of unemployment by as much as 8800, of employment by as much as 28 800, and of the labour force as much as 29 200.

Monthly statistical bulletin tables 1.1 to 1.5

ABS LFS data appears in the Monthly statistical bulletin tables 1.1 to 1.5. All data in these tables from and including July 2008 are now less accurate than they have been.

Greg Baker
Statistics and Mapping Section

Back to top

Facebook LinkedIn Twitter Add | Email Print