Feature Article Changes to the accuracy of labour
statistics
Recently the Australian Bureau of Statistics (ABS) has made
changes to the way it collects labour market data. These changes
have reduced the accuracy of results published as the ABS labour
force statistics.
Labour Force Survey
It is important to decision makers to receive timely and
accurate social and economic indicators of what is happening in the
economy. Among the key indicators are labour market data. The ABS
has been producing labour force data on a monthly basis for more
than 30 years.
It is not practical to ask every person in Australia their
labour market status on a monthly basis. Hence the ABS conducts a
sample survey, the Labour Force Survey (LFS). The LFS survey
randomly selects a number of households from the population and
questions the occupants on their labour force status and
experience. There are about 30 000 households surveyed each
month.
All people aged 15 years and over in these households are the
sample; in June 2008 this was 54 900 people. This sample
is used to calculate all LFS estimates including such important and
widely-reported statistics as the number of people employed, the
number unemployed, the unemployment rate, and the labour force
participation rate.
Sample size and accuracy
Ultimately the size of the sample dictates the accuracy of the
statistics calculated from that sample. In general terms a small
sample has lower statistical accuracy, a large sample has higher
accuracy. High accuracy is clearly desirable. However, surveying a
large number of people costs more than surveying a few people. So,
the sample size must be chosen to achieve an accurate outcome while
at the same time minimising costs.
Whenever a statistic such as the level of unemployment is
calculated from a single random sample we do not know whether the
statistic is high or low or just about right. This is because there
are a very large number of samples which we could have taken and we
only have one of them. Each sample will produce a slightly
different result so that some samples will produce a low estimate
of the statistic being estimated and other samples will produce
high values.
Statisticians measure accuracy—how close the statistic is
to the true value—using a statistic called the standard error
of the sample. There is a 68 per cent chance that the true value
lies within one standard error of the sample statistic and a 95 per
cent chance that the true value is within two standard errors of
the sample statistic.
This goes to the heart of the changes now made to the way that
the ABS collects data to produce a picture of the labour force.
According to the ABS, its tight 2008–09 budget has forced
reductions in its statistical work program. As a result the ABS has
decided to reduce the LFS sample size across Australia by 24 per
cent, i.e. from 54 900 in June 2008 to around 41 900 in
July 2008. With this has come an inevitable reduction in accuracy
which shows up as an increase in the standard errors associated
with each statistic.
This loss of accuracy can be shown in two ways. One is to
calculate what is called the relative standard error (RSE) which is
the ratio of the standard error to the value of the statistic. Thus
between June and July 2008, for the level of unemployment, the
relative standard error increased from 2.7 per cent to
3.3 per cent. The other way to show the loss of accuracy is to
simply use the standard errors which are conceptually easier to
grasp than RSEs and are published by the ABS.
Effect on estimates
Every statistic calculated from a survey such as the LFS has an
associated standard error. For the level of unemployment, for
example, the standard error was 12 400 in June 2008. This
meant that in June 2008, we could be 68 per cent
confident that the true value for unemployment lay in the range
12 400 less than the reported statistic to
12 400 more than the reported statistic. Thus at the
68 per cent level, we could say that the true value lay in a
range that was two times 12 400 wide, viz. 24 800 people
wide. At the 95 per cent level the true value lay in a range
from twice 12 400 less than the reported statistic to twice
12 400 more than the reported statistic; this made the range
49 600 people wide at the 95 per cent level.
In July 2008, the published standard error had grown by 2200
from the 12 400 of June 2008 to 14 600. This means
that the range had grown to 29 200 people at the 68 per
cent level and 58 400 at the 95 per cent level.
Not only is it possible to calculate standard errors for the
levels of the various LFS statistics, it is also possible to
calculate them for changes in levels, i.e. in movements. Movements
are often quoted in the media as, for example, the level of
unemployment rises and falls. Hence in July 2008 the published
standard error on the changes of level of unemployment was
15 300 which is an increase of 2000 from that for
June 2008 which was 13 300. This makes the range for one
standard deviation 4000 people wider and for two standard
deviations 8000 people wider.
The bottom line is that the accuracy of the estimates produced
by the LFS has declined. Depending on the certainty needed, the
width of the confidence intervals on levels and on movements have
grown, in the case of unemployment by as much as 8800, of
employment by as much as 28 800, and of the labour force as
much as 29 200.
Monthly statistical bulletin
tables 1.1 to 1.5
ABS LFS data appears in the
Monthly statistical bulletin tables 1.1 to 1.5. All data
in these tables from and including July 2008 are now less
accurate than they have been.
Greg Baker
Statistics and Mapping Section
Back to top