APPENDIX A

SHAPING PARLIAMENTARY COMMUNICATION AND TECHNOLOGY SERVICES FOR THE FUTURE

Report of the DPRS Client Survey 2000

Prepared for the Department of the Parliamentary Reporting Staff
15 December 2000

Acknowledgements

The consultants wish to acknowledge the cooperation and assistance of Senators and Members and their staff who completed the questionnaire and participated in the follow-up interviews. Similar cooperation and assistance was also generously provided by senior managers in Parliamentary Departments who participated in the research.

The consultants also wish to acknowledge the support of the Strategic Development Group within DPRS (Brian Cheetham and John Walsh) in assisting with the logistics of undertaking the research.

Consultancy Team

Michael Brooks, Marie Delaney (ARTD)
Martin Stewart-Weeks (Albany Consulting)
PO Box 216
Haberfield, NSW 2045

(02) 9716 4500 (tel)
(02) 9716 4300 (fax)
michael.brooks@artd.com.au
www.artd.com.au

Table of contents

1   Introduction

1.1   DPRS Client Survey 2000

DPRS is a parliamentary service organisation supporting the Parliament, Senators and Members and many sections the Australian community. The department's vision is to give all Australians the opportunity to see, hear and read the work of their national Parliament, through providing the Australian Parliament with world class, value for money services and support.

A key part of the department's approach is 'to work with clients to find the best way of meeting their needs.' (DPRS Corporate Plan 1999-2002). To ensure this happens, DPRS is committed to ongoing monitoring and evaluation of client's expectations and perceptions.

In August 2000, ARTD Management and Research werewas engaged by the department to conduct the DPRS Client Survey 2000. Key requirements of the survey were the:

• ability to derive quantifiable measures of client satisfaction for reporting against performance indicators in the Portfolio Budget Statement;
• need to obtain feedback from clients on key strategic issues for DPRS, bearing in mind changes in service delivery and department organisation;
• consideration of issues and solutions for organising the range of formal and informal sources of client feedback into a coherent system, including appropriate tools or a subset of questions to be used in a regular sampling program.

1.2   Methodology

The consultancy was undertaken in four stages:

• Development of a client feedback monitoring and evaluating strategy (stage 1)
• Self-completion questionnaire (stage 2)
• Follow-up client interviews (stage 3)
• Data analysis and reporting (stage 4).

1.2.1   Development of a client feedback monitoring and evaluating strategy

A client feedback monitoring and evaluation strategy was developed at the start of this project to ensure a systematic and coordinated approach was used in DPRS for gathering, collating and reporting client feedback, with clear links to service development and business improvement processes. The strategy builds on the both the DPRS Service Development Strategy (March 2000) and Service Charter (draft August 2000).

The strategy covers:

• formal consultation and feedback mechanisms
• informal feedback
• regular monitoring / research activities
• periodic evaluations.

The strategy was also designed to ensure appropriate data is available for reporting against key performance indicators relating to client satisfaction in the DPRS Portfolio Budget Statements (see Attachment 1).

1.2.2   Self-completion questionnaire

A self-completion questionnaire was mailed to all 225 Senators and Members, and sample of senior managers and officers in the other four parliamentary departments.

The survey contained 32 closed response questions and 6 open-ended questions.

Overall, 144 questionnaires were returned from clients, including 92 from Senators and Members (41 per cent response rate), 31 from senior managers in Parliamentary Departments (76 per cent response rate) and 21 from a random sample of Parliamentary Officers (23 per cent response rate).

Questionnaires distributed to Senators and Members offices were intended to be completed by the person in the office best placed to provide feedback on the range of DPRS services. It is estimated that one-third of these surveys were completed by the Senator or Member themselves, and two-thirds were completed by a relevant member of their staff.

In the context of the busy Parliamentary environment, this response rate provides a credible basis for assessing client satisfaction with DPRS services.

1.2.3   Follow-up interviews

Clients were also invited to participate in a series of follow-up interviews to elaborate on their views and to contribute their ideas about service improvements. Follow-up interviews were held with 1526 Senators and Members or their staff, and 10 senior managers in Parliamentary Departments.

1.2.4   Data analysis and reporting

A comprehensive analysis was undertaken of the quantitative and qualitative data from the questionnaire and follow-up interviews.

Section 2 of the report presents detailed client feedback on DPRS as a whole and on each of the five core areas of DPRS service delivery. These are:

• transcription services (Section 2.2);
• broadcasting services (Section 2.3);
• computer-related services (Section 2.4);
• client liaison and support (Section 2.5);
• other communication services (Section 2.6).

In addition, the frequency tables for the client survey are presented as cross-tabulations across three respondent categories - Senators and Members and their staff (n=92); senior manager in Parliamentary Departments (n=31); and Parliamentary Officers (n=21).

2   Survey findings

2.1   Client Service Standards

Findings from the Client Survey 2000 highlight that DPRS is perceived as a highly professional, client-focussed organisation. Among survey respondents there was a strong consensus that:

• DPRS provides professional and helpful service

  92% of clients [93% of Senators and Members] agreed that DPRS provides professional and helpful service at a world-class or very good standard. All remaining clients were satisfied that the professionalism and helpfulness of DPRS services was adequate [Client Survey Q6.1];

• DPRS responds promptly and courteously to client requests

  87% of clients [89% of Senators and Members] agreed that DPRS responds promptly and courteously to client requests. All remaining clients were satisfied that the promptness and courteousness of service delivery was adequate [Client Survey Q6.2];

• DPRS generally appreciates the time-constraints that clients are subject to

  Seventy-three per cent of clients [81 per cent of Senators and Members] agreed that DPRS appreciates the time-constraints that clients are subject to. Among Senators and Members who responded to the survey, a further 13 per cent were satisfied that DPRS' appreciation of time-constraints was adequate, with 6 per cent (4 respondents) raising concerns that this attribute of service was poor [Client Survey Q6.3];

• DPRS services are generally easily accessible

  75 per cent of clients [79 per cent of Senators and Members] agreed that DPRS services were easily accessible, including the provision of appropriate after hour support for critical business systems. Among Senators and Members who responded to the survey, a further 16 per cent were satisfied that the accessibility of DPRS' services was adequate, with 5 per cent (3 respondents) raising concerns that this attribute of service was poor [Client Survey Q6.4].

These findings are strongly reflected in performance data on benchmark indicators generated from the survey data. As part of the DPRS Portfolio Budget Statement 2000-2001, the department uses seven client satisfaction performance indicators. Summary data is presented in Table 2.1, with details in Table 2.2.

Table 2.1: Summary client satisfaction performance indicators

Source: DPRS Client Survey 2000

Table 2.2: Client Satisfaction indicators (DPRS Portfolio Budget Statement 2000-2001)

Source: DPRS Client Survey 2000

¹   Clients who indicated they were satisfied or very satisfied.
²   Average rating across 12 questions - Q1.1-Q1.5, Q1.7, Q2.1-Q2.3, Q4.2-Q4.4
³   Average rating across 6 questions - Q1.1-Q1.5, Q1.7
⁴   Average rating across 3 questions - Q2.1-Q2.3
⁵   Average rating across 3 questions - Q4.2-Q4.4
⁶   Clients who indicated they were satisfied or very satisfied
⁷   Average rating across 4 questions - Q3.1, Q3.2, Q3.5 and Q3.8
⁸   Average rating across 2 questions - Q3.1 and Q3.2
⁹   Average rating across 2 questions - Q3.5 and Q3.8

While recognising this strong base of support for the quality of DPRS services, clients also highlighted that there was scope for improving services. Key points included:

• ensuring DPRS remains proactive in anticipating the needs of Senators and Members and the Parliament

  Two-thirds (67 per cent) of clients agreed that DPRS was proactive in anticipating the needs of Senators and Members and the Parliament, with a further one-quarter (26 per cent) satisfied that the response of DPRS was adequate. Greater concerns were raised by senior managers of Parliamentary Departments, with less than half (47 per cent) fully satisfied.

• ensuring DPRS remains innovative in findings more cost-effective ways of meeting clients' needs

  Two-thirds (66 per cent) of clients agreed that DPRS was innovative in findings more cost-effective ways of meeting clients' needs, with a further one-quarter (28 per cent) satisfied that the response of DPRS was adequate. Greater concerns were raised by senior managers of Parliamentary Departments, with less than one-third (31 per cent) fully satisfied. However, it is important to note that over one-third of clients indicated they did not have enough information to make an informed assessment on this question.

• continuing to meet the needs of Parliamentary Departments

  Only around half the senior managers from Parliamentary Departments who responded to the survey were fully satisfied that DPRS services were easily accessible (52 per cent) or that DPRS appreciated the time constraints that their officers were subject to (55 per cent).

Specifically, clients raised a number of issues relating to service improvements in the five core areas of DPRS service delivery. These details are presented in the following sections, and cover:

• transcription services (Section 2.2);
• broadcasting services (Section 2.3);
• computer-related services (Section 2.4);
• client liaison and support (Section 2.5);
• other communication services (Section 2.6).

2.2   Transcription services

Clients were generally satisfied with the transcription services provided by DPRS including pink and green drafts, electronic proof Hansard and Committee transcripts. All key aspects of these transcription services had a base of client satisfaction close to 90 per cent or better (Table 2.3), although the proportion of very satisfied/satisfied clients (the DPRS benchmark performance indicator) was lower in regard to the timeliness of the Committee transcripts (67 per cent) and user-friendliness of electronic Hansard (67 per cent).

Table 2.3: Client satisfaction with transcription services

Priority issues for clients regarding transcription services included:

• maintaining the timeliness of Pink and Green drafts

  Seventy-seven per cent of Senators and Members were satisfied with the timeliness of the Pink and Green drafts which are intended to be ready within two hours of delivering a speech. A further 17 per cent of Senators and Members who responded to the survey indicated that the timeliness of the drafts was adequate, with 4 per cent [3 respondents] dissatisfied [Client Survey Q1.1].

  The reliability of Pinks and Greens was a high priority for Senators and Members who often depended on their timely delivery for the preparation of press releases. Instances of late or non-delivered Pinks and Green had led some Senators and Members to question the reliability of the service, and in a very small number of cases, had resulted in the perception that there had been a deterioration in service quality over the last 12 months.

  It is likely that future levels of client satisfaction with transcription services will be strongly linked to maintaining the reliability of delivery of Pink and Green drafts.

• timeliness of the Committee transcripts

  Less than half (41 per cent) of Senators were fully satisfied with the timeliness of the Committee transcripts, although a further 45 per cent of Senators who responded to the survey indicated that the timeliness was adequate, with 14 per cent [4 respondents] dissatisfied [Client Survey Q1.3]. The issue for Senators is that they would like committee transcripts available on a 'next day' basis rather than the usual 1-3 day minimum for priority transcripts. Some Senators indicated that the current system did not adequately recognise the short reporting deadlines that Committees were often subject to.

  Concerns about the timeliness of Committee transcripts were only raised by a small number of Members, with 74 per cent expressing satisfaction. However, several noted the need to improve the timeliness of transcripts from Senate Estimates.

  For both Senators and Members, their views on the timeliness of Committee transcripts is clearly influenced by speed of production of Chamber transcripts.

  In contrast, among the 18 Committee secretaries who responded to the survey, 89 per cent were satisfied with the timeliness of transcripts, with just one respondent expressing dissatisfaction. This is likely to reflect the fact that the timeframe for Committee transcripts are negotiated directly with Committee secretaries. Committee secretaries indicated that transcripts were almost always delivered within the negotiated timeframe of either 1-3 days (priority), 3-5 days or 10-15 days, albeit at the end of the selected time period.

  This client feedback highlights the importance of managing difference between the timeliness of Committee and Chamber transcripts, either by reducing such differences if feasible within available resources, or managing client expectations though promotion of clear service standards which highlight transcription priorities.

• utility of the user interface of electronic Hansard

  Two-thirds (67 per cent) of clients were satisfied with the user-friendliness of the electronic version of Hansard, withwith a further 22 per cent of clients responded to the survey indicating that the interface was adequate, and 11 per cent (13 respondent)s] dissatisfied [Client Survey Q1.6].

  The issue for some clients is the cumbersome user interface which they perceive as neither user-friendly nor intuitive. In practical terms, these clients indicated they experienced considerable difficulties in quickly locating extracts they were looking for, which resulted in them either using paper-based versions or getting the Parliamentary Library to do their searching for them.

  Others indicated that they had problems obtaining the correct formatting when printing and exporting text.

  Several clients commented that they preferred the earlier version of the Hansard database which they felt was a faster and easier system to search.    

  While there were mixed views on the future need for the hard-copy version of Hansard, the client feedback highlights that increased usage of the electronic version may require a higher degree of responsiveness to customer issues and concerns.

• responsiveness of x2020 operators to specialist transcription enquiries

  84 per cent of clients were satisfied with the ease of getting information, help or resolving problems relating to transcription services using the x2020 help-desk or through direct contact. A further 10 per cent of clients who responded to the survey indicated that the available support was adequate [Client Survey Q1.7].

  A small proportion of Senators and Members (3 per cent - 3 respondents) and Committee Secretaries (13 per cent - 3 respondents) had strong concerns about the lack of responsiveness of x2020 help-desk operators to specialist transcription enquires. They perceived that the x2020 help-desk often resulted in 'double-handling' on complex queries because they were not dealing directly with a transcription service specialist. These clients cited examples of slow and inaccurate responses after calling x2020, and frustrations with having to restate problems after the x2020 operator eventually referred the call to specialist Hansard staff.

  While the overall level of dissatisfaction was low, this client feedback highlights the importance of training for help-desk operators and the need for appropriate referrals of complex queries.

Other issues relating to transcription services raised by individual clients included:

• problems with the use of page numbers rather than paragraph numbers for referencing Hansard text;
• insufficient time allowed for Senators and Members to provide feedback on Pink and Green drafts;
• concerns about recent changes to the procedure for notifying editorial changes to Pinks and Green drafts (requirement to provide written rather than verbal advice);
• concerns that no confirmation is provided as to whether editorial changes have been incorporated or not;
• problems with receiving different sections of Pink and Green drafts out-of-order; and
• need to access notice papers and disallowable instrument papers in electronic format.

2.3   Broadcasting services

Clients indicated a high level of satisfaction with the broadcasting services provided by DPRS including radio and television coverage of parliamentary proceedings and customised broadcasting services (eg press conferences, pieces-to-air, tapes of proceedings) All aspects of these broadcasting services had a base of client satisfaction close to 100 per cent (Table 2.4).

No significant issues were raised by clients regarding potential improvements to broadcasting services, although a small number of Senators and Members and Committee Secretaries mentioned the need for improved responsiveness of x2020 help-desk operators to specialist broadcasting enquiries.

Table 2.4: Client satisfaction with broadcasting services

Issues relating to broadcasting services raised by individual clients included:

• more widespread use of webcasting of parliamentary procedures, including committee proceedings;
• intermittent problems with the signal dropping out for some television services (eg BBC World Service)

2.4   Computing and information services

Clients were generally satisfied with most of the computing and information services provided by DPRS. All key aspects of these services had a base of client satisfaction 85 per cent or better (Table 2.5), although the proportion of very satisfied / satisfied clients (benchmark PI) was lower in regard to a number of key aspects of service availability and delivery.

Table 2.5: Client satisfaction with computing and information services

Priority issues for clients regarding computing and information services included:

• quality and reliability of computing facilities

  About three-quarters of Senators and Members were satisfied with the range and quality (75 per cent) and reliability (73 per cent) of computer facilities in their Parliament House offices. A further 19 per cent of clients who responded to the survey indicated that the available computer facilities were adequate, with 6 per cent [5 respondents] dissatisfied [Client Survey Q3.1 and Q3.2]. However a more detailed analysis of the feedback from Senators and Members indicated that these satisfaction rating related not just to DPRS computer facilities, but were influenced by their satisfaction with the computer equipment provided by the House Department and the computer services funded by DoFA in electorate offices.

In contrast, higher levels of satisfaction were found among Parliamentary Officers, who were more familiar with the specific computing services provided by DPRS.

Clients who raised concerns about the quality and reliability of computing facilities cited:

• problems with the slow log-in time (time between turning on the computer and being able to access applications and files)
• a lack of 'speed' for the computers they use in Parliament House, compared to other computers they have access to
• examples of system 'crashes' where they have to re-boot or the computing facilities are temporarily not available.
• seamless connections between different computing facilities

  While there is a high base of satisfaction with DPRS computer facilities, ongoing attention is needed to ensure 'seamless' connections between computing facilities provided by the different agencies. Senators and Members had little interest in the different administrative arrangements covering the provision of computing services and highlighted an important ongoing role for DPRS as a broker of services to ensure available computer facilities are tailored to meet individual needs.

  Awareness of the proposed Extended Parliamentary Network was relatively low, with Senators and Members highlighting the need for seamless connections between computer facilities in Parliament House and electorate offices, and a common computer interface. Currently, various 'quick fix' solutions were being implemented in individual offices to overcome current difficulties.

• remote and mobile access

  Only around half the clients (54 per cent) who responded to the survey were fully satisfied with the ease of remote and mobile access to the Parliament House computer network. A further 33 per cent of clients indicated that remote and mobile access was adequate, with 13 per cent [13 respondents] dissatisfied [Client Survey Q3.3].

  The key issue for clients are the:

  • poor speed of mobile connections;
  • cumbersome log-on procedures;
  • difficulties finding appropriate telephone connections.
  • Senators and Members who were experienced computer users tended to express the highest levels of frustration with the current remote and mobile access facilities.

   

• critical support to the Parliament

  Not all senior managers in Parliamentary Departments were fully satisfied that adequate DPRS resources were available for supporting the critical business systems of the Parliament (eg Table Office). For example, only around half (52 per cent) of these clients were fully satisfied with the accessibility of DPRS services, including the provision of after-hours support for critical business systems.

  While these managers indicated a strong base of satisfaction with the overall reliability of DPRS computer facilities (97 per cent) and the ease of getting help to resolve computer problems (97per cent), they highlighted the need for strong inter-Departmental partnerships to maintain support for the critical business systems of the Parliament.

• ParlInfo information services

  Senior managers in Parliamentary Departments and, to a lesser extent, Senators and Members raised a number of issues about improving the user-friendliness of the ParlInfo information service. While, 81 per cent of Senators and Members and 69 per cent of senior managers were very satisfied or satisfied with the usefulness of DPRS information services such as ParlInfo, a number of clients commented that the ParlInfo interface was difficult to use and cumbersome. A small number of senior managers in Parliamentary Departments highlighted a lack of responsiveness from DPRSs to concerns that had been previously raised about the user interface.

• scheduling and relevance of computer training courses

  While there was a high base of satisfaction with the scheduling (95 per cent) and relevance (97 per cent) of computer training courses, less than two-thirds of Senators and Members were fully satisfied. The key issue related to the difficulty of only offering Parliament House-based training, because of competing priorities when Senators and Members and their staff while in the House. A small number of clients recognised the need for greater tailoring of training to meet the diverse needs of clients, including more experienced and specialised users.

  Specific suggestions for improving the responsiveness of computer training courses included:

  • scheduling training courses on the Friday morning after a sitting period;
  • scheduling training courses as 'breakfast' sessions before Parliament sits;
  • increasing the range of specialist training courses eg tips and tools for confident Word users; introduction to Web Page construction;
  • increased use of computer-based training or self-directed learning packages;
  • help-desk support for a wider range of application (eg Photoshop and Pagemaker).

   

• maintaining the high quality of help-desk support

  The vast majority of clients (85 per cent) were either satisfied or very satisfied with the ease of getting information, help or resolving computer-related problems. Feedback from clients highlights the high degree of responsiveness and professionalism of x2020 operators in resolving computer-related problems. However, a small number of clients cited examples where the service did not fully meet their expectations. Examples included:

  • difficulties getting after-hours support (which was defined as after 5pm in Canberra and after 2pm in Western Australia)
  • slow response time in coming to fix a problem
  • failure to respond to a request for assistance
  • While these examples were isolated, they highlighted the high standard of service that clients are accustomed to, and the potential impact on client satisfaction if any problems arise.

2.5   Client liaison and support

• ninety per cent were very satisfied or satisfied with the professionalism of x2020 support officers;
• eighty-two per cent were very satisfied or satisfied that they were kept informed about the progress of their request by the x2020 support officer;
• eighty-two per cent were very satisfied or satisfied that the outcome of their call to x2020 resolved their problem or meet their need in the best possible way.

Table 2.6: Client satisfaction with the x2020 service

2.5.1 Issues relating to Client liaison and Support

• lack of direct and immediate contact with specialists (mainly in relation to transcription services);
• lack of experience of client liaison officers to understand and resolve the problem;
• lack of customisation of the service to individual client needs.

2.6   Other communication services

Table 2.7: Client satisfaction with other DPRS services

2.6.1 Issues relating to Communication Services

• poor sound reinforcement in committee rooms;
• unpleasant sound of the new Ajoint bells@ system;
• difficulty hearing / seeing Division lights and bells in specific locations eg Great Hall; Senate Courtyard;
• delays in faxes being sent (example of fax stored in the system for three hours before being sent);
• poor quality of resolution for webcasting and intermittent problems with obtaining a connection.

3   Strategic issues for DPRS

Drawing on the results of the client survey, a number of strategic issues were identified for DPRS overall. These DPRS-wide issues provide a framework within which to think about the more detailed feedback on individual services presented in the previous section. The key strategic issues included:

• maintaining a very strong base of client satisfaction

  The research highlights the position of DPRS as a respected, professional service provider to Senators and Members and other Parliamentary clients. This reinforces the finding from the 1998 client satisfaction research, and is consistent with DPRS staff views about the quality of services.

  However in lines with broader public-sector wide trends, DPRS faces the challenge of maintaining client satisfaction in an environment of needing to 'do more for less' with ever increasing client expectations.

  This highlights the critical role of innovation in service delivery, as well as managing client expectations about the relative priority of different DPRS services. The preliminary work completed by DPRS on the Service Development Strategy (March 2000) and the draft Service Charter (August 2000) should be the cornerstone for all DPRS planning efforts to maintain the current strong base of client satisfaction.

• protecting the critical business support functions for Senators and Members and the Parliament

  The client survey reinforces the fact that the services provided by DPRS represent critical business support functions for Senators and Members and the Parliament. It is not just that DPRS services such as transcription services or the information and communication technology services represent highly valued and important services. DPRS services directly impact on the performance of Senators and Members and their offices, and the Parliament, at critical periods or 'moments of truth' - which usually means times which are especially pressured or stressful.

This highlights the need for DPRS to promote a management and workplace culture of identifying and managing risks. A recent DPRS staff survey (DPRS Staff Attitudes Survey 2000, ARTD) found that only 42 per cent of staff believed DPRS was currently effective in promoting a culture of managing risks, highlighting the potential for improvement.

The client's need for integrated, seamless services (across the various administrative arrangements), highlights the need for active partnerships between DPRS and other departments. While the research findings most clearly highlight the importance of these partnerships for computing facilities, priority should also be given to cross-Department working parties related to ParlInfo (particularly involving the Department of the Parliamentary Library) and Committee transcripts (particularly involving the Department of the Senate).

Responses to these 'niggling' problems could be greatly enhanced by the collection of day-to-day feedback on client problems. DPRS staff receive feedback from clients every day through their day-to-day interactions and much of this feedback is informal and is often not systematically reported. A 'comments' field could be added to the DPRS Service Requests Management System to allow DPRS staff to record positive and negative feedback from clients.

• Presiding Officers' Information Technology Advisory Group
• Senators' and Members' support staff feedback group
• Parliamentary committee staff feedback group
• Media clients feedback group
• Senior Management Coordination Group.

Details of these mechanism are described in the Client Feedback Monitoring and Evaluation Strategy prepared as part of Stage 1 of this project.