RPP 18 1 Jenny Owen Partnerships, projects and research skills: using postgraduate student projects for research and evaluation in public services: some reflections

Research Policy and Planning: The journal of the Social Services Research Group – Vol 18 (1) 2000

Partnerships, projects and research skills: using postgraduate student projects for research and evaluation in public services: some reflections

Dr. Jenny Owen, Lecturer in Applied Research in Health and Social Care, Department of Sociological Studies, University of Sheffield; Ms. Josie Tetley, Lecturer, Department of Gerontological and Continuing Care Nursing, School of Nursing and Midwifery, University of Sheffield.

Abstract

This paper examines the effectiveness of postgraduate student projects’ contribution to meeting public and voluntary service needs for evaluation and increased research awareness. We identify four sets of skills to which organizations need access in order to raise awareness and utilisation of research. Reflecting on specific projects, we identify areas in which both students and organizations have benefited. We also identify some of difficulties associated with this kind of project work and propose some constructive solutions.

Introduction

Evaluation, research awareness and evidence-based practice are increasingly important priorities for both professionals in health and social care and for service users’ organizations (Callum et al., 1997). Yet developing research skills in practice or finding resources for systematic evaluation can be problematic – for students, smaller organizations and hard-pressed hospitals or social services departments alike. We reflect here on these issues, using the experience of postgraduate team research projects based on live commissions from local agencies. This approach has advantages in both educational and service development terms. It incurs few financial costs, yet can offer the prospect of real research experience for students and of substantial evaluation or research findings for commissioning organizations. However, as recent correspondence in one relevant journal suggests, there has been little scrutiny of the process or the outcomes involved (letters page, Medical Education, 1999, 33:786).

To examine how much is achieved in practice, we begin by outlining our own points of departure as well as illustrating the outcomes, and the pitfalls, associated with one recent project. This was carried out by a small team of master’s degree students, with the dual aims of consolidating research skills and delivering a competent evaluation study to contribute to service development. Comparing this with a range of recent postgraduate projects, we then reflect on some of the wider questions and implications raised by our own experience of undertaking and supervising research in this way. How

much do the commissioning organizations really benefit? And, how successful is this approach in developing research skills which are appropriate to meet the somewhat complex demands arising within health and social care?

Starting-points

One of us came into research from a nursing background and recently completed a master’s degree in applied research; the other had been a community worker, an academic researcher and a health service trainer, before teaching on the same master’s degree programme. The programme combines taught modules with an individual dissertation and – less conventionally – with a team-based research project. This combination is intended to provide a breadth of experience to consolidate both applied research skills and skills in team work, negotiation and communication (Hall and Hall, 1996). Most students participate on a part-time basis, while employed in social work, nursing, midwifery or other professions.

To complete team projects like those described below, students take responsibility for a twelve-month research process, from design, in collaboration with a commissioning agency and a supervisor, through to data collection, analysis and dissemination. This approach is based on a model of ‘self-directed learning’ (Candy, 1991) intended to enable students to benefit from taking real initiatives themselves within a structured and supportive framework. The aim here is to gain access to those features of the research process which the methods texts tend not to explore, or to articulate in full: the steps in

data analysis, for example, which even the most systematic of accounts rather tantalisingly refer to as involving creativity and intuition (Ritchie and Spencer, 1994). In terms of learning to be a researcher, there seem to be many maps and guidebooks – but there is still a considerable need for more opportunities to try the journey out in person, in relative safety.

As many of the projects undertaken conducted in this way are carried out in health and social care settings, we first review some relevant policy issues.

Teaching and learning research skills in health and social care: policy changes and constraints

Policy developments in health and social care continue to shape the particular ways in which organizations seek to develop and use research expertise. There have been many analyses of the mixed and contradictory features of the reforms of the 1980s and 1990s. The separation of purchaser and provider roles, in the context of what came to be termed a ‘quasi-market’ (Ferlie et al., 1996), began to erode the free circulation of research and information in some respects, for example, between competing hospital Trusts. It has also, however, been associated with a push for evaluation processes which support increased professional accountability and ‘user’ representation (Barnes, 1997). These are genuinely ambiguous developments, open to both creative local interpretations as well as to crude and tokenistic implementation. In the last decade issues of contracting in health and social care have been further complicated by the increased use of voluntary sector services. Commentators such as Barritt (1993) have argued that an evolving ‘contract culture’ undermined the scope of research and evaluation. A particular concern has been that contracting processes would become dominated by simplistic cost benefit analyses, despite evidence that voluntary services are best at tasks that generate little or no financial profit (Kendall et al., 1996; Ware, 1993).

Under the current Labour government, many of the same tensions and ambiguities seem to be continuing. While there is a move away from the ideology of competition, there is increasing emphasis on regulation and performance management driven by central policies and monitoring processes. At a local level, this policy context still translates into two problems: a shortage of real resources for research and a pressure to ‘measure the measurable’ in the short-term, at the expense of more complex aspects of service quality and development (McIver and Meredith, 1998).

Recent studies suggest that specific skills in organizational development, in addition to a sound basis of data collection, analysis and dissemination skills, are important in supporting research-based practice (Eve et al., 1997; Eve and Hodgkin, 1997). We would suggest that organizations need access to four distinct sets of skills, whether internally or through partnerships with others, in order to make progress in developing research utilisation, awareness, and activity:

  • a broad repertoire of data collection and analysis skills, ranging from designing and analysing basic questionnaires to conducting focus groups and interviews ;
  • skills in critically evaluating research, with a view to identifying opportunities for implementation and further investigation;
  • project management skills;
  • skills in providing feedback and facilitating change on the basis of research.

How realistic is it, though, to expect postgraduate research projects to provide a good grounding in these areas and to deliver useful outputs for local agencies? We now review one recent example in depth, and then compare key aspects with the whole range of team projects we have supervised in the last two years.

‘Coffee, biscuits and cat stories…’: evaluating a voluntary sector ‘sitting service’ for carers

Our in-depth example is an evaluation of a sitting service for carers of elderly people, using qualitative methods (Tetley and Willis, 1995). In this section we illustrate the ways in which the study complemented national research, providing local evidence for a voluntary organization to use in negotiating a local contract and in planning internal training and development. This study enabled the student team to gain experience of designing an appropriate study for a sample of family carers, volunteer sitters and professionals. It also illustrates some of the risks and uncertainties which can characterise postgraduate projects, which we come back to later.

The ‘Local Sitting Service’ (LSS) is a voluntary organization providing a free respite service for carers of elderly people in Sheffield. In 1994, LSS commissioned a team of four MA students to evaluate their service. The expressed aims of the LSS are ‘to provide short periods of respite care to those caring for an elderly person at home, to enable carers to have space for themselves’.

National surveys in the 1990s estimated that there were over six million informal carers looking after dependant relatives at home (OPCS, 1992; Sadler, 1990). Locally, Craig (1993) found that Sheffield had a higher than average elderly population. More specifically, 34% of the population in the area covered by the study were elderly. Within this particular elderly population, approximately 700-850 needed almost constant care, 1900-2000 needed help with washing and dressing, 7,500 needed help with shopping and cleaning and approximately 2700 of those over 65 had dementia. Craig further estimated that some 4,000 carers were heavily involved in care in the geographical area covered. However, in 1993 only 50 were in touch with LSS.

Taking funding needs and pressures into account, the study aimed to:

  • provide LSS with information about the benefits and the outcomes for those using the sitting service; and
  • examine more closely the needs of current and potential users and the ability of LSS to meet these.

As the request was to evaluate, rather than to produce generalisable research findings, the team adopted a pluralistic approach – a small sample, combined with a range of methods of data collection, could address the needs of different stakeholders. Moreover, the use of mixed methods would produce findings from many perspectives – users, carers and funders, for example – enhancing the validity of the study (McEwan, 1989; Nolan and Grant, 1992, 1993; Smith and Cantley, 1988, 1985). The team carried out telephone interviews with family carers (n=15) and with potential referrers (n=27), addressing local knowledge and experience of the service. In addition, ten LSS volunteers were asked to complete semi-structured diaries on four occasions describing the benefits, the enjoyable aspects and the difficulties that they encountered during sitting sessions. Thirty two out of forty diaries (80%) were returned. The team also collated demographic data in relation to referrals and service use.

Findings

The team gathered evidence in relation to both the stated aims. In relation to benefits and outcomes, there was evidence of the range and flexibility of the service volunteers provided, reinforcing the message from other studies (Nolan and Grant, 1992; Ware, 1993). Data from volunteers’ diaries produced insights into their own experience as well as observations about the views of carers and people cared for. For example, handover on arrival and departure were important times for carers, not just for briefing the volunteers about practicalities but also for expressing more general worries about their caring situation, as in this sample comment:

‘Arrived 9.30. Briefed as follows:- “Workmen in kitchen etc. Cared for very confused today!”‘

Whilst the volunteers’ role is mainly to keep the elderly person company, the diaries showed them undertaking a wide range of social, physical and domestic activities. They also demonstrated that activities were tailored to individuals, bringing pleasure as well as dealing with physical care:

“[she enjoys] coffee and biscuits… My cat stories…”

” I had taken along a picture which I had painted, she takes a great interest in everything and was pleased to see it.”

Volunteers illustrated the ways in which having a break from the caring relationship enabled both carer and cared for to cope emotionally and physically, with access to different things to talk about:

” A chance to explore her guilt feelings about putting her husband into respite care”

“Someone different to talk to about problems of caring, a chance to leave the caring to someone else.”

Telephone interviews with carers provided further evidence of service benefits, including increased social contact both for themselves and for the person cared for:

“chatting and reminiscing…”

“getting a break without worry…”

“being able to relax…”

One carer felt that the service “has changed my life.”

Most carers had no family support and were paying for extra help, such as nursing or night sitting. Satisfaction with the service was very high overall, complementing findings from other studies. The value of respite care is well documented as an intervention to help carers better cope with the long term commitment to caring (Ayling, 1993; Budd, 1990; Gordon and Lawrie, 1990; Sadler, 1990; Gaze, 1990; Thom and Voss, 1987). Those who suggest respite as a strategy for carer support recognise that it must be flexible to meet individual need. A number of studies specifically suggest that such flexible provision will primarily come from the family and/or voluntary and private organizations (Ware, 1993). This study helped to document the current scope and extent of voluntary input.

The second prominent theme in the team’s findings concerned user and carer needs, and the ability of LSS to meet these. Key issues were volunteer training and development and publicity. Diary data illustrated problems encountered during sitting, which volunteers did not always feel well-equipped to respond to – for example, toileting, communication and confusion:

“Unfortunately today he was rather confused, I had a job deciding what was fact and fiction.”

“He usually wants me to ring her (neighbour) to have a chat. Sometimes he pesters me to go to her house.”

“Cared for had bowel problems today. Getting him on/off commode. Cleaning up etc.”

Carers also felt that the service should be extended to take in a wider age range (starting at 60 instead of 65) and the whole city instead of one sector. (Indeed since the study was completed, LSS itself has expanded; a similar service has also been established city-wide).

On the issue of publicity, telephone interviews with potential referrers to the service revealed a need for development. Of the 27 interviewed, eighteen (67%), had heard of LSS. However, nine out of ten family doctors sampled (90%) were not aware of the service although all were interested and requested more information. These findings prompted a recommendation that the LSS should reconsider their communication strategies. In addition to the poor knowledge of the service among family doctors, district nurse use of LSS was inconsistent. In fact, the most enthusiastic referrers were district nurses who had received a talk from the LSS co-ordinator, which suggested that the ‘personal touch’ was most effective. As coordinators’ time was severely limited, the report suggested additionally involving members of the management committee or trained volunteers in giving talks.

When the study was conducted, the annual cost of LSS to Social Services was £18,300 for part-time salaries and office use. However if one carer were to cease caring, the estimated cost to the health and social care services would have been £15,080 p.a. for a nursing or local authority home place, or £11,024 for a private residential home. This is a crude form of costing individual care and most participating carers received additional statutory support. However, our finding that informal care saves public spending is supported by Oliver (1985), for example, who estimated that if 1% of carers of the elderly gave up, the cost to health and social services would increase by 20%.

Overall, the evaluation indicated that LSS offered a valuable and cost-effective service to carers under stress. The findings were used in negotiations with Social Services and contributed to an eventual move from annual grant aid funding to a more stable partnership contract. However, the difficulties reported by the volunteers, such as toileting, general moving and handling and coping with mental health problems, were a concern. Meeting these needs was seen as essential to ensure safety and confidence in the sitting service. Thus the study recommended that LSS develop a new training programme, with regular updates, for all volunteers.

In terms of student learning, the project certainly provided in-depth experience in qualitative research design and methodology. Some aspects went as planned. Telephone interviews worked well with carers, who were easy to contact this way, providing effective data collection within limited researcher time. They were also successful with potential and actual referrers to LSS. In contrast, the team discovered that district nurses rely a great deal on answering-machines, leading to delays in confirming arrangements and to some confidentiality problems. There was no mention of answer phones in the methodological literature, so this problem had not been anticipated. Family doctors were the most difficult group to contact, although the team had allocated time to deal with this often reported difficulty (see, for example, Sibbald et al., 1994). The sample also highlighted issues regarding validity and reliability. For instance, time pressures meant that the sample of volunteer sitters was drawn from those who sat most frequently, which may have biased the findings. Less frequent ‘sitters’ might have reported a wider range or a different incidence of problems.

In relation to analysis, a great deal of rich data was generated by the diaries which enhanced the team’s understanding of events and interactions during sitting sessions. Analysing detailed diary data certainly underlined the importance of systematic record-keeping for team-based research. One team member kept an ‘audit trail’ as she analysed the diaries which enabled another team member to identify relevant extracts easily at the report-writing stage (see Ritchie and Spencer, 1994, for a related example).

Project management and contingency planning presented the biggest difficulties. The initial team of four shrank when two members left the MA programme for personal reasons. Another member later moved away and continued their study through distance-learning, whilst the remaining group member also had a baby (planned!) in the middle of the research. Communicating at a distance, with one team member conducting telephone interviews whilst caring for a new baby, became another challenge. The team negotiated a reduction in the range of stakeholder views to be included with LSS. This limited the study, but kept the workload realistic. From the outset the team chose not to gather data from people being cared for, due to time and resource constraints. Moreover, when the group was reduced to two, the views of Social Services funders and planners had to be omitted. In order to feed back the findings and difficulties encountered, one team member built up a personal dialogue with LSS. As this was a small organization, this proved manageable, with one identified person responsible for following up evaluation findings and recommendations.

While these may seem extreme circumstances, there were also benefits. Conducting a piece of commissioned but un-funded research can help postgraduate students learn to deal with the difficulties of real life research, as well as trying out previously unexplored research methods such as diaries and telephone interviews. With supervision and support, this can lead to greater confidence rather than to demoralisation. One of the authors of this article joined an EU funded, five-country research project after completing the MA, where planning and communicating under pressure have proved to be invaluable skills.

An overview of recent commissioned projects: comparisons and discussion

Over the last two years, we have supervised nine substantial postgraduate applied research team projects. Four have addressed organizational or service development initiatives in health service contexts (for example, the introduction of shared governance for nurses in an NHS trust and a review of strategies for recruiting sperm donors for an NHS fertility clinic); three have been evaluations for voluntary organizations (for instance, evaluating a new drop-in centre for young people and a pilot study on models of local consultation over regeneration); one focused on care management training in social services and one on the use of physical restraint in mental health care (completed by a distance-learning student in Kuwait). How do their outcomes, for organizations and students, compare with the example described in depth above?

For organizations, all nine projects produced detailed written reports, to agreed deadlines. Of these, five focused primarily on local needs, while four produced analyses relevant to national debates and policies (see, for example, Gavin et al., 1999).

Four projects were negotiated with organizations within which a student team member worked, so there was someone internal to ensure further feedback and discussion. Of the five others, three organizations initiated feedback and discussion by requesting a formal presentation of research findings. Another had focused on producing an article for publication rather than feedback to an organization and a draft paper was submitted to an appropriate journal. In the final case, relations between the student team and the commissioning (voluntary) body deteriorated during data collection. Here early evaluative findings revealed some major internal staffing and service quality problems, for example, repeated unplanned closure of facilities. After much deliberation, the team’s draft report was circulated to respondents for comments. When none were received, it was circulated to an appropriate, broader steering group. After a six month delay, team members were eventually invited to a steering group meeting to discuss the findings which proved to converge with observations from other sources. Overall, there is evidence of concrete action in response to these reports and discussions in seven of the nine sites, ranging from minor improvements to a local training programme to the adoption of a new model for community consultation about local economic regeneration.

In common with the LSS example, then, short-term outcomes were positive in most of these projects – usable research or evaluation findings were produced at minimal cost. In the case where working relationships broke down, the short-term outcome appeared very damaging. The experience produced severe demoralisation among the student team and brought acrimonious divisions to the surface within the commissioning organization. Here, student team members’ own knowledge of the service context, combined with supervisory support, made it possible eventually to identify a feasible approach to damage limitation, including the circulation and discussion of controversial findings. Examples like these provide cautionary but useful tales, both for potential commissioners and academic supervisors. Tackling such difficulties requires a shift in role, from conventional research planning to in-depth consultancy and conflict resolution. By definition, these shifts tend not to be planned but it is important to acknowledge and review them.

In terms of outcomes for postgraduate student teams, these projects (like the LSS example) did offer sustained opportunities to improve research skills in practice. This was most successful where the student team opted to explore a range of both quantitative and qualitative methods rather than keeping to their immediate ‘comfort zone’. Our experience is that qualitative methods tend to be perceived as the ‘comfort zone’ at the design stages by most practitioners (although not when it comes to analysis). The onus is on supervisors to keep quantitative options on the agenda too, in a context of appropriate support. The opportunity to reflect and to learn from mistakes as well as achievements has also been important:

The department are already talking to us about coming back in six months to move specific projects on, but that is too difficult to contemplate. I wish I had done an MA programme before I came here, as I would have baselined everything and kept field notes from day one, but the fact that I think that now is a measure of the success [of the programme]. (Bohannon, 1999).

As with the LSS example again, team members welcomed the opportunity to explore a range of techniques for data collection. In the sperm donor recruitment study, for example, the team was able to compare the results of an internal University e-mail survey with face-to-face questionnaire administration – discovering that the former generated more high-quality data which was amenable to rapid analysis within the time available. This was a valuable opportunity to assess an increasingly common methodological tool. However, the advantages had to be balanced against the limitations, which included respondents being identifiable from their addresses and enormous variations in the length and quality of responses (for more discussion of these issues, see Selwyn and Robson, 1999).

Developing critical evaluation skills was also a common feature, as would be expected within an MA programme. In cases where student team members were researching within or for their own organization, there was clearly scope to disseminate these skills more widely. In other instances, while access to a postgraduate team project made these skills available for the purpose in hand, there was no real scope for longer-term transfer of skills or awareness. However, successfully-completed projects, in our experience, can form the basis of published articles (Gavin et al., 1999).

A more complex picture emerges in relation to teamwork and project management skills. Student teams’ reflections on the former emphasised the importance of having an explicit framework within which to discuss roles and to acknowledge and resolve differences – illustrated by sample comments from one cohesive team:

“We were not afraid to speak our minds and express opinions… Frequent communication has proved essential. Although I found it nerve-wracking presenting pieces of my work to the scrutiny of peers, I feel this is good preparation for the future.”

The group very quickly established a system that if one person had a problem, another could step in… We used our time in supervision to discuss ongoing problems and to make decisions when there were differences of opinion. (Ash et al., 1998).

In relation to project management, only three of the nine studies mentioned above proceeded according to the original plan. One of these involved analysing previously-collected data and two were carried out for large and relatively well-resourced organizations (NHS trusts), with an internal tradition of research utilisation and excellent liaison arrangements. Five of the remaining six project teams had to negotiate revisions which were less drastic than those in the LSS example, but which were still demanding and sometimes stressful. For instance, the team studying options for community consultation in a local neighbourhood found that pre-arranged interviews often fell through, while ‘dropping in’ generated reliable opportunities to meet residents informally. This made data collection more time-consuming than anticipated. Generally, adult learners, combining postgraduate study with work and family commitments, have less flexibility than funded researchers; this means that developing project management and teamwork skills alongside formal research skills is crucial.

Finally, we noted the importance of feedback and facilitation skills, in connection with the dissemination and take-up of research findings, in our opening section. Observations from both students and supervisors suggest that conventional academic supervision and assessment frameworks provide insufficient recognition of the effort and skills involved. Originally, the team project MA module discussed here only required students to submit a conventional academic research report, similar to a dissertation (although shorter). Yet completing the projects referred to above required student teams to build up and sustain a relationship with a commissioning organisation, to provide both an appropriate context for research access and to discuss the findings. Informal aspects, keeping in touch regularly by phone or email, for example, have proved as important as formal contracts and meetings.

The real opportunities for follow-up and dissemination offered by these projects need to be integrated into learning and assessment processes, complementing conventional written work. We now align assessment and research dissemination requirements more closely. Student teams include outputs produced for research commissioners within their academic submission and successful completion is conditional on evidence that findings have been discussed and disseminated (with the qualification that students are not penalised if the commissioning organization withdraws from or defers agreed arrangements). Such evidence can include a range of materials, with lots of scope for imaginative approaches; this supports new national commitments to service user-involvement in research processes (Department of Health, 1999). This in turn requires supervisors to address feedback and facilitation skills – through practical examples of dissemination processes or interim opportunities to present draft findings (to fellow-students, for example) – and to give and receive feedback in a supportive context.

There is then, for most organizations, a clear benefit from the access to research skills and completed studies arising out of these forms of partnership arrangements with postgraduate student teams. However, there are risks too. For commissioners there may be missed opportunities for follow-up or, less commonly, unwelcome findings which reveal internal divisions. For student teams, disadvantages lie in the stress and uncertainty of combining work and study in what can be a volatile applied research context. This raises the question of what students, their academic supervisors and commissioning organizations can learn from this kind of experience:

  • for postgraduate students, it is essential to recognise the importance of project planning and management and of clear communications within a team. Few research projects proceed entirely according to plan and time needs to be earmarked in the early stages for teams to build confidence in addressing workload difficulties, research access problems or changes in members’ circumstances;
  • for academic supervisors, there is a concomitant need to integrate project management models and skills within supervision processes and to facilitate team communications actively, particularly (again) in the early stages. Later on the emphasis needs to shift to reviewing specific opportunities for feedback and dissemination and to providing recognition for these activities within formal assessment processes;
  • for commissioning organizations the initial request usually includes a named liaison person. Developing this into a short contract, including a framework for interim feedback, a defined audience, timescale and format, would offer safeguards. It would help ensure that any ambiguities are discussed at an early stage and that eventual reports remain relevant and amenable to follow-up.

Conclusion

Projects of the type described here offer a genuine opportunity to ‘try the journey out in person’ in applied research. However, the risks include a degree of stress and uncertainty for student researchers and some missed opportunities in terms of contributions to research-based practice for commissioning organizations. Reflecting on recent experience, some practical steps can be identified to help reduce these risks. Whilst unremarkable in themselves, they prompt reconsideration of various assumptions about both academic assessment processes and research planning and dissemination. In a context of heavy workloads and tight deadlines for academics and practitioners alike, these points are all too easily taken for granted or overlooked.

References

Ash, D., Gavin, M., Wakefield, S. and Wroe, C. (1998) Shared Governance. Unpublished MA project report. Sheffield: Sociological Studies, University of Sheffield.

Ayling, H. (1993) ‘Blessed Respite.’ Health Service Journal, November:33.

Barnes, M. (1997) Care, Communities and Citizens. London: Longman.

Barritt, A. (1993) ‘Contract Culture or Anarchy? Evaluation and the Voluntary Sector.’ Research Policy and Planning, 11(½): 10-15.

Bohannon, N. (1999) Restraint in a mental health unit. Unpublished MA report. Sheffield: Sociological Studies, University of Sheffield.

Budd, L. (1990) ‘The long night.’ Community Outlook, May: 27-28.

Callum, N., DiCenso, A. and Ciliska, D. (1997) ‘Evidence-based nursing: An introduction.’ Nursing Standard, 11(28): 32-3.

Candy, P.C. (1991) Self-direction for Lifelong Learning. San Francisco: Jossey-Bass.

Craig, N. (1993) The information needs of carers of the elderly in the South West of Sheffield. Unpublished MSc thesis. Sheffield: Sheffield University.

Department of Health (1989) Caring for People. Community Care in the Next Decade and Beyond. London: HMSO.

Department of Health (1999) Patient and Public Involvement in the new NHS. London:HMSO.

Eve, R., Golton, I., Hodgkin, P., Musson, G. and Munro, J. (1997) ‘Lessons from ‘FACTS’.’ Occasional paper 3/97, School of Health and Related Research, University of Sheffield.

Eve, R. and Hodgkin, P. (1997) ‘Getting the Message Across’ in Baker, R. (ed) Implementing Clinical Change with Audit. London: Wiley.

Ferlie, E., Ashburner, L., Fitzgerald, L. and Pettigrew, A. (1996) The New Public Management in Action. Oxford: Oxford University Press.

Gavin, M., Ash, D., Wakefield, S. and Wroe, C. (1999) ‘Shared Governance: time to consider the cons as well as the pros.’ Journal of Nursing Management, 7: 193-200.

Gaze, H. (1990) ‘A Good Night’s Rest.’ Nursing Times, 86(5): 16-17.

Gordon, D.S. and Lawrie, R. (1990) ‘Night Care: The evaluation of an experimental night sitter service in Aberdeen.’ University of Birmingham Social Services Research, Department of Social Policy and Social Work, No 1: 24-36.

Hall, D. and Hall, I. (1996) Practical Social Research: Project Work in the Community. London: Macmillan.

Kendall, J. and Knapp, M. (1996) The voluntary sector in the UK. Manchester: Manchester University Press.

McEwan, R.T. (1989) ‘Issues in evaluation: evaluating assessments of elderly people using a variety of methods.’ Journal of Advanced Nursing Studies, 14: 103-110.

McIver, S. and Meredith, P. (1998) ‘There for the Asking’, Health Service Journal, 19.2.98: 26-27.

Nolan, M.R. and Grant, G. (1992) ‘Respite Care: challenging traditions.’ British Journal of Nursing, 1(3): 129-133.

Nolan, M.R. and Grant, G. (1993) ‘Service Evaluation: time to open both eyes.’ Journal of Advanced Nursing Studies, 18:1434-1422.

Office of Population and Census Survey (1992) General Household Survey: Carers in 1990. OPCS Monitor. London: Government Statistical Service.

Oliver, J. (1985) ‘Eight days a week.’ Nursing Times, January 16th: 23.

Ritchie, J. and Spencer, E. (1994) ‘Qualitative Data Analysis for Applied Policy Research’ in Bryman, A. and Burgess, R.G. (eds) Analyzing Qualitative Data. London: Routledge.

Sadler, C. (1990) ‘Breaking Point.’ Nursing Times, June 13th, Vol 86(24): 21.

Selwyn, N. and Robson, K. (1999) ‘Using email as a research tool’. Social Research Update 21, Guildford: University of Surrey, http://www.soc.surrey.ac.uk/sru.html.

Sibbald, B., Addington Hall, J., Brenneman, D. and Freeling, P. (1994) ‘Telephone Verses Postal Surveys of General Practitioners: Methodological Considerations’. British Journal of General Practice, July 1994:297-300.

Smith, G. and Cantley, C. (1985) Assessing Health Care: A Study of Organisational Evaluation. Milton Keynes: Open University Press.

Tetley, J. and Willis, C. (1995) SWASS: an Evaluation. Unpublished MA project report. Sheffield: Sociological Studies, University of Sheffield.

Thom, L.M and Voss, W (1987) Volunteers Providing In-Home Respite Care For The Homebound Elderly. Journal of Community Health Nursing, 1987, 4(2):93-98.

Ware, P. M. (1993) The Implications Of The National Health Service And Community Care Act (Part III) 1990, For Voluntary Sector Services For The Elderly. Unpublished MA thesis. Sheffield: Sheffield University.