Getting Started | Partnership and Management Arrangements | Awareness Raising and Marketing | Involving Local People | Making Information Accessible | Next Steps - Embedding The Pilots Next Steps - Embedding The Pilots Introduction A major worry for the managers of pilot projects has been that they will generate a momentum that cannot be sustained. Given the target groups for this initiative, this has been an even more serious concern. When individuals have had negative experiences with learning previously, it will take consistent effort to break down barriers; if the interest then generated is subsequently frustrated, the position will be even more difficult to retrieve. This has placed the focus sharply on evaluating impact and creating a business case for future funding. How can the effectiveness of such projects be assessed? Can good practice be embedded? How can future funding be secured? These and other points have been researched by twelve demonstration outreach projects, funded by the Department for Education and Employment (DfEE) during 1997 and 1998. These were genuinely creative projects whose overall aim was to develop innovative ways of improving access to information and advice on learning opportunities for adults in disadvantaged communities. SWA Consulting was commissioned to draw out the key learning points from the twelve projects. A parallel report entitled "Demonstration Outreach Projects : Identification of Best Practice" provides a national overview and reports on each of the twelve projects. However, national reports are not the best medium for passing on practical tips! These have been written up in a series of six briefings. Some of the practical tips in these briefings may seem obvious to those with substantial experience already in this field. Nevertheless they have been written up in the hope that they will be of practical help to staff moving into this area of work. Evaluation Most of the projects have not commissioned a traditional 'formal' evaluation precisely because the projects were conceived as innovative and creative. There was little point in being locked into the measurement of fixed objectives if the original concept and design was clearly not working. The projects were, in effect, participating in action research, in which on-going monitoring and review was part of a constant feedback loop. This led not only to minor adaptations but also, on occasions, to major refocusing of the entire project. In Birmingham, for example, an initial intention had been to make small grants available to existing community groups to encourage them to involve learning and guidance in their work. In practice, the offer was only taken up by one group. On-going internal evaluation achieved two significant benefits for the project:- an understanding as to why take up had been so disappointing (one factor was the complexity of the paperwork); an understanding as to what could be achieved with the unspent budget at short notice with maximum impact (first aid courses which had a very high take-up in an area where there is traditionally a very low participation in learning opportunities). The experience of projects suggests that there are five components in evaluating projects of this type. First, it is important to research the needs of the area (eg. through surveys and focus groups). Examples of this in practice are given in the briefing entitled "Awareness Raising and Marketing". Second, it is important to record the baseline position so that the "distance travelled" can be assessed over time. Some of this data can be picked up through the "researching needs" exercise above (eg. the proportion of adults in an area participating in learning prior to the project starting). But there will be other useful material to note (eg. an audit of learning opportunities already on offer in the area; this could literally be 'nil', of course). Third, it is important to record progress made on an on-going basis. This might include management information on contacts made, training sessions given, the perceptions of key players, feedback from participants and notes on events' held. Fourth, it will be important to record an "end of year" or "end of project" position to set alongside the baseline position. This would include both management information and qualitative material (eg. perceptions of key players, including participants). Fifth, it is essential to put this material together in order to assess the "distance travelled" and learning points. These cycles had not been completed at the time of the national report being drafted and the preparation of these briefings. There are still some practical tips to share, however. For project workers in Bristol, it was crucial not to use lengthy and potentially off-putting questionnaires. This would have jeopardised the entire positioning of the project as informal, friendly and client-centred. The way in which they collected data on individuals expressing an interest in the project was to invite them to join the mailing list for future events and to enter a free draw; the three prizes were book tokens (value £10), together with either a free enhanced guidance interview or a £50 discount for a course provided by the Community Education Service or City of Bristol College. In Bradford, there was an intention to track individuals' progress from initial enquiry to longer term outcome. This seemingly innocuous objective has proved hugely challenging! The main difficulties are around data protection and systems compatibility. So far as data protection is concerned, the problem is that, whilst individuals may have been very willing to give their contact details to a one stop shop, say, or the Employment Service, neither provider is able to pass that on to another provider without obtaining the express permission of the individual. Thus, in practice, it is difficult to 'track' individuals without entering into precisely the bureaucratic paper trail which the projects were trying to minimise. Even if agreement is obtained, there is still the issue of systems compatibility, as different providers use different software for client records. Whilst the project has not identified simple solutions for these problems, it has certainly clarified the questions. Partners in the network are still discussing the best way forward. Feedback Feedback from participants has been a key feature of all projects. Examples of feedback on training received are given in the briefing entitled "Involving Local People". Other examples include feedback on gaps in education and training provision in the area. Often it is practical subjects such as First Aid, Food Hygiene, English as a Second Language or Fitness Training where residents feel the biggest gaps lie. Typically, the local FE and Adult Education providers are the principal recipients of these comments. Indeed, facilitating the 'feedback loop' has been another significant outcome from the projects. In Leicester, one of the volunteers from the project has been invited to sit on the Governors Sub-committee which is responsible for Community Education at the local school. Practical tips include:- Place feedback as an item towards the end of training events. Participants are more likely to discuss their views in an informal setting where confidence has already been built up than they are to provide written comments subsequently. It is important to make it clear when changes have been implemented as a direct result of feedback. Participants will be encouraged to contribute views in the future if they feel they are being taken seriously. Feedback is not an "optional extra". It should be a major component in decisions over future plans. Similarly feedback data is central to the business case for future funding (see next section). Next steps As the introduction made clear, project staff have been concerned over next steps. Assuming there is a will to take things forward, there are two main options. These are:- to embed the best practice from the project such that it becomes part of the established position rather than being seen as a separately funded, short-term enhancement; to secure additional funding from an alternative source to carry on with the existing work, develop it further, or extend it to another area. The options are not, of course, mutually exclusive! Practical tips around the embedding theme include:- Look for opportunities to develop further the roles of groups and networks originally set up as part of the project. Several "local fora" seem likely to become established beyond the life of the project. Encourage volunteers to continue their own learning, thereby acting as informal ambassadors for learning in their community, whether or not they are part of a tangible 'project'. Some projects have included incentives for future learning as part of the 'deal' with volunteers (see briefing entitled "Involving Local People"). Embed best practice around researching the needs of local areas and seeking feedback on services delivered. Empower local people themselves to take on paid delivery roles with the mainstream providers. This is a significant challenge to recruitment and training practices and equal opportunities policies. Practical tips around securing additional funding include:- Explore whether discrete strands of the project should be taken over for funding by one of the current partners (eg. maintenance of information data bases). Explore whether developmental aspects (but not continuing the status quo) meet the criteria for the Adult and Community Learning Fund. Explore whether funding from one or more local partners will be available for extending the project to other areas. This is where evaluation data is so crucial. The business case will be strengthened immeasurably by credible evidence that projects are making a difference. Explore funding streams from regional, national and international sources (eg. ESF Adapt or Integra).