"The Toolkit" - Practice, Progess and Value
Contents Page | Foreword by the Secretary of State | Introduction and Summary | What is a Learning City? | The Learning City | Taking the First Steps | The Structure of this Guide | The Three Strands of Development | Strand One: Partnership | Strand Two: Participation | Strand Three: Performance | Useful Publications

Strand Three: Performance

  • Towards a Model of Value Added
  • Internal Assessment
  • External Comparison: Benchmarking
  • Analysing the Added Value
  • Level 1: Getting Organised - Internal Assessment
  • Level 2: Towards a Shared Understanding - External Comparison
  • Level 3: Cycles of Learning - Evaluating Added Value
  • Moving Ahead - Three Year Development Plan

    3) Analysing the Added Value

    What value does a learning city add to its area or region? What have been the outputs of various initiatives? What has been the impact on the community? What benefits will follow for cities and towns in developing the ideas and partnerships on which learning depends?

    These are reasonable questions to ask, especially when authorities and organisations are being invited to commit public money to what can appear abstract purposes - to encouraging lifelong learning or economic and social regeneration. The essential task of the learning city is to:

    • analyse the relative achievement of different initiatives
    • learn why some initiatives were, and were not effective
    • form judgements about future action.

    This stage of the assessment framework allows judgements to be formed about the value which the learning city has contributed. The value added approach can be understood in two steps:

    • understand the "value chain"
    • evaluate the value added.

    The Value Chain

    The existing models of value analysis often emphasise the cost or efficiency of services. This is encapsulated in the idea of "value for money". A more sophisticated concept is needed to understand the value of a learning city. It should be one which expresses the variety of values and the layers in which they are added to enhance the quality of life in the locality.

    A variety of values needs to be recognised and celebrated. This is not only the new resources of finance and staff generated, but services produced, opportunities created for personal development and participation, the satisfactions engendered, and the longer-term enhancement of wellbeing in the community. The learning city needs to understand how value accumulates through all its activities and how these are connected together.

    The idea of the "value chain" derived from the work of Ian Sanderson and colleagues of the Policy Research Institute provides the sophistication required. This concept suggests that the value which is produced by services for communities can be broken down into a chain of processes, each stage of which adds value to the process. The five stages of the value chain are as follows:

    1. Inputs/Resources. These describe a wide variety of inputs: the appointment of staff; time contributed; financial resources added; materials and accommodation provided; the information and technology that is offered; the skill and knowledge of people at different levels of organisation. These resources are probably a necessary condition for many values to be contributed to the community: they provide the foundation on which other layers of value depend.

    2. Service Process. This refers to the processes of organising and managing the production of services and activities of the public sphere. In the context of learning cities the inter-agency partnerships and networks will form particularly significant processes which become a precondition for service provision. Likewise, the processes of relating to, and involving the public will identify the quality of producing services. Process adds value, not only in being the precondition for delivering a service but in determining the style of provision. A public service orientation in service delivery enhances the value of producing services which are responsive to public needs.

    3. Service Outputs. These refer to the services actually delivered to users "which may be measured in terms of capacity provided (e.g. facilities, places); throughput (customers/clients using facilities or occupying places); or other measure of the service provided or level of activity undertaken (e.g. hours of care for social services clients; hours of teacher contact for pupils; passenger mile of bus services). Related measures include awareness of the availability of services and levels of take-up."

    4. Intermediate Outcomes. Outcomes refer to the effects of services on both the direct user/recipients and the wider community/locality. They may be the intended benefits to users or may be unintended effects or even unforeseen side-effects. Short-term or intermediate outcomes refer to the effects which can be discerned soon after receipt of a service (e.g. satisfaction, immediate user benefit). For example, the provision of a creche may lead to an immediate benefit for lone mothers who are able to take up employment opportunities or places in education and training.

    5. Long Term Outcomes/Benefits. These refer to the wider impact of services on the life users and the community. For example, the provision of a creche may lead to long-term benefits to the community of higher employment, and or an increase in educational qualifications and capabilities of citizens in the community.

    Evaluate the Value Added

    If the work of the learning city wishes to enhance the value of the locality, it needs to test and evaluate the value which has been added. More resources may have been added, but have they been added economically? Services may have been developed but has this been done efficiently and are the outputs effective? These questions need to be answered before the learning city can be confident that it has added value. These tests for value added are set out below.

    • Tests for Value Added

    Economy and efficiency address "value for money"

    • Economy refers to the cost of resource inputs
    • Efficiency expresses the relationship between outputs and inputs

    Effectiveness addresses quality of service

    • Extent of achieving defined standards/objectives which express values sought
    • Effectiveness expresses relationship between intended and actual outcomes
    • Quality relates to standards of achievement/service
    • Process quality looks at attributes of organisational and management capability
    • Service quality looks at the attributes of service delivery

    Impact

    • Assessment of net added value of the service in terms of benefit to the community/locality

    Equity

    • Who receives the service and who bears the cost

    Key questions for planner

    • Economy: How do actual costs compare with planned costs?
    • Efficiency: What is the relation of outputs to resource inputs?
    • Awareness: Are the intended beneficiaries aware of the service?
    • Take-up: Are the intended beneficiaries actually receiving the service?
    • Quality: Are quality standards and customer satisfaction being achieved?
    • Effectiveness: Do actual outputs and outcomes achieve intended objectives?
    • Impact: What improvements are made to the quality of community life?
    • Equity: Is distribution of outputs, outcomes, benefits and impacts equitable?.

    Ian Sanderson et al, "Made to Measure - Evaluation in Practice in Local Government", Local Government Training Board, London, 1998

    These tests of quality can be applied to each stage of the value chain as illustrated below. This approach to analysing value - internally, comparatively and analytically - needs to be applied to the various aspects of the learning city's work.

    TESTS Inputs Process Outputs Outcomes Benefits
    Service
    (process)     		Service
    (outputs)     		Intermediate
    Outcomes     		Long-term
    Benefits
    Economy Resource
    Time
    Staff
    "
    "
    "
    "
    Efficiency
    "
    		Organisational
    capability
    Strategic planning
    Systems of evaluation
    "
    "
    "
    Effectiveness
    "
    "
    		 Services
    Information
    Advice
    "
    "
    Equity
    "
    "
    "
    		 Capabilities
    Skills
    Satisfaction
    "
    Equality
    "
    "
    "
    "
    		 Quality of life
    Culture
    Knowledge
    Democracy
    Citizenship

    Example: A plan to create a multi-purpose community centre

    Describe the elements in the value chain

    Inputs

    • budget: capital and revenue
    • staff: numbers and skills
    • materials, facilities

    Service Process

    • multi-professional partnership
    • collaborative decision making
    • user and community involvement in decision making

    Service Outputs

    • courses, training
    • library, IT resource centre
    • creche
    • advice
    • groups (women's group, youth club, health awareness campaign group, etc.)
    • leisure and recreational facilities

    Outcomes

    • learning
    • skill acquisition, qualifications
    • opportunities for job take-up
    • recreation
    • voice: articulation of individual and community needs
    • satisfaction

    Benefits

      greater opportunities; equity - for women; support of learning needs; child care; job opportunities

    Evaluate value added

    • clarify purposes
    • set targets
    • compare achievements with planned targets
    • compare achievements with other like centres (benchmark)
    • analyse value added
    • has the centre been economical?
      • compare actual with planned costs and costs elsewhere
    • has the centre been efficient?
      • what has been the relation of service outputs to resource inputs?
      • how does this compare with other centres?
    • has the centre been effective? Have outputs and outcomes achieved planned objectives?
    • has the centre been equitable? Have its services, benefits etc. been enjoyed equally by women as well as men, and by all generations?

    Performance Assessment

    The third strand introduces a cycle of collaborative learning that involves conscious, routine processes of reflection and evaluation. This enables cities to judge the added value which learning cities are contributing.

    The learning city is reflective, seeking to learn about itself and how it is changing. It develops the capacity to monitor and consciously reflect back upon its development so as to understand better how to make progress in the future. It is willing to reflect on and question the principles which are underlying the processes of regeneration.

    This makes the processes of evaluation central to the learning city. It will introduce mechanisms of monitoring, review, and comparative analysis to judge the value which the learning city is adding. Evaluation proceeds through levels of complexity:

    • building internal assessment

    • developing external comparison which generates dialogue about quality and encourages benchmarking, the comparative understanding of excellent performance between like organisations

    • developing evaluation analysis of the value added chain. The learning city will strive to generate a culture of research, to become a self-aware value adding city.

    Analysis leads to improved cohesion. The learning city values connectedness. This ensures that imaginative innovations are understood and connected together across the city. What is learned in one part is shared by all.

    • "Evaluation is in and of itself perhaps the central process: if the city is to learn from its experiences, it must be committed to effective and ongoing evaluation processes; it must reflect, consider and reconsider, monitor, think and rethink." (Charles Landry and Franžois Matarasso)

    • Evaluation and research are the key to learning in the learning city.

    LEVEL 1

    Getting Organised
    (Building)

    		 LEVEL 2

    Towards Shared
    Understanding
    (Dialogue)

    		LEVEL 3

    Cycles of
    Learning
    (Reflection)

    PURPOSE
  • internal evaluation
  • integrating into the strategic planning system
  • comparative analysis, understanding
    		•
  • the value chain
  • culture of reflection and action
    		•
    PEOPLE
  • policy planners, statisticians
  • policy makers
  • different perspectives
    		•
  • active involvement of citizens and service users
  • researchers
    		•
    PLANS
  • performance assessment system:
    - criteria
    - targets/standards
    - performance indicators
    - baseline data
    		•
  • comparative data
  • research, developing knowledge of medium, longer term processes
    		•
    PROCESSES
    of organisation
  • monitoring
  • supported self-evaluation
    		•
  • benchmarking
  • organisation integration:
    - performance review
    - policy planning
    - public service
  • inter-organisation professional development
    		•
  • the learning organisation:
    - understanding and resolving differences within and between organisations and the public
    		•
    PERFORMANCE
  • performance review
    		•
  • comparative discussion and evaluation
    		•
  • collaborative agreement and accountability
    		•

    Click here to go to the previous page
    Back    		 Click to return to our Home Page
    Home    		 Click here to go to the next page
    Next