Blog entry by Outreach Evaluation Hub

Picture of Outreach Evaluation Hub
by Outreach Evaluation Hub - Wednesday, 13 November 2019, 4:38 PM
Anyone in the world

HeppSY is approaching Phase 2 with a growing sophistication in its approach to data and evaluation. The recent introduction of the NCOP Progression Framework has prompted a review of HeppSY’s Evaluation Model and has made for exciting developments, enabling an even tighter integration between the programme and its evaluation.

Lucy Clague, Evaluation and Data Manager, explained how HeppSY’s work has been informed by Bourdieu's concepts of economic, social and cultural capital. The theory, which has underpinned the partners’ understanding of inequality in access to higher education, recognises that where a young person comes from a physically, economically and socially challenging background it is important to the barriers they face. All of HeppSY's outreach activity rests on four barriers a young person might experience around accessing higher education.  Informed by a collaboration of practitioners, academics and partners specialising in widening participation, the development of a programme Logic Model led to the four barriers, or strands, namely confidence and resilience; attainment; higher education knowledge and career knowledge. This has set HeppSY’s prototype for activity based around 'what works', forming the principles of the Evaluation Plan.

When the OfS tasked NCOP partnerships with developing Progression Frameworks for Phase 2, it made sense for the coding of the activities in the Progression Framework to follow the coding of evidence in HeppSY's indicator bank which had been informing its evaluation. The learning outcomes for the activities in the Progression Framework are skills based and incremental – designed to show the journey young people are going through, with the data and evidence collection organised in parallel to follow the journey. Plus, mapping of the framework against the Gatsby benchmarks and Ofsted inspection expectation for careers guidance has meant the value of the activities goes beyond that of the NCOP alone.  

The creation of the Progression Framework then led to the separation of the Phase 1 Logic Model into two, the first reflecting 'programme' outcomes and a second one focusing on 'learner' outcomes - therefore linking it neatly with the new Progression Framework. This ensured a cohesive design across all elements of the programme and its evaluation. Although separate, the Logic Models are interlinked and interdependent; to be successful both models require positive outcomes, and the iterative nature of the HeppSY programme is also at play here with evaluative evidence emerging from the overarching programme developing the learner experience, and the learner experience into the ongoing re-development of the programme.

The ambition doesn’t stop there. The HeppSY Learner Progression Logic Model mirrors the statements held in the Progression Framework, ensuring that evidence can be collected which maps onto both models. In the next few months the team are planning a data analytics tool, which will support activity planning and evaluation on the ground. An online diagnostic tool is being developed with schools and colleges. This will be available in future on the student-facing part of HeppSY’s website. The tool will use a series of questions measured by Likert scales to capture attitudinal change for 24 skills based outcomes (coded in line with the Progression Framework and Evaluation Indicator Bank).

Being able to feed learner voice data and analysis back into delivery is another reason that practice has now reached a critical place. 

Work is currently underway to ensure that the diagnostic tool scales represent reliable measures that predict HE outcomes and intention to apply to HE. The annual survey (which has been mapped against the four HeppSY strands since 2017), has now been mapped on to the Progression Framework outcomes to enable HeppSY to draw on and link with data from the 10,000 responses from each wave of the survey.

Sharon Woodward-Baker, HeppSY Programme Manager, says the programme feels more iterative and cohesive than ever and that having a clear theoretical starting point has helped to line up funding, activity development, delivery and evaluation aspects, with a view to target learners who are better equipped to face future challenges.  The personalised approach has reconciled a high-level Logic Model with a Progression Framework which focuses on the individual and allowed data and evidence to feed into the learner activity (and vice versa). A wealth of management information is being generated which will support both formative and summative evaluation of the higher education support needs of the target groups in South Yorkshire.

Considering it's still early in Phase 2, the team is pleased to have the data and evidence infrastructure firmly in place. In HeppSY’s case the formula, “more thinking = better ideas” seems to be paying off.

HeppSY's Recommendations:

A Logic Model developed through local evidence.
An Evaluation Plan which links in with the wider programme but is flexible enough to meet changing funding requirements.
A team who recognise the value in having a comprehensive collaborative approach between data and practice.
A Progression Framework with skills and outcomes which are quantifiable.
A close partnership between data and the evolution of student experience.


Sharon and Lucy at the NCOP practitioners conference

Sharon and Lucy at the NCOP practitioners conference

Author(s): 

  • Sharon Woodward-Baker, HeppSY Programme Manager
  • Lucy Clague, HeppSY Evaluation and Data Manager


[ Modified: Wednesday, 1 April 2020, 12:07 PM ]

Comments