Widening Participation and Arts Award Case Study
Lucy Shipp, Learning Associate (Widening Participation and Arts Award), The University of Cambridge Museums
About the project and my evaluation challenge
In my role at University of Cambridge Museums (UCM) I lead on Arts Award activity across the different sites and work strategically to develop projects that support Widening Participation (WP) at the university. UCM have worked with the university WP team for many years and the programme is developed alongside the schools programme and in partnership with local secondary schools and Councils.
My evaluation challenge was to unite a relatively diverse range of activity across the Arts Award and WP programme. Over the course of the Practical Evaluation project I created logic models and surveys to support a number of these activities (highlighted in the above diagram in blue). There were differences in delivery methods and aims between each activity, and, therefore, it was important to write individual logic models for each separate programme strand. We considered our aims for individual activities and reflected on our overarching goals for the UCM Arts Award and WP programme. This case study focuses on the development of the logic model, surveys and evaluation methodology for the Year 8 Arts Award Shakespeare Strategic School project.
Setting the scene
The Year 8 Arts Award Shakespeare Strategic School project has been running for 3 years. Although the project had been refined year on year, there was a lack of focus on the main goals for the project. The Practical Evaluation project provided an opportunity to review and clarify the aims of the project and align these with relevant UCM and Widening Participation team objectives. The Shakespeare Arts Award involves up to 15 students in receipt of pupil premium funding, over 8 – 10 sessions, and is predominantly based at the museums. The project introduces students to the context of life in the time of Shakespeare, using The Fitzwilliam Museum’s armoury collection, paintings and manuscripts to develop this understanding through object-based learning. The students work towards a Bronze Arts Award and take part in creative art, design and music activities with museum educators and artists, share skills with each other and review their experiences. The project also includes a visit to a university college, for students to gain an understanding of future study opportunities and raise student aspirations, linking to the WP agenda. The project is a good example of how we often plan activity to best achieve multiple outcomes, e.g. gaining an Arts Award, linking to school studies of English and History, and also providing an introduction to University.
Evaluation approach
Previously the Shakespeare Arts Award had been evaluated through a paper-based post-project survey for participants and the teacher. As part of the Practical Evaluation project, we used an online survey platform to charts changes in student attitude and learning through a pre and post-project surveys for participants, and a post-survey with the teacher. This case study focuses on the use of these automated surveys.
However, in addition to the surveys, we also monitor and track the progress of the students in the following ways:
- A weekly review by the lead educator to adapt sessions as the project develops, in response to student feedback and interests.
- Using Arts Award evidence templates (pro-formas) to review students reflections, personal progression and independent learning.
- Pre and post-project meetings with the lead teacher and school senior management to agree aims and objectives, which we then review at the end of the project
- Recording the number of students achieving an Arts Award
- Recording the number of young people progressing to Higher Education, measured using Higher Education Access Tracker (HEAT)
- Allowing students opportunity to set their own personal targets for each session, reviewing progress and reflecting at the end.
Reflections on using logic models and automated surveys with Arts Award and widening participation programmes
The Practical Evaluation project prompted conversations and questions about what the focus of the activities should be. Some of key questions which arose were:
- How much emphasis should there be on teaching shakespeare?
- Which of the WP criteria (for example: raising aspirations, raising attainment) should the project realistically seek to address?
- What was unique about the project taking place at the museum, and how could we maximise this opportunity?
- How much should we try to focus the sessions on themes within the curriculum, and how much should we extend students learning and curiosity beyond that?
- What was the role of enjoyment within the project? How important was this in relation to, for example, students feeling welcome within museum spaces?
- What was the role of creativity within the project, and what did we mean by this?
The conversations prompted as part of this project helped us to better understand how the WP agenda linked directly with the work of the UCM. It was clear very early on in the process that we were trying to achieve too much, and this prompted conversations which helped us to refine and streamline the key aims of the programme.
The logic model helped us to revisit how activities met the programme aims, creating a clear purpose for the activities and programme structure. This process forced us to start with realistic programme goals, and then consider how this supported the aims of the key stakeholders, rather than the reverse. The overarching aims were considered more fully within the planning phase, giving the project it’s own identity and resulted in better articulating the core focus of the project, anticipated student outcomes, and how these linked to the aims of WP and other stakeholders. As part of the process, we also looked for supporting evidence by drawing on current research on social mobility, widening participation and creativity. The logic model and survey, even in draft form, also created a useful starting point for conversations with teachers and partners who were involved with the project. We were able to further refine our planning based on more meaningful conversations with the teachers.
Working with the Practical Evaluation team meant that we continually had to be clear about the approach we were taking, and Eric acted as a critical friend by prompting us to be more precise with our aims and language. It was through this process that we realised that delivering the Arts Award as part of the project was intrinsic to the structure of the project and enabled us to meet key WP aims around developing reflective and independent learning skills.
The logic model then became the starting point for the creation of more focussed survey questions. Where previously questions had focussed around students’ enjoyment of the project, we were able to measure student progress against the intended outcomes. We asked students to gauge their level of agreement in how confident they felt working with museum objects on a likert scale, alongside more open ended questions regarding their thoughts around Shakespeare, the museum and Arts Award. Going forward the responses from the surveys will enable us to refine aspects of the programme that are and aren’t working, to better meet the aims and objectives we set for the project.
Using an online survey means that students were able to complete the surveys off site, freeing up more time for learning within the museum. It also allowed students time to reflect and give objective feedback, as they don’t need to hand their surveys back to the person who delivered the project. Having the survey responses online saves time in interpreting the data, word clouds from open-ended questions are generated automatically, along with graphs/pie charts from qualitative data. This enables us to share the information easily, and ultimately help us see what is working or not working about a programme. With the project taking place over multiple sessions, the digital survey model also opens up the possibility to gather data at multiple points over the programme.
The process of creating logic models and digital evaluation surveys has been beneficial in refining existing activity, creating new activity and comparing data across a wider programme activity across UCM. Creating logic models and surveys for the programmes has helped us to clearly articulate project aims and approaches across the UCM Widening Participation and Arts Award Programme which has been hugely beneficial for the whole team and for communicating with our project partners and funders. We now look forward to applying this approach to other strands of the programme.
Images above: (c) University of Cambridge Museums