UKOER/SCORE review introduction

Go to UKOER/SCORE-Review-Final-Report

Go back to HEFCE-Review-Impact-Summary

Go forward to OER journeys

Download this page as an open document format or pdf


This report offers a review of HEFCE funded UK initiatives that explored and supported open educational practices, which includes all phases of the JISC/HE Academy’s Open Educational Resources Programme (UKOER) and the Open University’s Support Centre for Open Resources in Education (SCORE) activities. Both UKOER and SCORE aimed to advise on and support development and publication of resources in more ‘open’ form as well as to raise awareness, draw out key issues and deepen understanding with regard to open practices.

The UKOER/SCORE OER review has been undertaken by the UKOER Synthesis and Evaluation Team led by Glasgow Caledonian University, using the core UKOER methodology developed iteratively across the three phrases of the programme, modified and expanded as appropriate to the distinct aims and activities of SCORE.

HEFCE-funded OER initiatives (2009-2012)

HEFCE funded UK OER activities built on previous investment to the sector around sharing resources, including JISC Exchange for Learning x4LJorumJISC Digital Repositories Programme, JISC Repositories and Preservation Programme and JISC ReProduce Programme. Some activity has arisen independently of formal reuse or OER projects as an extension of more general learning and teaching activity, e.g. through the HE Academy Subject Centre or National Teaching Fellow activity.

JISC/HE Academy UK OER Programme

Initiated in 2009, HEFCE funded three phases of the JISC/Academy funded UKOER programme and The Open University’s national role through SCORE, which both completed in 2012. Building on the substantial synthesis and evaluation across UKOER, the purpose of this overarching ‘HEFCE OER review’ is two-fold:

(a) to deepen understanding of open educational practices and resources

(b) to produce a solid evidence base and enhance the status of the work supported in the UK and in the international OER field

HEFCE funded OER work in the UK has been extensive and has impacted on strategy, policy, practice (of a wide range of stakeholders, including learners), research, curriculum design, delivery and support. The intention was to undertake an evaluation of UKOER and SCORE as a collective whole, whilst acknowledging each initiative’s unique aims and activities.

Throughout all three phases of the UKOER programme, evaluation findings from projects have been externally synthesised by the team through the development and refinement of an evaluation framework and associated methodologies, which have contributed to a series of reports and background data collated on a wiki platform. The three phases of UKOER activity led project participants through a range of activities and experiences which supported the emergence of an OER community in the UK. The programme has resulted in an increased understanding around the complexity of the open educational landscape – discussed in both the Open Practices Briefing Paper and a blog post by Amber Thomas, one of the JISC Programme managers – visualisations of the wider open education landscape.

Commencing with a pilot phase examining sustainable models of practice within institutional, subject consortia and individual settings, the programme examined the transferability of these models alongside refining our understanding of use within the second year of funding, and proceeded within the third to examine the applicability of “open” models to institutional and sector strategic priorities, attempting to use these to draw in the interest of senior managers.

Kernohan, D. and Thomas, A. (2012) OER – a historical perspective.

Open University Support Centre for Open Resources in Education (SCORE)

In parallel SCORE worked to support strategic development of OER across the Higher Education (HE) sector as a whole, drawing on OU competencies in this area to address sector-wide needs. As noted by the OU National Role Advisory Board  responsible for monitoring progress of SCORE supported activities, the plan suggested that:

it is time to move the sector from a supplier-led approach to the creation of web-based OERs to one which reflects the needs of both institutions and students“.

(paper NARB/10/1/3)

SCORE supported work over the 3-year funding period has set out to support and inform individuals, projects, institutions and programmes across the higher education sector in England. The funding has been allocated to support four types of activity: sector engagement, fellowships projects, events and production/publishing of OER. A retrospective review of SCORE work took place for the purposes of this review and was presented in SCORE Evaluation Report, November 2012. SCORE identified a number of trends in the UK OER movement which are particularly relevant to this report:

  • a shift from OER production and release to the use of OER in professional (including learning and teaching) practice, i.e. a more demand-led model;
  • concerns moving from pragmatic to conceptual issues such as epistemic approaches and broader political/ethical impacts of OER;
  • OER being included in other high level strategies/policies such as curriculum innovation, e-learning, digital literacy, professional development and QA;
  • the polarisation of those who see value in OER engagement at a time of rapidly changing context, and those who see it as an extravagance;
  • the importance of linking open research and open teaching in a broader discussion about open scholarship and open knowledge practice.

We explore the wider background and context  in the following section OER journeys

Purpose & Scope of this report

This UKOER/SCORE review extended the synthesis and evaluation framework used for the UKOER programmes to include the aims and activities of SCORE, thus promoting a unified emphasis on the programmes as part of a whole HEFCE investment in OER. The framework consists of a number of key focus areas, each with a range of evaluation questions that reflected programme aims and informed individual project evaluation activities . Individual project and strand/theme findings were mapped to the framework, providing an overview of key issues and trends across the programme. This approach highlights both key outcomes and significant outputs that demonstrate evidence of these.

The framework for UKOER has evolved throughout all phases of the UKOER programme. Evaluation & synthesis has been an iterative, two-way process such that projects and support teams contributed to the development of the framework throughout and that each iteration of the framework reflected current work. The OER phase 1 pilot programme enabled the large scale release of OER, the three strands of funding allowed different approaches, benefit cases and technical solutions to be trialled in a genuinely diverse mix of contexts. Phase 2 extended OER release but also supported activity areas around OER use and discovery. The phase 3 projects investigated the use of OER and open approaches to work towards particular strategic, policy and societal goals through a thematic approach: Theme A : Extend OER through collaborations beyond HE; Theme B: Explore OER publishing models; Theme C: Addressing sector challenges; Theme D: Enhancing the student experience.

All three phases have explored individual, institutional and community issues around embedding sustainable practice and widening engagement with OER. This review of both the UKOER programme and SCORE initiative consolidates and expands the representation, themes and findings from previous synthesis across the existing framework. This work is augmented by the retrospective review of SCORE activities against the framework and further work with the wider OER communities, to identify any new themes or further expansion of earlier findings.

The following pages offer an idea of the scope of the UKOER and SCORE initiatives:


In addition to the cumulative synthesis of all phases of UKOER evidence and SCORE work, evidence was drawn from OER-specific communities through a detailed survey and semi-structured interviews. Wider sector engagement was sought through an online poll designed for this review, as well as some analysis of social media activity. From initial findings, a focus on “OER and OEP journeys” emerged both as a way into interviews and to describing the impact of HEFCE funding. A wide range of answer options (including an ‘other’ free text option) in the detailed survey and the use of very open questions in interviews was intended to allow new and unexpected themes and gaps to surface. However, encouraging respondents of both the wider poll and the OER survey to choose/ state their ‘top 3’ also enabled us to identify more categorically the major influences and priorities across stakeholders.

Online communities, surveys & crowdvoting

A first entry into the OER community and wider sector perspectives was achieved through design and delivery of two online questionnaires during July 2012. The first was a short ‘poll’ of five questions distributed widely across professional contacts, HE/FE mailing lists and social networks, which aimed to to snapshot cross-sector awareness of HEFCE funded OER initiatives (namely the three phases of UKOER and SCORE). The second was a longer survey circulated specifically to individuals directly participating or indirectly involved in the UKOER programme and SCORE work.

Survey questions explored the way open educational practices are being perceived and sought to identify changing attitudes towards risks, benefits and barriers.

Central to supporting the OER community is the online presence of both UKOER and SCORE participants. Some data concerning activity and usage via the web, blog, social media, was reviewed, as this offers interesting indicators concerning sector engagement. What is interesting here is the general inter-connectedness of the OER community. Such community structures, growth and cohesiveness has proven to be a crucial aspect to support awareness of, and engagement with, the potential benefits of releasing and using OER. A description of this and outcomes are included in the SCORE evaluation report.

However, for those not necessarily involved or funded directly, we recognise that different terminologies, contexts and associations may apply. It is important that these are identified and incorporated into our cumulative knowledge and understanding. We therefore welcomed and encouraged wide engagement with a short, cross-sector ‘crowdvoting’ type online poll that aimed to gather a broader set of opinions and judgment – “wisdom of the crowd” – against which we can overlay our deeper analysis of open practices from individuals within the OER community.

See Appendix 1 for an Analysis of survey and poll

Interviews with direct participants

A set of follow up interviews were elicited from respondents to the detailed OER survey. The selection was made from those who (i) volunteered to participate further, (ii) were directly funded/supported as individuals or led an institutional project, and (iii) collectively represented one or more phases of UKOER and/or SCORE. Interviews aimed to deepen our understanding in the four key areas represented by the OER framework and pick up on any new emerging themes, including unanticipated or unexpected findings.

Interview questions were designed to draw out perceived institutional /subject related benchmarks, specifically:

  • awareness of UKOER and SCORE work
  • sector, roles and disciplinary nuances
  • individual and institutional drivers / motivations
  • perceived /experienced benefits of release and use of OER
  • perceived /experienced barriers to release and use of OER
  • evidence of impact within different communities of practice
  • evidence /examples of student /learner involvement and benefits
  • how OER are impacting future directions & the design of education.

A selection of 16 survey respondents were invited to participate in interviews and care was taken to ensure the sample (i) incorporated practitioners from both UKOER (8) and SCORE (4) ; (ii) senior staff directly involved in either or both (2 individuals in institutional management or mentoring roles); and (iii) did not favour only those highly engaged in OER (2 individuals unknown to the team): numbers as indicated.

Interviews were semi-structured, drawing out personal benefits and examples of impact and sustained outcomes in their organisation (e.g. in terms of culture, practice, skills, support, partnerships, resources). Relatively structured initial questions captured their institutional role, skills and experience to baseline, and follow up questions were used to clarify and expand on the areas listed below and any new themes emerging.

  1. the scale of personal & organisational engagement in OER at the start;
  2. their sense of primary spheres of practice (role, communities, disciplinary groupings, use of OER from different sources);
  3. their perception of their organisation’s ‘OER journey’
  4. evidence of benefits and sustained outcomes/ impact.

Interview questions were deliberately open and used to prompt a fairly free narrative yet draw out specific examples. This made the data complex to analyse, but this approach enabled new and interesting insight to surface and avoided the danger of only exploring those themes already highlighted in existing synthesis work.

The interviews explored the journeys of individuals and organisations in their move towards open educational practices. See Appendix 2 for an Analysis of Interviews

Parameters for analysis

The wider ‘crowdvoting’ poll elicited 129 responses, predominantly from the UK (58%) but also a fair spread of international engagement mostly from English speaking countries. 96% were working within an organisation/institution and most were working in the HE sector: US (7%), New Zealand (6%), Australia (4.5%), India (4.5%), Canada (4%), South Africa (3%) and one or more responses from 12 other countries. Some differences in countries represented by respondentsworking in the HE sector. Those working in school education were largely UK and India; other sectors included further/vocational education (predominantly UK), public sector (UK & US only), private sector (UK, India, Brazil, Netherlands & South Africa), charity/voluntary sector (entirely UK & US).

The detailed survey with the OER community elicited 50 full responses, 98% of which were HE, which is unsurprising given the HEFCE funding was HE focused. Respondents represented a spread of roles; the majority were teachers/tutors (52%) yet with a fair percentage indicating their role (solely or additionally) as managers (22%), pedagogic support (26%), researchers (28%) and OER support project (26%). 10% indicated a technical support, 10% librarian/information worker, few or none in marketing or admin, and 2% were students. Responses represented people involved in all initiatives as individuals (UKOER projects and/or SCORE fellowships, residential course or workshop attendees) or as part of an institutional project (as in UKOER projects) or subject community (UKOER pilot and phase 2).

The survey attracted a good mix of people involved across both initiatives, as individuals (predominantly SCORE), support services, lead institutions (only UKOER), partner institutions and/or user/recipients. Only UKOER phase 1 (pilot) and 2 indicated subject community involvement. UKOER phase 3 involvement was predominantly as ‘lead institution’ (50%). These profiles mirror the focus/nature of the UKOER programme phases and SCORE opportunities (personal fellowship projects, residential courses & workshops).

Quantitative statistics of responses presents a valuable picture of current perspectives and evidence for awareness, priorities and outcomes in some areas. It is, however, the qualitative data which yields the deeper understanding arising from ‘softer’ indicators of benefits and impact expressed through the narratives of personal OER journeys.

Interviews, in particular, enabled participants to recount their story, to build up a picture of what has changed since the initial UKOER pilot programme in 2009 (e.g. how they have moved from where they were to where they are now). Notably, interviews were not intended to cover technical details, although examples of technological practices do provide indicators of impact in terms of embedding. This kind of technological detail was usually evidenced through project reporting and from the JISC CETIS synthesis activity. Results concerning discoverability, including choices around granularity and hosting, also provide valuable emergent findings and evidence of organisational outcomes.

One focus has been at the strategic level within the higher education sector, but analysis has drawn heavily on the perspectives of individuals and communities who have both benefited from the HEFCE initiatives, contributed to outputs, and influenced open practices both locally within their organisation and collectively across the sector. Here, trends of particular interest are in terms of how OER are changing the design of education and how Open Educational Practice (OEP) is being informed and informing other aspects of academic practice, such as research, student employability and business/community engagement.

In particular, interviews were intended to help deepen understanding by expanding on early synthesis findings and analysis involved identifying examples of how practice is changing, for individuals, communities and organisations. For instance, the apparent shifts in relationships between academics and students (and ‘users’ more generally), such as their involvement in the development of OER in terms of “pedagogic co-creation“, rather than simply development of OER for use by other teaching staff. The way open educational practices are perceived in terms of who OER are being produced for is a key indicator of potential longer term impact. It suggests changes in attitudes towards risk that can lead to a greater and more widespread release and use of OER by teaching staff, enhanced student centred approaches, as well as acting as marketing tools leading to reputational gains for the institution.

Back to UKOER/SCORE-Review-Final-Report main page

Forward to OER-journeys