Subject Strand synthesis –Helen Beetham
The CETIS support project has summarised the technologies used and developed by the OER pilot programme. This summary reports on the reasons for different approaches to hosting, metadata and technical developent.
Projects took various approaches to hosting and syndication of resources, which can tentatively be related to their situation on the two axes identified as ‘integrity = reusability’ and ‘wild’ = ‘community-based’ (see ‘developing, managing and sharing resources’)
- wiki-based collaborative development of resource collections – C-SAP, TRUE (tend to focus on ‘integrity’ of resources)
- development of a community repository – Humbox, OERP (tend to focus on ‘community’)
- populating an open repository with discrete resources – often with a taxonomy or toolkit to support identification and reaggregation – ICS, FETLAR (tend to focus on ‘repurposing’)
- syndication to multiple web 2.0 sites and open repositories – Engineering, C-Change (tend to focus on ‘wild’ release)
Most projects used syndication to ensure that a single authoritative version existed for updating, any necessary take-down, ongoing refinement etc. but that there were multiple opportunities for potential users to find it. However, projects had variable experiences with and attitudes to web 2.0 hosting solutions e.g. slideshare, flickr, scribd, youtube.
Wrt metadata, all projects insisted that consistent metadata was critical to resource management and discovery. Metadata and tagging templates/guidlines and keyword lists or taxonomies were among the earliest outputs of the programme. However, projects took different approaches to authoring metadata, from doing it entirely within the core team to giving all potential users equal opportunities to upload their own metadata. There is a trade-off here between accuracy and sustainability. There is also a tension between using rich metadata records to meet the needs of potential users, and keeping it simple enough that potential contributors do not fall at the metadata hurdle.
What have we learned about appropriate tools and standards to support OER release?
Findings from projects
(Refer to CETIS support team final report for more detail on this issue)
Projects have learned a great deal about tracking resource use, both quantitative and qualitative. e.g.
C-Change: feedback will be captured through the use of a GoogleMail form45 that outputs to a Google Docs spreadsheet.
FETLAR: tools for reuse such as QTI tools (minibix) can add to the value of repositories by supporting specific kinds of repurposing.
- Use of content packaging was limited as 8 out of 10 Project Partners found this not to offer any improvement for the exposure of resources.
- Two of Project Partners have made significant use of eXe (eXe Learning, 2009) as now recommended by Jorum.
- Our experience with the RELOAD (RELOAD Project, 2004) content packaging tool was that it did not appear to offer any advantage over eXe and it was not particularly user friendly.
- Software formats for open release must freely available so potential adopters can use and adapt resources without having to transfer the content into another system
ADM-OER documenting a major shift in attitude away from closed VLE as repository for learning materials and towards open institutional repositories.
Project outcomes and evidence
Support team (CETIS) audits S4S: evidence of most appropriate technologies for disseminating OER
- aggregator to allow resource discovery of resources held in multiple locations
- technical workshop materials and example workflows
- OER passport – guiidance on describing licenses, attributions and other resource metadata.
- Depositing OER into JorumOpen – guidelines for depositing OER into JorumOpen
FETLAR: MathAssess fully available and workflows convenient for end users
OER Sim Legal: investigating use of Narrative Event Diagram and the SIMPLE (SIMulated Professional Learning Environment) toolset and platform to support sharing and re-use of simulations.
Bioscience: short guide to Preparing Packaging and Uploading
OER-CSAP – development of toolkit to support module ‘mapping’. Using the xml base, an xhtml preview facility allows a user to view the description of a module in a web browser and access the constituent materials. In addition the toolkit has the functionality to develop a dynamic taxonomy of metatags which can also be input to JORUM or any other repository or web 2.0 location.
What role could be played by taxonomies and/or controlled vocabularies?
Findings from projects
Core-Materials finding: The taxonomy has proven to be of help in supporting browsing, accurate searching and guided navigation, to enabling users to find suitable materials more readily.
Bioscience (interim report finding) concerns over consistency of tagging without controlled vocabulary
Outputs and evidence
TRUE resource types and reflections of their value to users
- List of keywords and tags based on existing pedagogic vocabularies such as JISC e-assessment glossary and Higher Education Academy Vocabulary.
- Developing taxonomy of learning/teaching approaches based on contributions via the toolkit
What are the issues with using multiple host sites (duplicating or syndicating)?
Findings from projects
In general, multiple syndicated sources is the preferred approach to hosting Most institutions are now mandating staff to deposit materials in a local repository. As we move towards a more coherent picture, still there can be a proliferation of local repositories as individual members of staff and course teams see the need to develop one (e.g. ADM-OER)
Most staff and students use search engines to locate OERs so it makes sense to use hosting solutions which maximise discoverability by search engines and enable good Google rankings.
- Resources uploaded to JorumOpen but also accessible via SC web site using dynamic link to JO, to ensure that usage statistics are not divided between the two, and syndicated to Merlot.
- Networking repositories to promote the automatic re-distribution through other national and international sites e.g. Merlot, is not difficult and can bring valuable new users (and potential contributors) to the OER collection and therefore ‘amplify’ the deposit.
- web 2.0 sites such as SlideShare and Scribd and alternative educational repository, Merlot, used to assess their value as alternative or complement to JorumOpen.
- Limitations: often only a limited range of formats can be uploaded; upload to multiple sites is time intensive without some method of automatically syndicating content.
- Benefits: number of downloads and page views can be tracked; there is a comment facility. Number of views compares favourably with JO for the same resource suggesting web 2.0 sites are useful route to dissemination.
Core-Materials: materials held in Core-M repository, uploaded to JO by RSS feed, some also duplicated in Flickr, YouTube, Slideshare and Scribd
C-Change: findings on ‘single authoritative copy’ approach:
- allows clear legal responsibility and takedown policy
- allows updating
(these both v important with legally sensitive and rapidly changing topics such as climate change)
- also if item is courseware then allows enhancements to be aggregated
- But metadata requirements may differ across host sites and may change over time as OERs evolve
- multiple dispersed host locations makes it particularly important to use standardised metadata and tagging. Upload to third party site risks losing essential context and brand identity.
- Solution: project branding and ‘back page’ info included with the resource (but this is time-consuming)
- Solution: super-search facility (but developing this proved problematic as APIs were often underdeveloped for the task or not fully supported by the suppliers of services)
OERP evaluation quote: I have been made more aware of the multitude of websites where one could deposit materials, but in fact I consider this to be another negative and my experience of these sites is that they are not an ideal source for locating specific educational materials. If there are too many places to visit, students/staff will not quickly home in on the best resources.
FETLAR: Opportunity to update or correct resources after deposit is essential
Project outputs and evidence
C-Change: observations on pros and cons of ‘single copy’ approach, with references only in other host sites.
OERP: progress towards a cross search service called the “SuperSearch” to address concerns about diversity of host solutions http://www.slideshare.net/EngSC/pull-yourself-together and http://www.engsc.ac.uk/an/oer-project/oer-project.asp.
What kinds of metadata are essential, what desirable, and what are the issues in creating and managing metadata?
Findings from projects
All projects agreed that appropriate metadata tagging is crucial to ease discovery, especially when searching across repositories and host sites. There are trade-offs between usability for future users and workload for providers.
Levels of metadata – many projects distinguished the core, essential metadata from further more detailed descriptions of the resources which differed from project to project, but often included contextualising and pedagogic information. What other info should be included with materials, e.g. take-down policy, attributions and references, date of original authoring/uploading/repurposing etc…?
There is also a trade-off between project team doing the tagging (quality and consistency assured, reduced burden on contributors) and devolving to partners or contributors with clear guidelines or simple tools (more sustainable)
Humbox interim finding: There has been considerable discussion in regard to tagging of items and metadata and we will be reviewing our practice in this area, in the light of current use/effectiveness of the current system of free tagging of items which has created a large and sprawling list which is cumbersome to search.
- different academics have very different and sometimes opposing ideas of the relative important of metadata fields
- descriptions which include keywords relevant to teaching make resources more discoverable and accessible
- more technical metadata fields are seldom used to find resources
- ensure metadata searchable by Google and federate metadata to other repositories
- if creating or managing a repository, make sure metadata is exposed to external searching
- metadata for cataloguing in repositories benefits from a skilled cataloguer or librarian. *simple metadata will suffice for basic searches
- a metadata ‘wizard’ may be useful to guide the record author towards a consistent standard.
SimShare Legal findings:
- In addition to the basic medatata profile mandated by JorumOpen we asked depositors to provide additional information about each resource (as an appended file) to allow potential users to understand the implications of using a simulation in terms of, for example, student role(s), support, staff time and run-time.
- The tension between ease of use for the donor and the effort of providing comprehensive metadata for end users surfaced in our survey.
- because the repository search facility filters titles, descriptions, classification and keywords for all resources, only lightweight metadata was used.
- after reviewing a range of available tools, TerMine was chosen to support the automatic generation of keywords for large text documents.
- assigning metadata and the process of cataloguing each resource was a labour intensive process requiring technical, subject-specific input from the Project Team.
- The ‘wrapping’ of OER resources should at least provide a full set of the resource metadata within its text. This means that the author/copyright holder and description are always visible to the user and readable by search engines.
- Use of IPTC, EXIF or XMP metadata within openly distributed images could support discovery, geo-tagging and due diligence data: to be pursued as proof of concept.
- In practice each partner developed the metadata schema to suit their own particular approach to repurposing, which resulted in slightly different information within their due diligence records.
- To record their due diligence records, partners chose to use different mechanisms: spreadsheets, text-based systems, digital systems and paper-based: unclear which is most appropriate.
- The following are potential locations for storing due diligence records: with resource in repository (not currently possible); in file header as document property; with originating individual; with originating department.
- using third-party sites and releasing OERs into the ‘wild’ can lose contextualising information
- solution is to include project branding and a consistent ‘back page’ detailing information about the resource e.g. attribution, licensing, which travels with the resource. (cf C-Change ‘passport). However, this can be very time consuming to do for every file.
- an important lesson is to release materials with guidance and assessment guidance where possible.
- staff who wrote the resources are best placed to provide the metadata for the resources – but this adds to their workload and potential disincentives for sharing
OER- CSAP there is a need to create pedagogical descriptions that can be rendered as machine readable, or at least capable of being read across differing contexts, but attempts to standardise such descriptions has not been successful – academics prefer to describe their practice in very open terms and not to use a prescribed taxonomy or set of approaches.
Project outcomes and evidence
Feedback on usability of Humbox interface and perceived ease of tagging
FETLAR: taxonomy and metadata template
OER CSAP: appropriate tagging and metadata structures to ensure shared understanding of educational rationalel; blogged comments and feedback on voicethread.
- Resource upload toolkit
- API toolkit and PIMPS interface
- Table of comparisons between OpenJorum and other hosting models
- Five scenarios for future JORUM use.
OER Sim Legal: design of metadata requirements and suitable online interface to support open release of simulations and their components
S4S evidence of most efficient and sustainable processes for metadata tagging
CORE-Materials metadata schema; review of automated tools for keyword generation; semi-automated classification and tagging interface
- standard Digital Rights Metadata Schema
- reflections on metadata fit for purpose and on use of IPTC, EXIF or XMP metadata tags to hold metadata within images.
- reflections on due diligence records
- reflections on ‘OER passport’
ADM-OER metadata and format meet JISC standards
OERP: Resources Description Guide – tagging methodology with examples, to ensure all partners added enough descriptive information to each resource.
How do existing repositories e.g. JorumOpen support the release, management, discovery etc of OERs in the UK?
It is widely reported that problems with JO functionality and delays in the development of JO impacted negatively on projects. There was initially a degree of resentment that a specific repository had been ‘imposed’, which was seen by some as being against the open ethos and the need to explore a range of possible solutions. On the other hand, very many projects expressed the view that an open national repository was a necessity to UK HE and were highly motivated to improve it. Much feedback has been given to JorumOpen by project teams, and has been used to improve the service for future contributors.
Issues raised in project reports include:
- Problems with log-in routes – different routes lead to different functionality with no obvious rationale for this
- Concerns about expired content and keeping content up to date – what is the take-down policy, and practice around it?
- Frustration with bulk upload processes – need to be certain RSS feed for metadata works robustly.
- A need for templates and concise guidance on the whole process including: IPR and copyright clearance; metadata and tagging.
- Desire for user profile pages and a comment/review facility to support user engagement
- Consider kitemarking or star-rating resources (not agreed by all)
- Consider ‘hallmarking’ resources e.g. with unique identifier, CC licence, date of upload
- Sub-admin accounts to allow editing of records by authorised contributors after upload
- Work on role-based authentication
- Consider mirroring resources on other sites (referatory model rather than repository model).
- Desire for better tracking information – at least download stats as well as views
- Full support for content packaging, resolving problems of how uploaded content packages display
- Streamline upload process and support range of different external interfaces (e.g. institutional) so upload can be embedded into everyday practice
- Encourage external linking to JO resources, especially from those who contribute and download, to ensure better Google (etc) rankings
- Make better use of keywords e.g. sharing common searches with depositors to guide their use of keywords; use of dynamic tagging via thesaurus.
Some selected project findings on JO: C-Change conclusions on JO
- refine resource discovery and presentation especially for GEES subjects which e.g. are not discovered well by JACS codes.
- provide good practice guidelines for entering data into the fields of the upload interface
- improve user access to syndicated resources i.e. not deposited in JO
- create links to subject repositories and other significant sources of high quality, relevant OERs
- improve use of aggregators so syndicated resources can be uploaded and searched by keyword, module code etc
- need option for materials to be owned by and associated with a project as well as a person (e.g. RAs may be very temporarily employed on a project, personnel may change).
TRUE: Not all available licenses can be selected; The cataloguing process is far more complicated than it needs to be.
- needed to be able to to provide back tracking data in the event of a Freedom of Information (FOI)63 request from a patient or their family: Jorum could not support this
- Survey found JO was less used than the Subject Centre’s own web sites or even Amazon and iTunesU. Intute fared rather better than Jorum with 18% of respondents using it.
General issues around repository use which were raised by projects include:
- Most academics still have very limited experience of repositories and use them very little
- Even fewer have experience of uploading to repositories and would find the process time-consuming
- There is a debate around duplicated or linked-to content
- The need for a reliable cross-search facility
- The value of metadata and upload tools to remove the need for manual upload of separate metadata record
- The idea that repositories could different degrees of access/openness to different resources, easing the path towards openness for some depositors
Humbox: Repository built on prior projects (JISC-funded Edshare, LanguageBox) which have found that the upload process must be kept simple with minimal metadata required.
Bioscience interim findings:
- Web2.0 resources difficult to trial with repositories until completed
- Academic staff in general appear relatively unfamiliar with the use of repositories of learning and teaching resources [but] many institutions are beginning to populate their own repositories and these would benefit from being networked to enable a cross-search.
- there is duplication of effort with upload to JORUM and other host site e.g. local e-Docs, web 2.0 sites.
- duplication of effort is also involved in uploading to a second repository (Merlot) even though this is by syndication, as the metadata has to be entered again. However, the advantage is that this brings a new audience for the resources.
Project outputs and evidence
Humbox, SimShare Legal, TRUE: feedback on user interface for resource discovery and community interaction
S4S reflections on use of UofH e-docs repository system
ADM-OER evidence that existing institutional repositories have been successfully populated with OERs appropriate for ADM UG programmes.
OER CSAP: a range of hosting solutions to be assessed for: fitness for purpose; ease of use; potential for sharing and review. Through consultations with partners; feedback from workshop users; observation; interviews
OER MEDEV: API Toolkit, documenting use of existing APIs to develop workflows and content management to upload ER to JorumOpen and other hosting sources
C-Change: extensive comments on the JO interface, reproduced on the JO blog
What issues arise when using public/third-party hosting solutions?
Findings from projects
Projects explored the implications of using a wide range of third-party, ‘web 2.0’ solutions to sharing open content more widely, e.g. youtube, slideshare, flickr, scribd. Less widely explored were university SL islands and i-tunes-U. With this approach, consistent cross-platform tagging is essential to allow cross-searching, aggregation, and tracking of resources. Many projects are using RSS feeds to aggregate outputs, e.g. on a project web site or blog. Some projects used RSS feeds to syndicate metadata to different host sites.
Early signs are that the use of third-party sites can bring in users who would not arrive at OERs through a repository route, and can sometimes generate impressive usage stats. They potentially offer more information about the use of OERs ‘in the wild’.
Problems encountered included::
- there is currently no Web 2.0 service which supports Flash animations
- inconsistency in Web2.0 sites which are at different stages of development and have different levels of stability/sustainability
- concerns about resource ownership e.g. Flickr does not allow upload by proxy – the person who has up-loaded the resource is, by default, the resource owner
- offer very different levels of tracking detail, but do allow users to ‘follow’ or ‘subscribe to’ a depositor’s account
Bioscience (interim report finding) Web2.0 resources are difficult to trial with repositories until completed
Core Materials: iTunes-U and YouTubeEdu do not allow individuals or projects to create depositor accounts. The Team were unable to find a file-sharing website that supported interactive Flash animations.
OERP: web2.0 sites such as YouTube, Flickr, SlideShare, Vimeo, Scribd maximise exposure of the materials to the largest number of potential users whilst having ease of access and re-use.
Project outputs and evidence
Humbox findings on pros and cons of different open content websites (HumBox, iTunesU, YouTubeEdu)
OERP guidance to partners on use of Web 2.0 services
- on: building personal learning environments in NetVibes and iGoogle; using RSS feeds
- with APIs for batch upload of materials to Flickr, YouTube, Scribd and Slideshare
OER CSAP: findings and feedback on use of Web 2.0 solutions
Bioscience:project feed mobilized to Mobile Phones using QRCODE technology (Kaywa Reader).
OER -CSAP – overview of web 2.0 affordances (part of toolkit)
How best to make hybrid, interactive and multi-media resources available for open access? What have we learned about specific file types and formats?
Findings from projects
Redeveloping complex, interactive materials into suitable formats for open release is a significant investment of technical time and expertise.
With complex resources there is a particular issue of granularity to support repurposing, i.e. whether complex resources should be disaggregated or left intact. Some projects decided that both the aggregated/integrated resource, and its disaggregated components, should be separately available. Several made use of content packaging. However, not all repositories support this successfully. Where there is a need to make clear the educational relationship between component elements, sometimes projects decided that a separate dedicated web site was the only way to acheive this.
With respect to file types, powerpoint was widely used and has the flexibility that files can be linked to assets such as images and video, so used to bring complex content together. Word was also widely used. Questions arose about what version and type of these applications were most ‘open’ and re-usable. pdf was avoided by many projects as being insufficiently open i.e. not allowing re-editing. However, it was found to be a good delivery option when originators were nervous about repurposing and wished to use the security features to prevent changes, e.g. TRUE:” ‘there were cases where academics were keen to have their resources seen but not altered, and uploaded in PDF format as a technical way to discourage editing by third parties’ “URLs in content were variously dealt with: a consist solution could be looked for. Flash movies, especially where multi-file, presented specific problems for open release.
CORE-materials: finding: OERs need to be provided in formats that are generally small scale (in ‘bite-size’ chunks), in order to promote reuse and embedding within teaching and learning.
Bioscience finding: Content packaging is not attractive to the end-user academic. OERs benefit from visible low ‘granularity’ so components can be adopted easily. ” ‘OERs are better delivered in small focused packages that can meet individual learning objectives across different institutions.’ ” ” ‘repackaging software such as Articulate or Engage (and perhaps Xerte) may not produce fully accessible resources, so consider providing a range of materials to meet all needs’ ”
SimShare Legal findings:
- Simluation assets built using the SIMPLE platform are already designed for re-use.
- Assets that were originally XHTML files were converted into word-processor documents; and a viewer was developed that can represent the process of the simulation as a timeline.
- We embedding considerable flexibility into the upload process to the Simshare repository site, acknowledging the wide range of possible formats, but we adopted a single model for describing a simulation and its metadata.
- We encouraged donors to ensure that the simulation narrative was clearly accessible and that the role of different artefacts was clear enough for them to be re-used as free-standing objects.
- Institutional support is particularly important for potential users of simulations to actually engage.
- Repurposing materials from e.g. Authorware to Xerte is painstaking. It is often better to start from scratch, particularly with complex resource types.
- Not all repository structures allow for the effective presentation of related multiple resources into a coherent learning pathway. In some instances it may be better to create a separate website to support such resources.
- The current Digimap39 licensing does not allow OS mapping to be stored in open access repositories. However, this situation is under review for the next Digimap licensing agreement.
- Use of Microsoft Office Creative Commons Licence Plug-In not recommended.
- Presentations can be made more usable and discoverable as follows: deposit presentation in JO and Slideshare; reference embedded resources in Flickr (.jpeg) and YouTube (.wmv).
Project outputs and evidence
HumBox list of resource types
ICS guidelines (‘basic principles’) on design and presentation of reusable learning objects
- repackaged aluMATTER modules, redeveloped in Flash to allow them to be downloaded and run on local systems
- TLTP eCorr package (orig. Toolbook) in Adobe Flash
- [[www.core.materials.ac.uk/guides | guide to portability of interactive materials]
TRUE: evaluation of approach to granularisation of resources, to enhance repurposing
SimShare Legal: shareable and re-purposable simulation designs and associated resources (generic files, SIMPLE format); reflections/findings on the practical issues around sharing simulations as complex, multi-layered, multi-format entities
- powerpoint presentations in JO and Slideshare, referencing images in Flickr (.jpeg) and YouTube (.wmv)
- reflections on use of Microsoft Office Creative Commons License Plugin for relevant file types and on use of specific metadata tags with images (see metadata section)