Thursday 25 June 2009

The REF: Results of Pilots and Future Developments

I recently attended a one day programme organised by Kings College London on The REF: Results of Pilots and Future Developments. Supported by HEFCE, the day gave those involved, and not involved, with the pilot alike the opportunity to learn more about the planned direction of the REF. The day consisted of a number of presentations to all delegates in the morning including two speakers from HEFCE, with parallel sessions on various related topics running in the afternoon.

A large emphasis within the REF Pilot had been on citation analysis, with debate circulating as to the use of such analyses to replace the peer review process used previously within the RAE. What papers would be considered within the REF was also the subject of debate with the pilot considering three possible methods of identifying academic’s papers for citation analysis (as explained to the WRN at this month’s Gregynog Colloquium Repository Stream by Lyndsey Savage, Bangor University): all academic papers identified by author’s name; all academic papers by author’s institutions; selection of academic papers for authors.

From attending the London event, it seems that the REF will bear a resemblance to the previous RAE, with a selection of an author’s best papers being put forward for peer- review assessment, and the citation analysis for each paper being provided to inform the panel. What still remains unclear is which sources will be used to inform the citation analysis. Web of Science and SCOPUS were both used within the pilot but concern was expressed by a number of delegates within one of the parallel sessions I attended that their institution either subscribed to one service or the other, and not both. The use of Open Access sources such as repositories was not going to be used to gather citation analyses as far as I could tell from discussions.

The assessment of ‘impact’ of research will also form part of the REF, a factor that none of the discussions I'd heard previously had even considered. How institutions will construct, record and store such information along with the author data, the research ouput data and the citation data is a new challenge to be met.

The use of repositories for the management of such information to construct an REF return was touched upon. Two JISC projects presented in one of the afternoon sessions were R4R (Readiness for REF) and a project at the University of Reading, with both looking specifically at how a repository can be developed for this purpose.

The second afternoon session I attended ‘ICT implications for the REF,’ discussed research conducted on behalf of JISC that surveyed REF pilot institutions as to their use of ICT in preparing information for the pilot. The study found that the use of ICT was varied between each institution, and that institutions would have to develop much better processes for capturing research outputs, the crux point for all being trying to successfully link research output data to staff data.

A number of systems were being used within the institutions of the attendant delegates, with both research management systems and repositories in place. A second JISC study, ‘Repositories and Research Management Systems,’ found minimal integration between the two systems or their processes when in place within an institution. Some of the newly funded JISC projects, including Cardiff University’s I-WIRE project, which are looking as to how such systems can be integrated were presented on the day.

There were around 300 delegates in attendance on the day and it was interesting to see the variety of job titles; with a large majority either affiliated with the research office, the library or the repository. This highlighted to me the number of stakeholders involved in an institution with an RAE/ REF return and brought home further the need for not only integration between disparate systems but collaboration between disparate departments.

Monday 22 June 2009

Gregynog 2009 - Presentations now available

The presentations from the repositories strand held at the recent Gregynog Colloquium 2009 are now available online:

  • 'Copyright & Repositories' Jackie Knowles, WRN Presentation slides

  • 'Multimedia Deposits: Complications and Considerations' HannahPayne, WRN Presentation slides

  • 'EThOS and the Aberystwyth Experience’ Dr. Nicky Cashman, Aberystwyth University Presentation slides

  • ‘Repository@Bangor and the REF pilot’ Lyndsey Savage, Bangor University Presentation slides

  • ‘Integrating ORCA: Cardiff University's journey to an institutional repository with a service oriented approach’ Tracey Andrews, Cardiff University Presentation slides

  • ‘Repository Management: the University of Liverpool experience’ Shirley Yearwood- Jackman, University of Liverpool Presentation slides

  • ‘The Welsh Repository Network: Where do we go from here?’ Jackie Knowles, WRN Presentation slides

  • 'Repositories and JISC' Andy MacGregor, JISC Presentation slides


The WRN team would like to extend their thanks to both the presenters and the participants attending the strand whose enthusiasm and hard work contributed to the success of the event. The extended two day programme we offered this year proved to be well justified with excellent attendance across the board. If anyone has any queries about any of the sessions, or if you would like follow up on any particular topic, then please do not hesitate to contact the team using the usual address.