Notes from team meeting 23 February

Development

Work on authentication complete. Blog post to follow. User authentication working over OAuth. API framework in place. Recruited Harry Newton as Developer on Orbital, starts end of March. Jenkins Continuous Integration quality assurance environment set up. Staff profiles producing RDF dump. EPrints integration awaiting work from EPrints Services.

Work packages

WP3 complete
WP4 complete end of Feb
WP6 complete end of Feb
WP9 complete end of Feb
WP10 complete end of Feb

Events

Dev8D/CodeIgniter conferences (14-19/02). Helped organise both conferences. Nick presented on API-driven development, Lightning Talk on OAuth, part of data repositories panel, both presentations featured on Slideshare home page. Joss Chaired data.*.ac.uk panel.

Business case for open source (Lincoln, 07/03 am)

MRD Policy meeting (Leeds, 12-13/03)

RDM Policy drafting workshop (Lincoln, 02/04 am)

OR12 paper/workshop (05/03 deadline). API-driven development.

MRD HackDay (Manchester, 3-4/05)

MongoDB conference (London, 20/06). Lee/Harry.

Actions

Nick: Workshops on source control, testing, CI and deployment. 3rd week of March.

Nick: Documentation, write up tech review, data review.

Nick/Lee: Review choice of database.

Nick/Mark: Review security.

Nick/Bev: Meet to look at metadata requirements

Joss: Write Implementation Plan

Paul: Arrange meeting with Nottingham/ADMIRE project.

Paul: DAF survey

Paul: Lit Review by end of week.

Building on the ERIM and REDm-MED projects

On January 20th, Dr. Mansur Darlington from the ERIM & REDm-Med projects came to Lincoln to discuss his work in relation to the Orbital project. Mansur has a consultancy role on the Orbital project and will be joining us again later on in the year, to help us evaluate our progress. It was a very useful and interesting meeting for all of the Orbital Team and the Engineering Researchers working with us. What became clear to us is that while ERIM offers the Orbital project a great deal of the underlying research and analysis of how Engineers work with data, Orbital can reciprocally feed back observations and issues arising from ERIM’s recommendations, which are theoretically robust but have not yet been tested in implementation. Similarly, with the REDmMEd project, which finishes in May/June, I hope that we can take the outputs of that prototyping work and build on them in the development of Orbital.

Here are Mansur’s slides from the meeting and below that, my notes.

  1. Purpose of the meeting
  2. Introductions: Bev, Annalisa, Bingo, Chunmei, Joss, Stuart, Lee, Mark, Nick, Paul, Mansur. Apologies, Chris Leach.
  3. Engineers: Bingo, Chunmei, Stuart
  4. See slides. ERIM research offers good spread of Engineering research data.Industry collaboration is vitally important.
  5. MRD in general:

* Need to find out which RC (%), the funding into Engineering School comes from.
* All institutions have to put together a roadmap for RDM by May 2012 for EPSRC.
* Siemens/Lincoln spend a lot of effort in discovery of existing data to base investigations on.
* No national, dedicated Engineering data archive
* Need to look at API integration with DPMOnline (DCC)
* Orbital as tool for managing research projects?
* Ask DCC to visit Lincoln for Policy development and training.
* Reporting to DCC is a formal requirement.
* Include costs of MRD in the university overhead when bidding for funds.
* Datasets as an outcome of research projects. More ‘efficient’ to deal with RDM as part of project.
* ‘Market’ for data. Expectation of costs and benefits of MRD

6. The Nature of Engineering Research Data:

* ERIM: Engineering Research Information Management: Research activity data as well
* Problems with terminology. Need for definition. Both theoretical and practical/empirical outputs from the project.
* Good slides for terminology and understanding domain
* How does Orbital fit into the VRE puzzle?
* Transparent logging and capture of as much activity data as possible.
* Knowing the context is vital for understanding data. Orbital needs to concentrate on contextual data as much as ‘research data’.
* Orbital supports research lifecycle from bidding to completion?
* ‘Engineering research data’ covers pretty much all types of data.
* Need to identify other types of Engineering users to broaden scope of ‘Engineering data’
* Look outside Engineering for variety of data types/activity. Look beyond Engineering. Generalisable.
* Data types is one thing; methodologies and the data they produce are another.
* We manage data so that it can be RE-USED (by someone)
* Must not add to bureaucracy of research

Report to the Research, Innovation and Enterprise Committee

Part of the Orbital project governance is that I report to the university’s Research, Innovation and Enterprise Committee. The Committee meets every three months and I send a short report to each meeting and attend every other meeting. Here’s my report for the February committee meeting.

The Orbital Project

Progress report to the Research, Innovation and Enterprise Committee

30th January 2012

Author: Joss Winn, PI/PM.

Progress since the last update to the RIEC on 13th December:

  1. The short-term focus for the project continues to be the development of the technical infrastructure for managing research data, while being mindful of the long-term requirements to develop policy and a supportive environment for research staff.
  2. Software development has begun. We have finished setting up the development environment for the Orbital system. This is a major software development project for the university and we have spent some time designing the server architecture and quality assurance procedures for development.
  3. Orbital will make use of ‘cloud computing’ and is working with ICT as a pilot project for integrating cloud computing into our local infrastructure. A meeting took place with Eduserv, a non-profit provider of cloud computing to the HE sector (running on Janet) and a further meeting is taking place with Rackspace, a major commercial provider of cloud computing services. This work sits alongside ICT’s need to refresh their server infrastructure next year and will provide ICT with a real opportunity to investigate the business case for cloud computing as well as issues around actual implementation.
  4. A full-time post for a Web Developer has been advertised and we expect the post to begin late March/early April. This is the second full-time Web Developer post on the Orbital project.
  5. We are pleased that Dr. Ling from the School of Engineering and his PhD student, Chunmei Qing, will work with closely with the Orbital project in the development of the software, policy and training materials. Similarly, we are working with Prof. Chris Bingham and Stuart Watson (Siemens), and have recently joined their fortnightly research meetings, which are extremely useful to the Orbital project. At this stage, we welcome involvement from any Researcher in the School of Engineering and further into the project intend to broaden our use cases to other research disciplines.
  6. A meeting has been held with Dr. Mansur Darlington from the University of Bath. Dr. Darlington led the JISC-funded ERIM project, which studied the Research Data Management (RDM) issues for the discipline of Engineering.[1] The meeting was very useful for the Orbital team, including partners at Siemens and Researchers in the School of Engineering, who attended. The ERIM project provides a very robust, theoretical basis, which Orbital will attempt to build upon and implement. Similarly, a follow-up to the ERIM project will provide prototype tools, which we hope to build on for Orbital.[2] This is a key external relationship for the Orbital project.
  7. One issue flagged by Dr. Darlington concerned national funding bodies’ RDM policies. Each funding body has an RDM policy which requires universities to have effective methods in place for managing, preserving and disseminating research data.[3] The EPSRC has told all universities that we must provide them with a RDM roadmap by 1st May 2012 and must be compliant with these expectations by 1st May 2015.[4]
  8. The Orbital project is required by JISC to produce an RDM Policy for the institution. A national meeting is being organised by JISC to assist with the development of such policy in March. Following this, I suggest that a workshop is held in March where the Orbital project and other key staff from the Library and Research and Enterprise Office begin to draft this Policy and the required EPSRC roadmap. This can then be presented to the RIEC for discussion and approval prior to submission to the EPSRC.
  9. A meeting has been arranged for March 7th, 9.30-12pm, to discuss the Business Case for Open licenses. This discussion will be of interest for anyone concerned with licensing research outputs (‘Open Access’), software development projects (‘Open Source’), and teaching and learning resources (‘Open Educational Resources’). Staff from the JISC-funded OSS Watch, University of Oxford, will present at this meeting. Andrew Hunter and James Murray will attend and members of the RIEC are also welcome. Please RSVP to Joss Winn by end of February.
  10. Joss is working with JISC to organise a national event focussing on issues around software development for Research Data Management, which will be held in May.

Piloting the cloud

Last month, Nick wrote about how Orbital is being designed as an application to run in the cloud. This week, we met with Andy Powell from Eduserv to discuss the use of their ‘Education Cloud‘ for the Orbital project.

In the run up to this meeting, we’d been talking to colleagues in ICT Services about our need for more flexibility and autonomy when we required servers in order to do our work. Outside of work we’re quite used to spinning up servers on Rackspace or AWS to try things out and increasingly we’ve been looking for ways to take control of our servers in this way at work. We’re not the only Researchers who need this flexibility; colleagues in LiSC have also been telling us that for some of their work, the scalability and reliability of cloud services is looking increasingly attractive.

This is not to say that ICT services is inflexible and unreliable by any means. I’ve always found my colleagues very willing to help where and when they can, but I think we’d all agree that a central ICT department in a university, with the multivarious responsibilities it has, is not the same as a dedicated cloud provider and, in our case, does not offer the resilience nor the scalability that Rackspace or Amazon are offering for example. The availability of resources, the business model and available support are quite different. When I joined the University in 2007, ICT Services were implementing a new VMWare server farm, which has given us more flexibility than having to work with physical boxes in every instance. Typically, if I want a Linux server with 4GB RAM and 100GB of HDD, I put in a request, transfer approx. £1200, and some time later, a virtual server is provided to me at no further cost. If I need more RAM or HDD, I put in another request, transfer some money, and some time later, I get what I need. This process can take weeks or months.

However, our VMWare farm is now almost five years old and nearing ‘end of life’ and I know that ICT are thinking about the next five year cycle and how cloud computing fits into their future plans. Colleagues in the Online Services team have been using Rackspace recently as a CDN for the Common Web Design framework as well as hosting our popular Gateway website, and have been very impressed with the service. The main hurdle was not technical but organisational: billing for the use of the CDN is by credit card and Pay As You Go (PAYG), meaning we don’t know exactly how much it will cost each month. This is in contrast to how departments normally make payments which are known in advance and invoiced in arrears. Nevertheless, that hurdle has been overcome and hopefully set a precedence.

So the meeting we had with ICT Services was in light of all this and we recognised and agreed that Orbital was a timely and appropriate project by which the university could pilot a more extensive use of cloud services and look at how we might integrate servers in the cloud with our existing server farm. It would also allow us to think about new business models where the real costs of running a server are more transparent to everyone, rather than being absorbed by ICT as the server ages.

Nick has been setting up the Orbital development environment and basic architecture (more on that in another post) using Rackspace and the Orbital project pays for this each month via our departmental credit card. This works fine if a) the department is happy to use the credit card in this way; and b) we have dedicated project funds for this, but it’s no way to run a long-term service that is to be sustained by the institution. Our interest is not really in whether we use Rackspace or Eduserv for hosting during the period of our project – both offer Linux boxes afterall – rather we’re interest in working with ICT to ensure that by the end of the project, there are formal processes in place for a) running sustained services in the cloud; and b) providing researchers with the ability to spin up and manage adhoc servers as and when they are required.

The plan is to evaluate both Rackspace and Eduserv over the coming months, looking at which service fits best with the future plans of ICT Services. Rackspace has a much more mature offering, but we’re really keen to work with Eduserv too, recognising that they’re a new not-for-proft provider of cloud services, running on JANET and with a long history of providing hosting and other technical services to HE and government.

At our meeting with Andy, he went through much the same presentation that Nick and I had seen at the MRD start-up meeting, answering our specific questions along the way. He also demonstrated (for the first time??) the vCloud Director interface for setting up and managing the servers, and this should, in principle, integrate with our existing VSphere system. One of the nice things about the Eduserv offering is that unlike most other cloud providers, they provide the entire vCloud Director application to their customers, including a full API, rather than a cut-down interface. We’ve yet to see how vCloud Director will allow us to create access controls for different types of users, but that’s what the Orbital project will be helping to investigate and I’m pleased that we’re able to work with our ICT department in this way. There are other important questions, too, around data protection and liabilities, and Andy was keen that we review Eduserv’s Terms and Conditions and SLA and feed back our thoughts on it.

This experience will allow me to better understand the business model of the cloud and how to make the business case for developing and running cloud-based services. As Nick previously said, it also allows us to make our costs more transparent, too, so that the actual costs (per Gigabyte and per Gigahertz) of managing research data are clearer to both Researchers and the institution. Having a clearer idea of the costs will help us create a more sustainable service in the long run.

Orbital is go! (Team meeting notes 24-11-2011)

Attending: Nick Jackson, Annalisa Jones, Bev Jones, Chris Leach, Paul Stainthorp, Joss Winn

Apologies: Lee Mitchell, David Young

Agenda

  1. Review Project Plan and Workpackages
  2. Status updates: Literature Review, User Requirements Analysis, Technology/Standards evaluation
  3. Forthcoming meetings and conferences (Agile method, Open Source policy, ERIM, Engineers, OR12, DCC, Start-up)
  4. Poster, papers, website
  5. Staffing and accommodation
  6. AOB

Notes

Joss Winn (JW) reported in detail:

  • JW reported on the work done to date (mostly relating to workpackage WP1), and reported back on:
    • The successful first meeting with users from the School of Engineering
    • The first Steering Group meeting on 3 November
    • The submission of the project plan
    • The appointment of NJ as lead developer
    • The relocation of NJ and PS (part-time) to CERD’s offices to work on Orbital
  • JW ran through the project outputs and workpackages in detail, identifying deadlines – most notably the Implementation Plan, which must be submitted by February 2012, with the following four pieces of work completed by then:
    • Data sources (NJ/CL)
    • User requirements (NJ)
    • Literature review (PS/BJ/CL)
    • Technical review (NJ/JW)
  • The group discussed the further user-engagement work to be completed in workpackages WP5, including Nick Jackson’s work with the School of Engineering to assess their requirements (through workshops, questionnaires, observation, and use of the Data Asset Framework – DAF), and on a planned round-table meeting about ERIM in late January
  • ACTION (NJ): dates needs to be set for user requirements exercises.
  • ACTION (PS): Date in late January needs to be set for ERIM workshop with Engineers.
  • PS reported on the work that he and NJ have begun to benchmark against the EPrints deposit workflow (WP8). NJ will work closely with BJ on this.
  • The group discussed WP9—the planned assessment of data sources—and CL’s role as library user. There are three obvious areas where Orbital crosses over with the Library’s priorities:
    • Integration with the Library’s Discovery selection & implementation project (CL)
    • Integration with the Repository (BJ)
    • Authentication (CL)
  • The Research & Enterprise office (i.e. AJ) will lead on WP11 – developing training materials & workshops.
  • JW will carry on the work with the University’s IP manager, James Murray on the correct approach to Open Sourcing code from Orbital – WP13.
  • ACTION: JW to follow up contacts with EPrints Services and OSSWatch.
  • Dissemination (WP14):
    • PS has been invited to speak at two events in January/February. The group will aim to have a publishable conference paper ready by Summer 2012. Submit abstract to OR12 by ?.
    • NJ, PS and JW are attending the project startup meeting in Nottingham on 1-2 December; presenting a poster. Also attending the DCC roadshow in Cardiff in mid-December.
  • Any other business:
    • JW is convening a meeting (8th December) about agile software and project development methods.
  • ACTION: as many people as possible from Orbital to attend ‘agile’ meeting.