Following up on our post from the MRDHack day, what follows is an evaluation of ownCloud as an institutional alternative to Dropbox. Our DAF survey showed that researchers at Lincoln require better managed storage space than the current 1GB FTP “H: Drive” provided to each staff member. Many of them are using portable drives, USB sticks and cloud-based services such as Dropbox to store and share their research data. Services like Dropbox provide compelling advantages to more traditional storage. Dropbox provides, for free, double the storage on offer to Lincoln researchers at present; it is always backed up, existing on both the local machine and on Amazon’s servers, it offers versioning for files up to 30 days old with the free account and ‘forever’ for paid accounts, it is accessible from almost all devices with Linux, OS X, Windows, iOS and Android clients available. Files can be published to the web or shared privately with other Dropbox users.
We know, however, that researchers using Dropbox are doing so without a clear understanding of the terms and conditions of the service and would ideally like a similar service to be provided internally by the University, where we can retain control of the data and its associated security.
We were first made aware of ownCloud when D’Arcy Norman blogged about his initial trial of it. ownCloud is an open source tool, which provides the same features as Dropbox (and more). With the release of version 4 in May, it appears to be a credible alternative to Dropbox for institutions wishing to provide a modern storage solution for their staff and students. D’Arcy’s initial experience with ownCloud was promising but he found issues with the syncing of files. Our recent tests of ownCloud have found that these problems have now been resolved with a recent update and what follows is an evaluation of ownCloud version 4.0.6, looking at it from three perspectives:
- ownCloud as a general purpose storage technology for an academic community
- ownCloud as a storage technology for research data
- ownCloud as a technology for integration with Orbital
Storage for an academic community?
ownCloud is an AGPLv3 licensed open source project, which started in January 2010. The project is run by a company, also called ownCloud, which provides commercial services and support for its software. The company resides in both Germany and the USA. The development of ownCloud is also open and supported by standard tools for open source projects: a source code repository, bug tracker, IRC channel, mailing list, wiki and forum. There are currently 13 core members of the project and 34 contributing developers. Development of the code is currently very active with changes made several times a day. The ownCloud project was started by the KDE community (it has no dependencies on KDE) and therefore benefits from the involvement of experienced open source developers. The software is written in PHP and can use MySQL, PostgreSQL or SQLite for its database. As of version 4, ownCloud has the following features:
- Web user interface for file uploads and management of account and other features. Files can also be uploaded from an existing URL.
- Windows/Mac/Linux/Android/iOS synchronisation clients.
- WebDav integration for direct access to file storage. ownCloud has its own WebDAV server.
- Folder and/or file sharing: publish to the web or share with groups or individuals.
- File versioning.
- An API for application integration.
- Previewing for a number of filetypes.
- Server side file encryption.
- LDAP integration.
- Notifications.
- ownCloud can be installed in a PHP/MySQL environment on both Linux and Windows servers.
ownCloud also provides a number of other applications that can be activated or not, including a calendar, task list, contacts management, a built in text editor, image management, and experimental support for FTP, Google Drive and Dropbox integration. These are all available from the built in ‘app store’ and configurable by administrators. Maximum quotas for each account can be specified on an account-by-account basis, and a maximum file upload size can also be specified if needed.
The roadmap for version 5, due in August 2012, list the following:
- Inter-ownCloud Sharing
- Ajax interface
- Mozilla Sync Integration
- Improved permissions
- Mounting of Dropbox and Google Drive
- Improved version control
Storage for research data?
A requirement for Orbital is that researchers have a network drive to organise and, optionally, share their data among research partners both inside and outside the university. The current FTP drive falls short by not offering enough space to many researchers and does not facilitate the sharing of files. ownCloud could meet both of these requirements.
In terms of usability, file storage in ownCloud is simple to access through the web client and researchers who are accustomed to Dropbox or Google Drive, for example, should find ownCloud straightforward to use. With the desktop client, files are held locally and synced to an external server. The mobile client displays the folder and file structure and downloads files when they are selected. Files can also be uploaded from the mobile client.
It should be noted that multiple file uploads do not work in current versions of Internet Explorer. This is not an issue with ownCloud, but rather multiple file uploads are not permitted for any application using IE9 or below. Some applications, including Dropbox, work around this limitation by implementing a flash-based uploader, which does allow multiple file uploads. This problem is fixed in IE10 (under development). Windows users are therefore advised to use an alternative browser when wishing to upload multiple files through the ownCloud web UI.
Setting up ownCloud to use LDAP highlighted a known bug where WebDav and sync clients do not handle case-sensitivity correctly. This will no doubt be fixed soon and a work around was simply to enter our usernames as they appear in our LDAP directory, in mixed case.
It was also found that the LDAP authentication method did not explicitly create users in the ownCloud users database table which prevented “share with” fields from auto-populating with users who had not previously signed in, although further experimentation with the LDAP configuration revealed that it was possible to circumvent this by properly configuring the search parameters, prompting the service to directly query the LDAP service when needed.
As part of the evaluation we are working on integrating ownCloud with our own in-house SSO solution. Due to how ownCloud handles authentication for its external synchronisation clients this needed work both in the ownCloud extension and also to create a new mode of authorisation for our OAuth server to allow direct authentication of usernames and passwords using HTTP Basic Auth. Future versions are planned to include direct OAuth compatibility, which will hopefully address this issue and allow us to fully integrate ownCloud with OAuth. For the time being we have created an authentication extension based on ownCloud’s own LDAP implementation, although this does need some more improvements to speed up user lookups before it can be considered for anything more than a pilot.
Connecting to ownCloud using WebDAV worked fine using the OS X Finder, however, we found that it also uploaded hidden files over WebDAV. We were unable to use WebDAV on Windows 7, even after following the setup instructions.
Although WebDAV is a widely adopted standard, it does not offer the benefit of local/remote sync, which the native ownCloud clients provide. We feel this is a useful feature as it means that copies are always held in two places (the local PC and the university server), and were we to roll out ownCloud to the university community, we could use commodity storage hardware and cheaper backup regimes than if the data was only stored on the university servers.
Configuration of the native ownCloud clients assumes that you’ll want to sync a specific folder in the ownCloud user’s root directory, rather than the entire user space, which seems unintuitive to us. Overall, the sync client is not as simple to set up as the Dropbox client but with clear documentation for researchers it would be fairly trivial to set up. With the recent adoption of ownCloud by the Fedora project as well as SUSE and Ubuntu, it is likely that there will be a renewed focus on the desktop client.
Integration with Orbital
If we are to adopt ownCloud for the Orbital project, it should provide the ‘workspace’ functionality that we identified in our earlier Implementation Plan. This is networked file storage that allows researchers to store and share project files and research data. It should integrate with the researchers’ desktops and integrate with the Orbital Manager web application. Researchers should be able to select files from their workspace for analysis as ‘dynamic data’ (i.e. depositing the data from a CSV file held in ownCloud to Orbital’s MongoDB datastore). A researcher should also be able to archive and publish files held in their ownCloud workspace for permanent citation (i.e. by moving them out of ownCloud and into Orbital’s file archive).
Integrating ownCloud with Orbital requires the ability for Orbital to read and write from and to the ownCloud file storage system, as well as the ability to create new folders and manage sharing of those resources. At the moment ownCloud lacks any kind of web-accessible API which Orbital would use (although conversations with developers have revealed that this is in the pipeline) meaning that the only way to allow Orbital integration at the moment is to create our own ownCloud ‘app’ which sits on top of the ownCloud core and exposes the necessary functionality. This is certainly within the realms of possibility given that other ownCloud apps rely on similar functionality, such as the Bookmarks app allowing a bookmarklet to remotely create links.
Orbital’s initial integration will be a read-only view of the available files in a shared folder, but this will then be expanded over future iterations to allow:
- Automatic creation of shared folders when an Orbital project is created.
- Automated adding of project users to the shared folder for instant access to a project’s shared workspace.
- The viewing of a workspace’s contents within the Orbital web interface.
- The ability to use ownCloud as both a source and destination for Orbital’s data workflow, allowing files to be moved from ownCloud to either Dynamic Datasets or Archived Files, and vice-versa.
Ultimately ownCloud addresses a great many needs out of the box, and although it is not yet completely ready for institution-wide adoption it’s rapid development cycle and ease of extensibility and integration make it an ideal solution for Orbital’s Workspace aspect.
This is a really interesting article. We work with research institutes and universities across the country and it still astounds us when we are told people are still using flimsy consumer-based products or USB sticks to store their information.
As a reaction to this we have spent a lot of time and money investing in our own file sharing platform. A platform that has taken usability and security to the Nth degree. It is a product we has already been moved out to our customer based on a free trail.
head to http://www.probox.eu for more information.
Hi Joss
I would agree that it is probably true that “researchers using Dropbox are doing so without a clear understanding of the terms and conditions of the service” – but then again this is probably true of most services, including in-house services.
I’m interested in the evidence you may have that researchers “would ideally like a similar service to be provided internally by the University, where we can retain control of the data and its associated security“. A few years ago Chris Sexton, IT Services Director at the University of Sheffield suggested that staff at the University wanted to make use of the GMail service which had been made available to the students. More recently, as I described in a post entitled“Our students love Google!”: Thoughts on the Strategic Web Team a student survey at Sheffield University showed the popularity of a range of Google products.
My question is then is it researchers who would prefer in-house solutions or perhaps researchers at <strong Lincoln University who have such preferences. Note I recollect a few years ago a survey which suggested that, unlike Sheffield, staff at Oxford University wanted their email to be provided in-house. So I do wonder there are different institutional preferences.
Brian Kelly, UKOLN
Hi Brian,
The evidence you ask for is to be found in the results of our DAF survey, in particular the comments from respondents, which have not been published.
The Orbital project is a pilot project with the School of Engineering and has always has a focus on the security and commercial confidentiality of research data. The use of third-party cloud-based services for storing our commercial research partners’ data on is unacceptable. Orbital must meet or exceed the existing security measures in place, and the use of Dropbox had already been ruled out by the research group we’re working with.
The implementation of ownCloud is not to rule out or ‘ban’ the use of Dropbox, but simply to provide an alternative that integrates with other tools we’re developing. Some researchers may prefer to carry on using Dropbox – I’m sure I will too – but other researchers have expressed a need for better, managed storage for their data which is provisioned by the university.
It is not just the use of Dropbox either, but the widespread use of USB sticks, DVD-Rs and external HDDs, which an implementation of ownCloud might also help cut down on, too.