A JISC-funded Managing Research Data project

Posts tagged Agile software development

As part of Orbital’s development we need to keep what we’re doing on track, and ensure that what is produced is actually what people are after. We’re building the project using agile development methods, which mean that instead of generating a load of documentation and exacting requirements up front and then building software, we generate a basic set of requirements, start developing and then return to look at new or changed requirements at regular intervals.

Keeping tabs on this kind of thing requires a management tool, and in our case we’re using the wonderful Pivotal Tracker, and here’s why.

Pivotal allows us to break down user requirements (gathered through a variety of means, including meetings, surveys, observation and so-on) into discreet bundles called ‘stories’, each of which represents something that a user needs (or wants) to be able to do with the final product. An example may be “project administrators must be able to assign roles to project users”, or “users must be able to manually add a data point”. By creating these stories it starts to become clearer what actually needs to be done.

From there we can start to fully analyse each of these stories, providing them with information such as a ‘score’ of how difficult to achieve each story will be, or including sub-tasks for actual development purposes. Stories can be assigned to various people based on who needs to be involved, and go through a clearly defined workflow of being started, being finished, being delivered in a product version and being approved by the customer.

On top of this management of user stories we can also pack out Pivotal with higher-level package deliverables and deadlines, along with bug reporting and general project chores. Once we’ve got all these things into the Tracker we’re able to re-order them as priorities shift, giving us an instant overview of what’s happening in the current iteration (a 2-week long development cycle) as well as what’s going to be happening in future iterations. At this point, Pivotal Tracker comes into its own with something called ‘emergent planning’.

Emergent planning takes a look at how we’re actually performing in terms of crunching through our list of user stories and dynamically adjusts which stories we’re going to be tackling in upcoming iterations. If we’re doing well we begin to see more points worth of development per iteration, and if we’re slipping then Tracker gives us fewer. Since we’ve told Pivotal what needs to happen before certain deadlines are met (when we ordered stories and tasks), and since Pivotal knows roughly how fast we’re working, it’s easy to see if we’re predicted to hit or miss development milestones.

Want to see what we’re up to? Our Pivotal Tracker project is open for you all to see.

Attending: Nick Jackson, Annalisa Jones, Bev Jones, Chris Leach, Paul Stainthorp, Joss Winn

Apologies: Lee Mitchell, David Young

Agenda

  1. Review Project Plan and Workpackages
  2. Status updates: Literature Review, User Requirements Analysis, Technology/Standards evaluation
  3. Forthcoming meetings and conferences (Agile method, Open Source policy, ERIM, Engineers, OR12, DCC, Start-up)
  4. Poster, papers, website
  5. Staffing and accommodation
  6. AOB

Notes

Joss Winn (JW) reported in detail:

  • JW reported on the work done to date (mostly relating to workpackage WP1), and reported back on:
    • The successful first meeting with users from the School of Engineering
    • The first Steering Group meeting on 3 November
    • The submission of the project plan
    • The appointment of NJ as lead developer
    • The relocation of NJ and PS (part-time) to CERD’s offices to work on Orbital
  • JW ran through the project outputs and workpackages in detail, identifying deadlines – most notably the Implementation Plan, which must be submitted by February 2012, with the following four pieces of work completed by then:
    • Data sources (NJ/CL)
    • User requirements (NJ)
    • Literature review (PS/BJ/CL)
    • Technical review (NJ/JW)
  • The group discussed the further user-engagement work to be completed in workpackages WP5, including Nick Jackson’s work with the School of Engineering to assess their requirements (through workshops, questionnaires, observation, and use of the Data Asset Framework – DAF), and on a planned round-table meeting about ERIM in late January
  • ACTION (NJ): dates needs to be set for user requirements exercises.
  • ACTION (PS): Date in late January needs to be set for ERIM workshop with Engineers.
  • PS reported on the work that he and NJ have begun to benchmark against the EPrints deposit workflow (WP8). NJ will work closely with BJ on this.
  • The group discussed WP9—the planned assessment of data sources—and CL’s role as library user. There are three obvious areas where Orbital crosses over with the Library’s priorities:
    • Integration with the Library’s Discovery selection & implementation project (CL)
    • Integration with the Repository (BJ)
    • Authentication (CL)
  • The Research & Enterprise office (i.e. AJ) will lead on WP11 – developing training materials & workshops.
  • JW will carry on the work with the University’s IP manager, James Murray on the correct approach to Open Sourcing code from Orbital – WP13.
  • ACTION: JW to follow up contacts with EPrints Services and OSSWatch.
  • Dissemination (WP14):
    • PS has been invited to speak at two events in January/February. The group will aim to have a publishable conference paper ready by Summer 2012. Submit abstract to OR12 by ?.
    • NJ, PS and JW are attending the project startup meeting in Nottingham on 1-2 December; presenting a poster. Also attending the DCC roadshow in Cardiff in mid-December.
  • Any other business:
    • JW is convening a meeting (8th December) about agile software and project development methods.
  • ACTION: as many people as possible from Orbital to attend ‘agile’ meeting.

I am currently completing the Orbital Project Plan prior to submission to JISC next week. The writing of a Project Plan, using JISC’s template, which I think is a derivation of PRINCE2 documentation, is undeniably a useful exercise in defining the project we’re embarking on. It is also undeniably a tedious process, too, as it requires a particular style of thinking and writing: granular, incremental and forward looking, yet reflective; ambitious and creative yet restrained; serious yet mostly in a dumb tabular form. I find myself having intense bursts of concentrated writing and then having to step away from the document  to restore myself both physically and mentally.

I’m currently at the Quality Assurance section of the document, which is in a tabular format to aid both author and reader. However, what I really want to write is this, taken from the Agile Manifesto:

We follow these principles:

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

Welcome changing requirements, even late in
development. Agile processes harness change for
the customer’s competitive advantage.

Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.

Business people and developers must work
together daily throughout the project.

Build projects around motivated individuals.
Give them the environment and support they need,
and trust them to get the job done.

The most efficient and effective method of
conveying information to and within a development
team is face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development.
The sponsors, developers, and users should be able
to maintain a constant pace indefinitely.

Continuous attention to technical excellence
and good design enhances agility.

Simplicity–the art of maximizing the amount
of work not done–is essential.

The best architectures, requirements, and designs
emerge from self-organizing teams.

At regular intervals, the team reflects on how
to become more effective, then tunes and adjusts
its behavior accordingly.

I think I will print this on A3 and stick it to our wall.

What attracts me most to Agile methods of software development (I lean towards XP), is the emphasis on human interaction and the focus on values such as trust, respect, simplicity, autonomy and courage. All too often when running a project, the objective of delivering the product dominates and diminishes the creative and social process of producing something that improves our environment.

For me, as Project Manager, the Orbital Project is not only an interesting Research and Development project but also an opportunity to practise a method of human sociability and creativity over a defined period of time. Although I’ve tried to use attributes of Agile methods on projects in the past, this is the first time that I’ve started a project from scratch with this is as the principle method, and a project where I know the Lead Researcher and Lead Developer are likewise keen to work in this way.

This blog is a record of our project over the next 18 months or so. For my part, I’ll be reflecting honestly about the ups and downs of running the project and learning to work closely with people according to the principles quoted above. I’m sure we will fail at times and the process will get lost to the product, but we will learn, even during those times. And gradually, we’ll get better and produce better.