Executable Specifications

October 29, 2013

How do we specify requirements? Chatting is inherently imprecise. Email looses context. Even the ideal requirement document starts to decay once it is done. The most precise way we know to store knowledge is code. Could we use code to write specifications that are easy to understand by non-technical stakeholders and that check the behavior of the real application? Read this article and you’ll find out not only that we can, but also what tools to use to do it.

A few years back, I was working in a large outsourcing company. My colleague John, who was sharing the open space office with me, was working with a real estate company to develop a new version of an internal web application that was used internally for finding apartments, houses and offices based on certain criteria. One morning when he arrived at the office he was very annoyed. His customer had call him to report a bug. The bug report was “The search button is not sexy enough”. Real story, except I changed my colleague’s name to protect the innocents.

There are different ways to describe specifications for software projects. They differ from each other in terms of precision. This post will present them from the least to the most precise.

1. Chatting

Situation: You accidentally meet a stakeholder at coffee. They tell you “You know, it would be great if [insert feature request here]”.

This type of requirements has the following sources of imprecision:

  • Verbal communication is a lossy type of knowledge transfer. You might not understand completely what the stakeholder wants. If you tell your team what the request was, the knowledge dilutes even more. It is the start of the telephone game.
  • You haven’t thought at all cases. Unless you have discussed at least the UI changes, the error and edge cases and the acceptance criteria, you shouldn’t start developing.
  • You will not remember the design details that prevent you from implementing the feature. Often when you talk to your team mates about the feature, one of them will tell you “remember that we cannot do X until we refactor the data access code. Well, that uses X.”

Making a commitment to include this feature in a specific release is equivalent to playing roulette with your schedule.

2. Email

Situation: You get an email from a stakeholder with a feature request.

Email is often used to ask for requirements. While email communication can be more precise because you have more time to consider your words before sending the message, it also has one important source of imprecision: Context is lost with email.

3. Document

Situation: Your stakeholder sends you a document with user interface mock-ups, requirements description, validation criteria etc.

A good requirement document means that the terms are well defined, it covers all the situations, it contains interface mock-ups, diagrams, graphs and developers can understand them easily. It eliminates ambiguity and minimizes time to read and understand. I’ve seen a few such documents in my time, and I much appreciated their authors. The reason is simple: writing such a document is hard, requires a lot of mental power and a clear head and takes a lot of work.

The sad part is that even the best requirement document ever written starts to decay once it is finished. The code evolves while documents often remain frozen in time. Trusting a requirements document to understand what the application does is like trusting 2000 years old coins to understand how the stock exchange works.

4. Executable Specifications

The most precise way to store knowledge is executable code. Code does not lie. Code has to be written in a precise form or it will not be executed.

Imagine that instead of using a document, you could write requirements in English and execute them against the real application. The requirements would be synchronized with the code. The requirements would be precise. Any stakeholder could add a new requirement, and when it executes successfully would know that it was done. The discussions of the type “this is not what I asked” would end very quickly.

Fortunately, this is not a dream, it is possible. A few types of applications exist that allow for executable specifications:

Fitnesse

Fitnesse is a free wiki that anyone can edit and that uses the you create to run tests against the code. You edit the inputs and the outputs that you expect as tables in a wiki and run the tests from the wiki. It requires a little glue code between the wiki and your code. Here’s a sample from the 2 minutes example:

Fitnesse

BDD Frameworks

Behavior Driven Development is a technique for documenting specifications that involves writing the tests in a specific form that is easily readable by non-technical people. For example:

Feature: Login
    As a user I want to login to the website in order to access my private stuff

Scenario: Correct login
    Given an existing user name and the correct password
    When I click login
    Then I am logged in

Scenario: Bad username
    Given a non-existing user name and a password
    When I click login
    Then I get a message "Bad user or password"
Scenario: Bad password
    Given an existing user name and an incorrect password
    When I click login
    Then I get a message "Bad user or password"

A BDD framework is able to parse this text and use conventions to call the code that you write to support them. In the example above, it will look for methods like An_existing_user_name, I_click_login, I_am_logged_in inside a specific library that you create, it will execute each step and will tell you if the tests passed or not. Of course, this implies that each of the above methods calls the production code to set up data, pass the user name and password specified above and verify that the login actually happened.

Cucumber is the most popular BDD framework, being translated in 20 languages and adapted for Java, C#, C++, PHP. Jasmine is the most popular BDD framework for Javascript. Many other BDD frameworks exist for all tastes and programming languages.

You can write precise specifications, that are easy to understand by non-technical stakeholders. They enable you to have productive conversations around what is required. They enable you to show that features are more complex than just the happy path case. They eliminate the useless conversations of the type “that’s not what I asked for”. They prove when a feature is complete.

Writing them before the code is finished (as it is normal for requirements) means that you are doing Acceptance Test Driven Development, or ATDD. Business analysts, testers and developers will benefit equally from this practice, no matter if your project is developed in an agile, lean or traditional model.

Are you trying to write good requirements specifications? What challenges do you face? Let us know in the comments!


If you want to better understand executable specifications, we can help with:

Want to learn something else? Let me know and we will create a customized package for your needs.

Image source: http://farm1.staticflickr.com/61/172600054_f8787e8ed1_o.jpg

Categorised in: , ,

2 Comments

  1. We are on a fast track towards executable specifications using Specflow and Behat BDD frameworks initially and using Relish for publication via Github.

    It is clear that the difficulty teams are primarily faced with are not necessarily technical or with the tools that are used (although these are real and need to be addressed) but more in clearly defining the desired behaviour in a clear and ubiquitous language. The language often used by teams learning to write specifications by example is often imprecise and not necessarily testable, therefore teams needs time to mature and develop their writing and language skills in this context.

    In addition, promoting collaboration through specification workshops that actively involves the whole team in creating and updating specifications presents a challenge to culture and conventions where it is viewed that specific roles have specific responsibilities in the process.

    • Thank you, Ian, for your excellent contribution.

      Indeed, BDD and ATDD are first of all techniques for requirements elicitation, which is a fancy way to say that they allow stakeholders and technical teams to define a common understanding of what needs to be done and of what shouldn’t be done. Defining common understanding is harder than it sounds, because different roles have sometimes very different views. That’s where ubiquitous language comes into play.

      Any change in an organization faces challenges related to culture, shifting responsibilities, changing roles, normal human resistance to change etc. We plan to treat in more details the problem of organizational change in future blog posts. Thank you for contributing your experience to the conversation.

Leave a Reply

Your email address will not be published. Required fields are marked *