BDD and Automated Testing

Behaviour Driven Development (BDD) is a framework designed to aid communication and collaboration during the specification, development and testing phases of software development projects. The aim for those implementing this framework is to create a shared understanding between business users, developers and testers at the start and maintain it throughout the life of the project.

Purpose

The overall purpose of BDD is to improve communication between all stakeholders in a software development project. It is all too common that business users, developers and testers end up talking at cross purposes. This happens when each role describes what’s required in their own language, according to their particular concerns, and from their own perspective. For example, a developer might be concerned about the low-level logic and implementation, but a business user may be more concerned about the business logic and graphical user interface. While their desired outcome might be the same (e.g. some new behaviour in the application to meet a particular business scenario), different roles tend to focus only on the aspect of the project that concerns them (i.e. business need, coding and testing, respectively).

Now, misunderstandings might arise simply from a lack of discussion and working through the requirements. Or it may be that the parties are talking but not understanding one another or able to make themselves understood. Formalising the way in which software features are discussed and agreed upon can go a long way to removing these unnecessary obstacles.

BDD aims to define the way in which communication between stakeholders should take place – how it should be specified and documented. The way in which the requirements and specifications are defined is critical because if the different parties are speaking different languages, it will be very difficult to reach a consensus.

Through providing a predefined layout and terminology, the BDD framework moulds each person’s ideas into a simple logic that everyone can understand. Those who are inclined to communicate too simplistically are forced to flesh out their ideas, and those who are inclined towards complexity are forced to distil their ideas.

If a common language is agreed upon and used by everyone, this will provide a degree of consistency. Once that agreed way of communicating is in place, it is significantly easier to define the desired behaviour of the software in a way that business user, developer and tester can understand.

In short, the purpose of BDD is to create a framework that allows business users, developers and testers to discuss and define requirements in a language they all understand. This leads to tight, clear and detailed specifications that can be used from the start of the project (capturing requirements) to the end of the project (regression testing).

Ultimately, this approach should lead to a much more precisely defined requirements. The knock-on effect of this is that the testers can engage earlier in the process and help to identify potential issues before a single line of code is even written. This leads to fewer defects being created in the first place, less reworking later on, and less maintenance required over time.

Thus, there are two main things to be created during the specification phase of the project:

  • Clear definition of the business rules
  • Clear definition of the acceptance criteria

If the business rules to be implemented are clearly defined, it will be easier for the developers and testers to do their jobs. It will be possible to create acceptance criteria for each stage of the project, ensuring that what’s being developed is always in line with the delivery requirements.

And if everyone in the team has been part of defining the business rules and acceptance criteria, this brings together the business users, developers and testers in a way that hasn’t ever been achieved with other approaches.

In fact, this approach helps to provide business users with an understanding of the complexity of implementing their requirements. It also provides the developers and testers with insight into what the business users are really trying to achieve. All of this removes the communication barrier between the business and IT teams. By identifying issues during the specification phase rather than during implementation, we can prevent defects creeping in from the start right through to the end of the project.

So, three core advantages of the BDD approach are:

  • Shared understanding from the start
  • Prevention of defects in the early stages

Acceptance criteria that can be turned into automated tests

Principles

The principles underlying the BDD approach come from working with its use of domain-specific language that is based on natural language. That language is used to define the desired behaviour of the component, feature or functionality that needs to be implemented. In creating that definition the different parties work within a defined structure that forces them to write the specifications in a way that everybody can understand.

Domain-specific Language

The starting point, then, is the Domain -Specific Language (DSL) that is used to communicate concepts and ideas within a particular area of a business. For example, in financial trading the DSL uses terms like shares, futures, puts, calls, etc. If you used these terms in a different domain they might not make a lot of sense. Within your specific business domain it is accepted that these DSL terms carry a meaning. Every business domain has an accepted glossary of DSL terms that are understood by all business users, developers and testers. You can think of a DSL as simply a list of terms used within a business domain. You may have all in that domain. All the meanings of those terms may be defined, but you need a way us still need to string them together so that they have context. This is where we start to explore the type of language used.

Natural Language

In the early stages of a project, you could the developers may be tempted to dive straight into coding. In this case, the coding language is being used to define the requirements, behaviour, tests, etc. However, the business users are unlikely to understand the coding language and are, therefore, left out of the process.

That is why BDD emphasises the importance of using natural language to define these things. A ‘natural language’ is a language that has evolved naturally without any conscious planning; that is, it is simply the language you speak, whether (English, French, Spanish, etc.

The reason for this is so that all three parties involved in the specification process can understand each other. There is no point starting out with code because the business users are unlikely to understand this.

So natural language is used at this stage as it is the one language that everyone can understand. ), which has evolved naturally without any conscious design. So, with BDD, natural language is used from the outset so that everyone working on a project can understand what is required.

However, natural language is not known for its compactness and simple structure. There is an infinite number of ways a specification could be written. Because any given statement can be expressed in many different ways, BDD uses a defined structure to impose simplicity and universality on specifications.

Defined Structure

Imposing a predefined structure on a natural language reduces ambiguity, making it easier to focus on the behaviour of the component, function or unit. Allowing complete freedom would result in everybody writing their specifications in different ways, which would inevitably lead to misunderstandings. So, by using natural language, everyone can understand the words, and by limiting the way it is used, everyone can understand the content being conveyed.

One method of simplifying a specification is to use the Given-When-Then construct, which is part of the Gherkin language (more on this later). This construct forces a degree of organisation and clarity into the specifications that then makes it easier to write the code and create the test cases (both manual and automated).

Ultimately, this BDD approach to defining specifications becomes the ubiquitous way of communicating between all stakeholders, about every element of the project. That is, it becomes the way used by everyone to describeformat for all stakeholders to communicate about the desired behaviours of the components, features or units that need to be implemented. When this approach is used ubiquitouslused in this way, it can then be integrated with a range of software development and testing tools (e.g. Cucumber).

This is considered an outside-in approach to defining specifications, because we start by defining the functionality and logic at the user interface level. That definition could then lead to more detailed specifications at lower levels as work progresses towards the innards/guts of the systemMore detailed, lower-level specifications can then be built on top of this definition. The whole process is geared towards spending more time focusing on the user requirements and capturing those requirements in detail at the start.

Paradoxically, this highly structured format actually frees you up to focus on getting the requirements right. It leaves you no room to waste time thinking about the way in which youhow to write the specifications, or by writing unnecessary waffle.

and removes the temptation to waffle needlessly.

Practice

In practice, the act of ‘doing’ BDD comes down to just two disciplinprocesses: the ‘Discovery Workshops’ and the production of the ‘Executable Specifications’. The discovery workshops are where the business users, developers and testers get together and thrash out the specifications. The product of defining these specifications results in ‘Ethese workshops is the set of executable Specifications’ that will be used from start to finish.

Discovery Workshops

The discovery workshop is the part of the process where the three main roles come together to define the specifications. These roles are business user, developer and tester, sometimes referred to as ‘the three amigos’. They are the key players with the greatest knowledge, experience and talent to create accurate and implementable specifications.

The aim of bringing these key players together is to build thata shared level of understanding. Understanding that ultimately leads to of the acceptance criteria that will drives the rest of the development, testing and implementation of the feature or functionality ..

Those acceptance criteria are defined using the given-when-then syntax with concrete examples that help drive a shared understandingand accompanied by concrete examples. There should be no theoretical, high-level, abstract concepts written in a way that some peoplemembers onf the team can’t understand. Concrete examples that are written in a format that is clearly understandable by user, developer and tester.

The key to

An important part of these discovery workshops with the three amigos is defining the scope and size of the feature or function that needs to bebeing specified. Success comes from breaking large pieces of functionality up into smaller more manageable chunks that can be worked on in short 30ñ to 40 ‘minute sessions. Anything longer than this needs to be broken down further. When you start breaking it down it’s not unusual for it suddenly to become apparent just how big a piece of work you have on your hands. This is the start of the shared understanding that makes BDD so useful. When the three amigos start working on this together, that shared understanding quickly starts to driveIt’s not unusual at this point to begin to realise just how big the project really is. This is an important outcome of the process: when the team begins to collectively develop a more realistic view of what can and can’t be accomplished within a sprint.

Bear in mind that the three amigos don’t have to be limited to just three people. You could have more than one business user, more than one developer and more than one tester. Each of these roles is critical; therefore,The important thing is that there should be at least one of each. There’s nothing to stop other roles also playing a part ñ perhaps a UI specialist, DB analyst or a performance tester. If those rolesey are likely to havmake a degree of useful inputuseful contribution to the process then, include them, too as well.

How, then, should a Discovery Workshop be run? The typical approach is for the business user to present the problem or opportunity they think can be addressed by the addition or modification of functions/ or features. He or she will walk throughcarefully explain thise problem or opportunity and explain it to the others in the meeting to the others using concrete examples, and showing why the current functionality falls short and what needs to be implemented in order to deliver the solution to the problem or the enhancement to capitalise on the opportunity proposed solution or enhancement. It is important that the business logic is clearly explained together with any restrictions that may be prese need to be taken into account.

The other members of the meeting then ask questions based around to apply the given-when-then structure. So the business user explains a potential requirement and the others drill down into that requirement by asking detail pointed questions like ‘if this is given, when this is the situation, then what would the outcome look like’. While there may be a desire for the developer and tester may be tempted to drill down in more to the technical details, it’s important at this stage to stick to an outside-looking-in approach. Avoid getting drawn into the technical implementation, but ask questions that will help stretch peoples understanding of how that implementation might clarify exactly how the feature should work. The goal is to formulate a set of acceptance criteria, not an implementation detail plan.

In simplified terms, the process is:

  • Present the problem or opportunity
  • Suggest solutions to solve this
  • Question the validity of those solutions

It might seem that each of the above fits neatly with each of the key roles:

  • Problem/Opportunity -> business user
  • Solutions -> developer
  • Validity -> tester

However, in practice any role (business user, developer or tester) can raise problems, present solutions or question their validity of solutions. The key is that this iterative process builds that. In this way, a shared understanding is built between all the roles.

As we’re working towards this shared understanding we’re also thinking about This forms the bedrock upon which the next objective in the process – the creation of ‘ecutable specifications are created.

Executable Specifications

The executable specifications are expressed in the form of Given-When-Tthen Gherkin scenarios. Those scenariosey don’t necessarily need to come out of the discovery workshop. They may naturally come out of these sessions, or they may get created afterwards. The most important outcome is the shared understanding that forms the bedrock upon which to create the executable specifications.

Executable Specifications

Executable specifications are meant to bridge the gap between the shared understanding of the specifications and the test cases that will prove that the software delivers what’s required. The shared understanding could might be expressed, to begin with, in the form of a set of rough notes, whiteboard drawings or just in people’s heads. What’s needed from here is’ need to get that information into a format that will aid  make creation  and execution of the acceptance tests. Those acceptance tests might b as simple as possible, whether they are run manually or, ideally, b are automated. What we need though is the acceptance criteria in a format that makes it simple to create the tests. It’s even possible to define them in a way that enables tests to be created automatically (more on this later).

The first thing to point out isEven the creation of acceptance tests can be automated if they are defined in the right way (more on this later).

Be aware that there are different methods or languages that can be used to create executable specifications. The most popular is Gherkin, which is a plain- text language for writing executable specifications in the Given-When-Then format. When defined in this format, and saved to a text file, the fileA file that is written and saved in this format can be processed by a variety of tools. Gherkin’s structured format enables these tools to turn the elements and logic in Gherkinse files into automated tests. So how does that happen?

We start out with a

For example, consider this scenario written in the Gherkin format:

  • Feature: User should be able to place a trade
  • Scenario: The user places a valid trade
  • Given: The user has an account
  • When: The trade data is entered
  • Then: The trade is booked

That is scenario is then saved as a .feature file ñ a plain text file that can then be processed by various tools (e.g. Cucumber).

The big advantages of this method are:

The format of these executable specifications is simple enough for anyone to write, modify and read.

The format is in plain text, which means that itíplain text format makes these files easy to view in many text editors and is simple for external tools to parse.

Everything about this approachThus, the framework if you like, is kept as simple as possible. The complexity, when it comes, is in the domain-specific language. T so that the business users, developers and testers should be can focusing on the content rather than the framework. The complexity is in the desc rip. The complexity is in describing the specification of in the domain-specification: language; that is, inÖ

A. How you break up the specification into manageable chunks,

B. How you represent the detailed logic in the Given-When-Then statements, and

C. How well your description of the scenario matches what you really need to implement in the software.

What we have at the end of this, when it’s done well, isen this is done well, we end up with a living document that represents exactly what the software does. It should continue to represent what the software does be an accurate representation because these executable specifications are defined at the start of the process (specification definition) and executed at the end of the process (test execution). If a test fails, then one of two things need to happen. E: either the software needs to be updated and modified to make the test pass, or the specification needs to changed because it’s incorrect.

Therefore, this whole approach leads to a feedback loop that encourages, almost forces, the team to keep the executable specifications up to date. If the tests start to fail, either the specifications are wrong or the software is wrong. Thus, a test fail is a flag that encourage tells you to investigate the root cause and fix the issue.

Traditional methods without executable specifications tend to lead to documentation that is out of step with what is actually being delivered. And once it’s out of step there’s no motivation, it’s unlikely to be brought up to update it again. There’s motivation to fix the software or fix the test case, but no motivation to go back and fix the documentation. Whereas with BDith BDD, however, you have to go back and fix the documentation in order to make the test case pass. The result is that everything is kept in step and in sync.

This is a big advantage. The Another big advantage is the speed of the feedback loop. With the executable specifications being converted into tests automatically with tools like Cucumber, the tests can be run almost as soon as the specifications are updated. In reality there is some development work for those automated tests behind the scenes, but it’s still a lot faster and a lot more integrated than most other approaches.

This leads to yet another advantage: it becomes difficult to live with vague specifications. If the specification is written poorly, then it’s almost impossible to implement the test. If you can’t implement the test, you are forced to revisit and clarify the specification so that the test can be defined accurately and then work towards getting it to pass. Only once the test is passing do we know that the executable specification is accurate and that the software feature has been implemented correctly. Thus, specification, software and test are kept in alignment. 

This is another reason that BDD has become so popular. Other methods, like unit testing, are prone focusing on testing the implementation rather than what the business user needs. In many ways this defeats one of the core principles of testing.

Testing

In software testing we’re looking for both validation (does the functionality work) and verification (does the functionality do what the end user needs). Many approaches to testing miss the crucial parts of what the user needs. They end up, like unit testing, focusing only on the functionality that’s been implemented. That functionality might work well, but does it do what the end user needs? Approaches like unit testing tend to focus on the functionality that’s been implemented in the module or component. It’s almost too easy to ignore the end user’s needs and isolate the development and testing parts of the process. This approach tends to lead to a lot of rework because it’s only once the final deliverable is complete testing the mechanics of the code and ignore the question of whether the software is delivering what the end user gets a say in what’s been implemented. Often that’s too late.

BDD is a different mindset altogether needs.

With BDD, the business user is involved from the start. T, and the tests are written up front before the coding has even been started (ideally). This forces engagement and involvement up front from all the interested parties. There is no room for exclusion of interested parties with BDD. Without the input business user up front, you’re not practising BDD.

This leads to people thinking in terms of scenarios rather than getting caught up in the detail, forcing them to step back and start seeing the wood from the trees. The team is thinking about the end customer right from the start. And so long as the executable scenarios are being run at the end (which is a key tenet of BDD), the team will remain focused on the business user from start to finish.

Leading on from this then is the question of whether the acceptance criteria should include any internal implementation details. The acceptance criteria should be written for all team members and should not include detail that business users or testers would not understand. They can, and should, include domain-specific details. However, they should not include implementation details. This helps with test maintenance because the code can be modified, refactored, or completely rewritten, but so long as the acceptance criteria remain the same, the automated tests should remain the same. It’s all about abstracting the acceptance criteria defined at the start to a level that removes them from the implementation. In doing so, it makes the automation of the tests, run at the end, far easier to write and support.

Another tough concept for many to work with is the idea that the gun. Thus, the entire process, from planning to development and testing, is moulded around the real-world problem that needs to be solved.

Therefore, tests should not pass when they are first written. They should be failing at the start because the code to be tested is not yet in place. They can be written up front, before the code is even implemented. Nothing unusual there you might think. The unusual bit is that they can, and should, be running before the code is even in place to test. Therefore, start out with lots, if not all, failing tests and work towards getting all of those tests to pass.

This is tougher, in some ways, to implement. There is an upfront investment required. Pulling everyone together to go through detailed scenarios at the start seems time-consuming and counterproductive. The urge to start coding early is always strong. The urge to have something physical to test early on is another strong pull. These need to be resisted with BDD. Deliver the executable specifications up front so that they’re ready to run as the software is being developed. This will result in a lot less rework because more thought has gone in up front. The testing will be less about whether the functionality works and more about making sure the business users get what they need.

The benefits from BDD come n the developers’ task is to get them to pass. This means that when the tests do pass, the acceptance criteria will have been met.

This will involve a time investment up front and will require the developers to resist the urge to start coding. But all this is well worth the reward of not having to rework the code because the requirements weren’t properly understood up front.

Conclusion

So, there are a number of benefits to using three main categoriese BDD approach. First, it provides more guidance on how the discussion between business users, developers and domain experts should be carried out. For example, BDD explains how the acceptance criteria should be collaboratively defined from the conversation among the three partiesonducted to collaboratively define the acceptance criteria. Second, the language used to specify the acceptance criteriase is based on the natural language. This means that nobody needs to learn a new language in order to specify the acceptance criteria. They’re written in English understood by everyone on the team. Third, wit’s possible to generate the automated tests to cover the functionality. That is, assuming you have the framework in place to cover this.

The main idea behind BDD is that with the ‘user stories’ in place it should be possible to build on these stories and create executable scenarios, which, as a result, will be based on real-world examples. In fact, it’s those real-world examples that often deliver the real benefit of aiding understanding between business user, developer and tester.

Those real world examples are defined in a language called Gherkin. This is the right tools, automated tests can be generated from the specifications written in the Given-When-Then format that you may already be aware of. This format is structured well enough that it can be understood by tools (e.g. test automation tools). Yet is clear enough that it can be read by end users and business users. With the focus being both on the start of the process (defining the specific.

BDD, properly implemented and maintained, increases team member participations) and the end of the process (running automated tests) we avoid the curse of throw-away documentation. Typically, specification documents are written up front and then discarded once the implementation is complete, quickly falling behind reduces development time, thereby contributing terms of how they represent what has actually been implemented. When the documentation is used as part of the final acceptance tests, it’s difficult not to have your attention drawn back to the documentation to make sure it’s up to date and keeps the acceptance tests passing. The whole BDD process encourages the team to keep everything in sync. o the smooth operating and profitability of the organisation.