BHive: Behaviour-Driven Development Meets B-Method
by
John Douglas Carter
A Thesis presented to The University of Guelph
In partial fulfilment of requirements for the degree of Doctor of Philosophy in Computer Science
Guelph, Ontario, Canada
© John Douglas Carter, March, 2017 ABSTRACT
BHIVE: BEHAVIOUR-DRIVEN DEVELOPMENT MEETS B-METHOD
John Douglas Carter Advisor: University of Guelph, 2017 Professor W. B. Gardner
Behaviour-Driven Development (BDD) is an agile, “outside-in” approach to software development built upon semi-formal mediums for specifying the behavior of a sys- tem as it would be observed externally. Through the representation of a system as a collection of user stories and scenarios using BDD's notation, practitioners automate acceptance tests using examples of desired behavior for the envisioned system. A for- mal model created in concert with BDD tests would provide valuable insight into test validity and enhance the visibility of the problem domain. This work called BHive builds upon the formal underpinnings of BDD scenarios by mapping their “Given,”
“When,” and “Then” statements to “Precondition,” “Command,” and “Postcondi- tion” constructs as introduced by Floyd-Hoare logic. We posit that this mapping allows for a B-Method representation to be created and that such a model is useful for exploring system behavior and exposing gaps in requirements and test plans. In combining BDD with B-Method through BHive, a bridge between the worlds of agile and formal is created, reaping benefits of both approaches. We also outline exten- sions to BDD tooling required for the described integration and present benefits of the BHive approach to integrating formalism within a BDD project. Acknowledgments
I would like to thank the following people for their support throughout this project:
• Dr. Bill Gardner for his supervision, instruction, support and patience throughout the research project. • My advisory committee for their 11th-hour contributions of time. • The support staff at the University of Guelph who were always helpful despite my living 500 km off campus. • My parents, Lynn and Don Carter, for their support. • My grandparents, the late John and Jean Murray, and Doug and Eileen Carter. • Dave Rooney, for many productive discussions on agile methods and book recommen- dations. • Marion Scott, for her support throughout the work, not the least being her enthusiasm for last-minute babysitting.
iii Dedication
This work is dedicated to my family: My wife, Katherine Anne Scott, who never doubted my completion and has kept the world turning for the past months of “shock work.”
My sons Huck, Willie, and Abe who are anxious to have their dad back and their tree house built.
John Douglas Carter
Richmond, Ontario, Canada March 2017
iv Table of Contents
Acknowledgments ...... iii
Dedication ...... iv Table of Contents ...... v
List of Tables ...... viii List of Figures ...... ix
Chapter 1 Introduction ...... 1
1.1 Agile? Formal? Why not both? ...... 5 1.2 Bridging agile and formal development ...... 8 1.3 Road map ...... 10 Chapter 2 Context of BHive ...... 11
2.1 BDD Development Process ...... 12 2.1.1 Concentric Development Processes ...... 13 2.1.2 Going Green with Test Doubles...... 15 2.2 Using B-Method to Discover Problems with Global Mutable State ...... 16 Chapter 3 Related Work ...... 18
3.1 Development Approaches Leading to BDD ...... 18 3.1.1 Extreme Programming ...... 18 3.1.2 Test-Driven Development (TDD) ...... 20 3.2 Behaviour-Driven Development (BDD) ...... 25 3.2.1 BDD Tooling ...... 26 3.2.2 Invoking Tooling ...... 28 3.2.3 Tooling Packages ...... 29 3.3 Approaches Related to BDD ...... 32 3.3.1 Specification By Example ...... 32 3.3.2 Domain-Driven Design (DDD) ...... 32 3.3.3 Example-Driven Modeling (EDM) ...... 34 3.3.4 Acceptance Test Driven Development (ATDD) ...... 35 3.4 Formalisms...... 35 3.4.1 B-Method ...... 35 3.4.2 Event-B...... 37 3.4.3 Hoare Logic ...... 37 3.5 Combining agile and formal methods ...... 38 3.5.1 Specification-Driven Development ...... 38 3.5.2 Formal Specification-Driven Development ...... 40
v Chapter 4 Approach ...... 42
4.1 The Search for a Third Way...... 42 4.2 Theoretical Underpinnings of BHive...... 46 4.2.1 Naming...... 48 4.2.2 Types and Constants...... 48 4.2.3 State ...... 50 4.2.4 Scenarios ...... 51 4.2.5 BHive Directives ...... 53 4.2.6 Guidelines and Practices for Applying BHive ...... 57 4.3 Joining BHive and Behave...... 58 4.3.1 Features ...... 60 4.3.2 Scenarios ...... 60 4.3.3 Steps ...... 62 4.3.4 Step Functions ...... 62 4.3.5 Implementing with Behave ...... 68 4.3.6 Integrating BHive...... 72 4.3.7 Synthesis of “Start_Machine” ...... 75 4.3.8 Test Case Generation ...... 76 4.3.9 Scenario Outlines ...... 79 4.3.10 Typing in BHive...... 79 4.4 Complete Example...... 80 Chapter 5 Case Study ...... 85
5.1 Introducing the IPTV Application ...... 87 5.1.1 Launching IPTV...... 88 5.1.2 Home Screen ...... 89 5.1.3 Lander Screen...... 89 5.1.4 Media Details Screen ...... 91 5.1.5 Player Screen ...... 91 5.2 IPTV Backend System...... 91 5.3 First Development Iteration with BHive ...... 94 5.3.1 Fetch Profiles Scenario...... 107 5.3.2 Populating Content Restrictions...... 110 5.3.3 Select Profiles Scenario ...... 111 5.3.4 Populate Lander Scenario...... 113 5.3.5 Generating Test Cases for the IPTV Data Layer ...... 117 5.3.6 Application of BHive Guidelines...... 119 5.3.7 Benefits...... 120 Chapter 6 Conclusions and Future Work ...... 121
6.1 Contributions ...... 122 6.2 Future Work...... 124
vi Bibliography ...... 127 Appendix A: Sample BHive Custom Types File ...... 132
A.1 Custom Types File ...... 132 A.2 Synthesized CustomTypes Machine ...... 133 Appendix B: Specification for IPTV Back End ...... 134
vii List of Tables
Table 5-1. HTTP GET Operations for the IPTV Backend...... 92
viii List of Figures
Figure 2-1. BDD development flow ...... 12 Figure 2-2. BDD concentric cycles of development [CAD+10] ...... 14 Figure 3-1. TDD’s “Circle of Life”...... 20 Figure 3-2. Relationship Between BDD Entities...... 26 Figure 3-3. Gherkin feature file example for a customer queueing device...... 27 Figure 3-4. Design by Contract schema sample [Mey92] ...... 39 Figure 4-1. File system layout of a BHive Project...... 49 Figure 4-2. Generalized Gherkin scenario...... 52 Figure 4-3. Generalized B machine operation ...... 52 Figure 4-4. Rewriting B machine...... 52 Figure 4-4. Diagram showing the relationship between Gherkin components...... 58 Figure 4-5. Block Diagram of Behave...... 59 Figure 4-6. Block Diagram of BHive...... 60 Figure 4-7. Feature file for a “Now Serving” ticketing machine...... 61 Figure 4-8. Mapping Feature file and scenarios to B machine and operations...... 64 Figure 4-9. Feature test results, taken from the PyCharm IDE...... 70 Figure 4-10. Feature tests results, showing all steps passing for the “Start_Machine” scenario.... 71 Figure 5-1. IPTV Components...... 86 Figure 5-2. Wireframe for IPTV login screen...... 89 Figure 5-3. Wireframe for IPTV home screen ...... 90 Figure 5-4. Wireframe for an IPTV lander screen ...... 90 Figure 5-5. Wireframe for the IPTV Player Screen ...... 92 Figure 5-6. Entity-relationship diagram of IPTV data entities...... 94 Figure 5-7. Obtain Billing Id scenario ...... 96 Figure 5-8. Empty steps created for the Obtain Billing Id Scenario...... 97 Figure 5-9. First version of the DataLayer class...... 97 Figure 5-10. Second version of the DataLayer class ...... 100 Figure 5-11. Basic network handler and mock network handler...... 101 Figure 5-12. Revised Given step for Obtain Billing Id scenario ...... 102 Figure 5-13. Synthesis of B machine after completing Obtain Billing Id Scenario...... 104 Figure 5-14. Completed When and Then steps for the Obtain Billing ID scenario...... 106 Figure 5-15. Fetch Profiles scenario ...... 107 Figure 5-16. Synthesized operation for Fetch Profiles ...... 107 Figure 5-17. Amended Fetch Profiles scenario...... 109 Figure 5-18. Re-synthesized Fetch Profiles operation...... 109 Figure 5-19. Select_Profile Scenario ...... 111 Figure 5-20. Steps for Select_Profile Scenario...... 112 Figure 5-21. Populate Lander scenario ...... 114 Figure 5-22. Steps for Populate Lander scenario...... 115 Figure 5-23. Synthesized IPTVDataLayer machine ...... 116 Figure 5-24. Generating Test Cases in ProB Tool...... 118
ix Chapter 1 Introduction
A little rudder far from the rocks is a lot better than a lot of rudder close to the rocks. 1
Consider these common classes of failure in the development of complex software systems:
• Failure to deliver. Examples include projects being late, over budget, or a develop-
ment team shipping a system that fails to meet the needs of the customer, including
cases where customers failed to articulate their needs, or the project did not correctly
respond to changing requirements of the system.
• Catastrophic failure. Examples include systems such as aerospace, automotive,
defense, financial and medical applications that have a cost of failure beyond mere nui-
sance. In such instances, the costs of system failure could be human injury, death or
extensive financial loss.
• Failure to maintain. Projects that are insufficiently documented in environments
where knowledge of the system has been “siloed” run the risk of moving into an un-
maintainable state. New development, whether fixes or enhancements, becomes
increasingly difficult as the software ages [Par94]. It may develop problems or even
1. This quote comes from Captain David Marquet from his book Turn the Ship Around!: How to Create Leadership at Every Level [Mar13]. This phrase became the leadership motto on the USS Santa Fe which implemented a new paradigm of management under his command.
1 cease functioning from not keeping up with changes to interfacing components (e.g.,
operating system, database management system).
This thesis begins with a discussion of two development methodologies, each of which was created to solve one of the above types of failure, beginning with failure to deliver and its potential remedy via agile 1 development processes.
Agile is a movement seeking to address issues of software teams delivering projects that are late, over budget or fail to provide significant customer value. The drive to remedy these common occurrences, often described previously as “the software crisis” [Dij72], has propelled agile methods from a shared set of values [BBvB+01] to the de facto way of working for modern software teams. As “agility” is very much on a spectrum and open to interpretation by teams themselves, even the most ardent followers of traditional methods such as Waterfall introduce agile concepts such as daily status meetings or task boards into their workplaces. In fact, the term “Waterfall” itself has largely become pejorative, associ- ated with lumbering, sequential, documentation-driven development processes that fear change rather than anticipating, accounting and reacting to it, as proponents of agile meth- ods claim to do. Agile concepts are familiar to customers of software development teams.
They recognize the dynamic and incomplete nature of system requirements and a potential remedy in a methodology that can course-correct early and with minimal disruption or re- work. Customers ask for agile, and developers do not seek to turn back the clock to tradi- tional sequential methods such as Waterfall.
1. Throughout this thesis, “Agile” is capitalized when used as the proper name of a method (compare “Waterfall”). It appears in lowercase in the adjectival context (“agile methods”), even if the noun is assumed (“Customers ask for agile”).
2 While Agile is designed to avoid delivery failure, it does not address the second class of failure. Unfortunately, Agile as it currently practised is not particularly compatible with speciality methodologies for producing systems that require the highest standards of assurance. Classically, systems that have the possibility of catastrophic failure are candi- dates for the application of formal methods and derive value from the assurances associated with these methods. Traditionally, formal approaches are applied in the context of docu- mentation-driven processes like Waterfall. The affinity of formal methods for highly sequential methods like Waterfall is due in part to the nature of formal-based development where a formal specification informs all subsequent development. Experts create a model of the system using a formal specification medium that will serve as a blueprint for the actual system implementation. As with non-formal sequential development methodologies, there is an escalating cost proportional to how late in the development process a require- ment changes. The spectre of this cost drives the need for quality requirements engineering and well-informed modelling.
This work proposes an approach to bridge Agile's iterative nature with the assur- ances obtainable through formal models. Of particular importance is a route that allows incremental delivery of features alongside modelling activities. This thesis describes a new approach that integrates B-Method, a well supported, state-based formalism, with a com- paratively new agile development process, Behaviour-Driven Development or BDD. With this hybrid approach, development teams leverage assurances of modelling and model checking, while avoiding the inherent delivery risks associated with “big design up front”
(BDUF) processes.
3 System development using formal methods offers assurances not found in systems that rely solely on empirical testing for the means of verification and quality assurance. The term “formal methods” collectively refers to methods of specification, modelling, and ver- ification that are grounded in an underlying mathematical rigour. If applied correctly to a development process, the resulting system can be proven to have (or not have) specific properties or behaviour. Productive application of formal methods typically requires con- tributions from methodology “gurus,” and formal process literacy amongst the project team. The perceived level of investment for formal methods often reserves their application to project domains requiring the highest standards of assurance—mission-critical applica- tions where system failure could result in widespread loss. Formal specification and verifi- cation activities strive to “steer the ship” away from the proverbial rocks of catastrophic system failure.
Reports of increased developer productivity and improved customer satisfaction have caused many software development organizations to shift from traditional “plan- based” software engineering methods to so-called lightweight “agile” methods. “Agile” is not a single process in itself, but rather a loose framework of guiding principles surrounding development and delivery. Methods that fall under the agile classification typically focus on regular, frequent delivery to the customer. Milestones are incremental; the system does not arrive as a single drop at the end of the project; instead, smaller parcels of functionality are delivered, prioritized according to their value to the customer. These parcels of func- tionality should be complete, correct and ready to ship if a customer deems the system com- plete. The customer is typically an active and equal participant in agile development, providing insight on requirements and their respective business value. Here, the customer
4 provides the “little rudder”: through active involvement they can identify requirement gaps and misaligned priorities that could set the ship off course.
As for the third class of failure, software systems that fail to be maintained, fail to provide value in the long term. In his introductory textbook Software Engineering , Som- merville [Som10] refers to four purposes for system documentation:
• It should act as a communication medium between developers.
• It should be a “system information repository” used by maintenance engineers.
• It should provide information to management for planning and scheduling purposes.
• It should inform users and administrators how to use the system.
Thus, any development approach ought to empower and encourage the development team to create and update appropriate documentation. While documentation is outside the scope of this present research, plans for the introduction of automated documentation generation are discussed in Section 6.2 on page 124. Documentation provides the “navigational chart” by which the development team and future maintainers continue to steer the ship.
1.1 Agile? Formal? Why not both?
There is a noticeable “impedance mismatch” between agile and formal methods that pre- vents widespread integration of both approaches. Using the Agile manifesto [BBvB+01] as a framework for contrasting agile vs. formal, the mismatch arises from a combination of the following considerations:
Individuals and interactions vs. processes and tools
• Formal methods appear to shift focus from people to tools.
5 • Compared with traditional plan-based approaches, agile methods are relatively new, so
teams using formal methods may feel less inclined to change their well-documented
existing processes.
• Desire for agile and formal method teams to work together may be lacking. Fervent
“agilistas” may begin with the forgone conclusion that the old way of creating software
is inherently broken, and hence the “old ways” that formal method shops have been
using are beyond repair.
• Formal specification mediums are typically only readable by a very small population of
the project team. This “siloing” of key project documentation may be viewed as a
departure from open collaboration by a cross-functional team.
Working software vs. comprehensive documentation
• To those who are unable to understand them, formal specifications may be viewed as
unnecessary documentation, even impediments to the prime goal of producing working
software, and their value may not be recognizable to the customer in the short term.
Customer collaboration vs. contract negotiation
• Development houses which are engineering systems requiring formal method-backed
assurance may be developing the system to meet the requirements of a previously
agreed-upon legal contracts, not necessarily the needs of the customer: “I have to
deliver what's in the contract, not what you (now claim you) need.”
• Existing development approaches may be tightly coupled to certification processes
required by prospective customers.
6 • Projects may be estimated according to the likelihood of future change requests. Such
arrangements diminish the emphasis on getting the requirements right initially if the
customer will pay for something that was unclear or requires later change.
• Formal specification mediums are not suitable for collaboration with non-technical
stakeholders. The specialized knowledge needed for formal development may reside
solely with a few “gurus” who may not be available for or open to frequent customer
collaboration.
Responding to change vs. following a plan
• modelling the entire system before development delays the delivery of early mile-
stones. “Front-loading” of specification and modelling can appear at odds with the
agile notion of evolutionary development—requirements changing alongside a shared
understanding between developer and customer.
• The approach of constructing a single monolithic model is problematic in that a
requirements change may invalidate the model, requiring a massive rework by the for-
mal methods gurus.
Despite these fundamental differences in the worlds of agile software development
and the application of formal methods, a formal model integrated with the agile approach
of “behavioural driven development,” introduced next, could serve to resolve some of these
roadblocks.
7 1.2 Bridging agile and formal development
BDD [Nor06] evolved from Test Driven Development (TDD) [Bec02] which itself origi- nated from eXtreme Programming (XP) [BA04]. Within TDD, the first code written is a failing test. When this test subsequently passes, the developers know they have correctly implemented a piece of functionality, and set about refactoring said functionality while maintaining a passing test. TDD places considerable importance on the quality of tests themselves, and determining the set of salient tests is a non-trivial task, as noted by North
[Nor06]:
I had a problem. While using and teaching agile practices like test-driven development (TDD) on projects in different environments, I kept coming across the same confusion and misunderstandings. Programmers wanted to know where to start, what to test and what not to test, how much to test in one go, what to call their tests, and how to understand why a test fails.
North elevated TDD to BDD. Whereas unit tests are the language of developers— they create and run such tests—they are of minimal use to the rest of the project stakehold- ers, for whom acceptance and integration tests are of greater importance as they represent steps toward the project reaching a deliverable state and customer acceptance. BDD takes test authoring out of the realm of developers, replacing it with executable acceptance tests captured in a syntax, called Gherkin, useful for all project stakeholders, in the spirit of
Evan's “Ubiquitous Language” [Eva03]. Within BDD, a system is specified as a collection of features , with each feature comprising one or more scenarios written in the form:
Given: (a precondition) When: (a triggering event) Then: (a postcondition)
8 BDD tooling provides the means to execute such a scenario by “driving” the system under test and interrogating its behaviour. The structured natural language of the Gherkin medium provides meaningful documentation for cross-functional agile teams and provides a heretofore untapped integration point for formal methods. The “Given-When-Then”
(GWT) syntax is related to the concept of a “Hoare Triple” as introduced in Floyd-Hoare logic [Hoa69]. Within Floyd-Hoare logic, Given-When-Then is represented as:
{P} C {Q} where P is known as the “precondition,” C the “command,” and finally, Q the “postcondi- tion.” This triple and its accompanying axioms provide a means to reason about program correctness from a system specified using the formalism. The Hoare triple is fundamental to state-based formalisms such as Z [WD96], B [Abr96], and Event-B [Abr10] and it is from this underpinning that we posit an integration with B-Method is possible [CG16].
This work, titled “BHive,” strives to bridge the expressive power of BDD with the assurance capabilities of a state-based formalism, B-Method. The successful application of such a bridge suggests that agile and formal methods can productively coexist in a mature development process and offer value to development teams. Through the combination of agile and formal methods in the realization of BHive, a development team is able to miti- gate delivery failure, is able to inject an enhanced level of correctness into the system and employ a deeper understanding of the system under development, thereby creating more robust and maintainable systems.
9 1.3 Road map
Chapter 2 provides the context of the proposed work and its areas for potential application.
The origin of BDD and prior work combining agile and formal methods are given in
Chapter 3. The BHive approach is described in Chapter 4 with worked examples, and an extensive case study appears in Chapter 5. Conclusions and future work are discussed last.
10 Chapter 2 Context of BHive
BHive is intended to introduce additional assurances of correctness into a traditional BDD- based development process. The goal is not to apply BDD into the formal development of a safety- or mission-critical application, but rather to inject a degree of formalism into a
BDD development process for the purposes of increasing correctness by “girding” the application with formalism. This approach is similar in spirit to Gardner’s concept of
“Selective Formalism” [Gar05], where the critical backbone of an application is specified and verified formally, but it controls a set of “user-coded functions” developed using a tra- ditional, non-formal programming language.
As BDD [Nor06] is an example-driven methodology, diligence is required to avoid problems associated with underspecification caused by an insufficient number of examples or weak specification of pre/postconditions within the BDD development process. By injecting formal modelling into the BDD process, it is envisioned that gaps in examples will become apparent through model checking, that a better understanding of the system is gained through animating 1 the model, and that the model informs the testing process. This chapter introduces the phases of BDD development and outlines the typical activities involved in each phase. The chapter closes with a discussion of problems associated with
1. Model animation is a tool supported activity where a developer may interactively explore possible system states and behaviour using an application that resembles a traditional software debugger.
11 Figure 2-1. BDD development flow
global mutable state, and how BHive seeks to aid developers in their handling of shared
state.
2.1 BDD Development Process
A typical BDD development process is depicted in Figure 2-1. Artifacts for each numbered stage are above the block diagram, and mechanisms of the development process are below.
The first three stages are as follows:
1. A customer engages a development team 1 with a nebulous set of high-level require-
ments for the product they envision. The set of requirements may exist in a variety of
1. A “cross-functional” team is composed of business analysts, developers, quality assurance analysts, designers and their supporting management structure.
12 formats and maturities ranging from “sketches on a napkin” to a full-fledged require-
ments document produced using a commercial requirements engineering system.
2. Through a requirements workshop —a collaborative endeavour between the develop-
ment team and the customer—all stakeholders work to develop a set of feature files for
the product. Each feature file contains scenarios 1 outlining various system flows.
These scenarios can be parameterized using a table of examples, provided by the cus-
tomer and vetted by the development team.
3. With the feature files created, the development team can execute this specification. No
implementation has yet taken place, but the scenarios and contained steps serve to pro-
vide a starting point for developers.
2.1.1 Concentric Development Processes
Though not strictly required, it is advisable to follow a so-called concentric development process [CAD+10] consisting of inner and outer development cycles, depicted in Figure 2-
2. The outer cycle is known as the “scenario cycle or “application cycle” with an inner “step cycle” or “object cycle.”
The Scenario Cycle. In the outer cycle, the developer creates a scenario and its step definitions. Lacking an implementation, the scenario and its steps will fail, represented by the “red” in the traditional “red-green-refactor” idiom [Bec02] central to TDD. To make a single step pass, the developer enters the inner loop, the step cycle, discussed next.
1. The term “scenario” is used with a wide variety of meanings in different software development contexts. For a comprehensive literature survey of the term, see Chap 3 of [Car07]. In the context of BDD, scenario refers to a 3-tuple of precondition, command and postcondition.
13 Figure 2-2. BDD concentric cycles of development [CAD+10]
The Step Cycle. When development shifts to the step cycle, the goal becomes get- ting the step to pass (“go green”) for a single example or execution path as simply and quickly as possible. Once a green point is reached in the step cycle, the outer scenario cycle is executed. If the outer cycle is still red, the developer will know which example is failing, and this informs the next iteration of the step cycle. Following each “green” point in either cycle is a refactor stage where the previous “whatever means necessary” is replaced with an actual implementation. Technical “debt” created to get that cycle to go green is repaid
(for example, removing hardcoded values), and the implementation is improved and gen- eralized. Changes to the code are evaluated by running the previously passing tests.
14 2.1.2 Going Green with Test Doubles
Getting a step to pass may involve using components that one does not intend to implement in this iteration. To decouple the system during development steps from other dependen- cies, the developer will employ “test doubles,” a general term for objects that can stand in for other objects and vary in terms of sophistication. These offer the ability to compile and execute code with the minimal investment necessary when one is unsure of their design.
Usage of test doubles is a common unit testing practice, allowing the unit to be tested in isolation without outside influence or dependence on other system components.
Three relevant classes of test doubles are:
• Spy: An object that replaces a real object while tracking the invocation of methods and
the modification of properties.
• Stub: A shallow implementation that allows compilation to pass without error, perhaps
returning a single hardcoded value.
• Mock: A stand-in object that allows for more dynamic behaviour than a stub. Numer-
ous approaches to mocking exist, with many offering domain-specific languages for
defining their behaviour.
The use of test doubles (#4 in Figure 2-1) may last only until the next “refactor” in
the current cycle (#5 and #6 in Figure 2-1), or possibly longer, but are removed before the
code is considered ready for production.
15 2.2 Using B-Method to Discover Problems with Global Mutable State
Software applications using shared mutable state (even in cases with no concurrency) are prone to errors that are less likely to be found in applications that make state passing explicit using functional programming techniques or approaches such as dependency injec- tion [Pra09]. Novice developers eventually discover that shared mutable state leads to unpredictable program behaviour, impedes effective unit tests and may contribute to their code being harder to read. Such developers learn the mantra “global variables are evil.”
Despite the best intentions, or from a lack of experience, global mutable state sneaks into applications, often in constructs more sophisticated than the maligned global variable. Design patterns such as the singleton [GHJV95] often give developers a false sense of security by merely sweeping the problem with global variables under the prover- bial rug. When the state of the application spreads across multiple components, each main- taining its own state and influenced by other components, the likelihood of subtle and hard- to-find errors increases. In systems with a high degree of concurrency, state sharing is par- ticularly problematic leading to situations such as deadlock, or race conditions including the “lost update problem” where variations in scheduling lead to variations in data stored.
What's more, exact traces that result in an unwanted state may be difficult to find or repro- duce empirically. Our previous research [CG06, GCGS09, GGC15] explored issues of con- currency, and providing a framework for sharing state between concurrent components using the event-based formalism Communicating Sequential Processes (CSP) [Hoa78].
In modelling the system and its state using a technique such as B-Method [Abr96], the project can specify unwanted behaviour and prove it cannot occur within the system.
16 The handling of shared state becomes explicit, and BHive introduces this capability into a
BDD development flow by bridging BDD with B-Method.
17 Chapter 3 Related Work
This chapter surveys the origins of Behaviour Driven Development (BDD) and related work, beginning in Section 3.1 with approaches that led to the development of BDD, fol- lowed by the introduction of BDD and common tooling packages in Section 3.2. Next, approaches similar to BDD are described in Section 3.3. The chapter concludes with an out- line of pertinent formalisms (Section 3.4) and work related to combining agile methods with formal methods (Section 3.5).
3.1 Development Approaches Leading to BDD
This section discusses numerous development techniques related to BDD, including the approaches that influenced the development of BDD and methods that might be seen as similar to or competing with BDD.
3.1.1 Extreme Programming
Extreme Programming (XP) [BA04] is a significant departure from traditional plan-driven approaches to software development. XP presents a radical shift of focus, moving emphasis to both unit testing and acceptance testing activities. In XP, before embarking on a pro- gramming task, a developer (or two developers pair-programming) develops a suite of “unit tests” sufficient to determine if the feature being developed is correct and complete. Once a programming task is complete in isolation, the developers will integrate their new feature
18 and run an existing set of “integration tests” on the updated system. As features are reported
complete, teams will run “acceptance tests” on the system under development, to verify that
the features reported as completed are in line with the system specification set out by the
customer.
In the original work [BA04], the fundamental principles of XP are listed as:
• Rapid feedback: The customer contributes a better understanding of their require-
ments through having a partial system available to them. Feedback from customers is
passed on to the developers as soon as possible for the purpose of small “course correc-
tions” instead of larger ones later.
• Assume simplicity: XP puts forth the notion of solving “today's problems” in the
immediate and “trusting your ability to add complexity in the future where you need
it.”
• Incremental change: Changes to the design are not introduced all once: “The plan
changes a little at a time.”
• Embracing change: XP assumes that change will happen, therefore “The best strategy
is the one that preserves the most options while actually solving your most pressing
problem.”
• Quality work: Emphasis on software craftsmanship. Use the tools and techniques you
need now to perform the task well. Development skills are ever evolving, and a good
developer “keeps their tools sharp.”
19 • Emergent Design: Begin development tasks as simply and with as few assumptions as
possible. This allows for discoveries or clarifications made during the development
process to shape future understanding and refinement of the system’s features.
XP’s robust set of guidelines regarding testing gave way to a related approach bor- rowing XP’s testing techniques but without adopting XP’s philosophy completely. That approach is discussed next.
3.1.2 Test-Driven Development (TDD)
In the origin story, Beck doesn't credit himself with “inventing” TDD, instead as having
“rediscovered” it [Bec02]. He cites an “ancient programming manual” containing a sugges- tion for development: “Type the output tape you'd like to see before creating an input tape and compare the two on successive runs.” Beck’s notion of writing tests before code proved popular. The technique, dubbed Test-Driven Development and published in 2002, intro- duces an iterative process of development with three steps shown in Figure 3-1, repeated until completion of a programming task or feature:
Figure 3-1. TDD’s “Circle of Life”
20 • Write a test: Create a test that will pass when your task is complete and correct. The
test could depend on an interface or collaborating component that has not yet been cre-
ated. To create the test, you may have to “invent the interface you wish you had.” This
approach allows for the discovery of additional requirements of the involved compo-
nents that will inform the design of the system or subsystem.
• Make it run: After creating a test, and perhaps inventing a set of additional objects and
interfaces to support the test, a developer must make the test pass, preferably as soon
and as cheaply as possible. When getting a test to run, Beck places emphasis first on
passing the test, then improving the solution [Bec02]:
If a clean simple solution is obvious, then type it in. If the clean, simple solution is obvious but will take you a minute, then make of note of it and get back to the main problem, which is getting the bar green 1 in seconds. ... Quick green excuses all sins. But only for a moment.
• Make it right: After your test is running (and passing), the developer can go about
“making it right.” The code that was created quickly to make the test pass is re-exam-
ined, subjected to additional scrutiny and its duplication removed. Faked or hardcoded
sections are replaced with genuine counterparts. Beck contends that there are two parts
to TDD, “clean code” and “code that works.” With TDD one starts with code that
works and then make it clean. He juxtaposes this with what he calls “architecture-
driven development” (not rigorously defined) where you “...solve ‘clean code’ first,
then scramble around trying to integrate into the design the things you learn as you
solve the ‘that works’ problem.”
1. Typical unit testing harnesses (such as JUnit) run a developer’s set of unit tests and report results with a highly visible colored bar—red for one or more failing tests and green for all tests passing.
21 Dedication to getting tests to pass is what drives development forward and furthers understanding of the system under development. Often a TDD developer creates and uses system components that do not yet exist or that are difficult to use in isolation. To avoid having to pause development to create or integrate such components, test doubles (men- tioned in Section 2.1.2) offer a stand-in for such system components. One approach to test doubles, “mocks” or “mocking,” is discussed in Section 3.1.2.2.
3.1.2.1 Benefits of TDD “Test first” development represents a substantial paradigm shift in the longstanding tradi- tions of how teams design, develop and test software. Parsons et al. undertook a study of the application of TDD by a software team [PLL11]. They found that a switch to TDD from a traditional design-develop-test approach was beneficial in a number of ways:
1. Schedule risk and the amount of “buffer” required in estimates are reduced.
2. Bug counts are lower, test time is less and maintenance effort is decreased.
3. TDD allows for “merciless refactoring” [PLL11]. Teams have increased confidence
that improvements to the code base did not introduce regressions, so the risk of requir-
ing additional activities to improve the health of the code base is greatly reduced.
4. Testing requires a developer to flesh out an interface for system components before
any system code is written. Proponents of TDD argue that in separating the develop-
ment of the interface from the implementation, the quality of both are improved.
The study also revealed a less tangible advantage to TDD: a fundamental shift in the way the project team thought about the system and its problem domain. Parsons et al. frame this discussion as a contrast with “post hoc” testing, where the development of the system becomes a conjecture. This conjecture is tested after development, and falsified by
22 failing tests. With TDD, tests form the initial development tasks, making the tests the con-
jecture rather than an implementation, as well as serving as the specification for the system
being developed. Parsons claims [PLL11]:
This has the effect of redirecting the emphasis from negative falsification to positive confirmation. In other words, post hoc testing emphasizes the negative desire to break the pro- posed solution, whereas TDD emphasizes the positive desire to implement a solution that passes the test.
But when testing is applied after development:
The programmers are required to consider the existing inter- face (implemented code) of the class and decide how these are to be tested. Preconditions and postconditions, the con- tract under which the unit under test operates, are implied rather than explicit.
Thus, TDD starts the development process with the project team specifying the pre- and postconditions of a unit to be tested in a unit test. Next we examine approaches to mocking, mentioned previously.
3.1.2.2 Mocking and TDD A challenge to effective TDD, particularly in the early stages of development, is testing code that depends on other components that do not yet exist and testing code in isolation.
For instance, in developing a portion of an online store a unit test may need to collaborate with a hypothetical login system to allow a customer to place items in a shopping cart. It is possible that the login system does not yet exist, or the developer wants to isolate the shop- ping cart unit tests from the highly-stateful login system.
For the shopping cart unit tests, the developer requires only the ability to know if the user is authenticated or not. In the first instance, if the login system is incomplete, the
23 shopping cart developer can introduce a mock with an application programming interface
(API) of their choosing for the login system with only the operations and properties
required to check for authentication. This is described by Freeman et al. [FMPW04]: “Lack
of supporting code can be remedied in many cases through mocking and stubbing.”
Depending on how this activity is carried out, it may provide valuable insight into the inter-
faces of objects not created. The developer of the system under test (SUT) can “code to the
interface they want,” not the one already provided—this allows the design to be discovered
and emerge rather than being anticipated. Freeman describes this as “Need-Driven Devel-
opment.”
In the second instance, the unit test developer wishes to remove unnecessary com- plexity and state from the unit test. Rather than the unit test having to “pull in” the login system into its unit test and set up its state (perhaps by logging into a test account), the unit test developer can create a stand-in object with the interface of the login system that behaves in the way the unit test developer requires of it. Through using mocks in this way, the developer achieves far better isolation in his or her unit tests. Freeman et al. remark,
“The means to put the system into the state necessary may require significant additional interfaces, solely for the purpose of testing.” Moreover, these interfaces may provide access to the internal state of objects that they should not. Most developers would balk at including an additional interface purely for setting up test fixtures. The authors’ suggestion to remedy this problem is that mocks should exist as “roles” not objects [FMPW04].
Mock data and the unit tests are often derived from customer-provided examples of system usage, for instance, “If a customer orders items a, b, c their total bill should be amount d.” “Specification by Example” captures this practice as an approach, discussed in
24 Section 3.3.1. The practice of TDD inspired Dan North to create the “outside-in” approach
of BDD, discussed next.
3.2 Behaviour-Driven Development (BDD)
BDD began as a shift in naming and thinking about writing unit tests for a TDD approach.
North noted that by changing the name of a test there was a noticeable change in how devel- opers thought about and created their TDD test cases and the systems being designed
[Nor06]:
I came across the convention of starting test method names with the word “should.” This sentence template—The class should do something—means you can only define a test for the current class. This keeps you focused. If you find your- self writing a test whose name doesn’t fit this template, it suggests the behaviour may belong elsewhere. For instance, I was writing a class that validates input from a screen. Most of the fields are regular client details—forename, surname, etc.—but then there is a field for date of birth and one for age. I started writing a ClientDetailsValidatorTest with methods like testShouldFailForMissingSurname and testShouldFailForMissingTitle.
BDD encompasses the practices of TDD and Acceptance Test Driven Development
(ATDD) described in Section 3.3.4. BDD also incorporates key concepts from example driven approaches (Section 3.3.3) and Domain Driven Design (DDD), described in
Section 3.3.2. ATDD and BDD are so similar they are often used as synonyms, however, there is a very important distinction. With ATDD, acceptance tests are written in concert with all project stakeholders but the syntax resides squarely in the technical domain of the problem; the union of all tests forms a de facto functional specification. With BDD, in con- trast, things are meant to be “outside in”; the feature files are first order artifacts and they test from the perspective of a user of the system with no knowledge of internals.
25 BDD requires more self-awareness on the part of technical participants “having the
right conversation with the right people, using the right language at the right time.” This is
half of what's necessary; the other half is the technical means to execute a specification.
Those means are discussed next.
3.2.1 BDD Tooling
Figure 3-2 shows the relationships between the various entities in a BDD tooling package.
Figure 3-2. Relationship Between BDD Entities
At the highest level, a system developed using a BDD-based approach decomposes the system under development into a set of features, described next. An example feature file is shown in Figure 3-3 on page 27.
3.2.1.1 Features A feature, stored in a “feature file,” is textual user story that names a feature and provides a high level description of the background and motivation for the feature. The syntax for the feature file is a domain specific language (DSL) named Gherkin [Cuc16]. Each feature file contains one or more scenarios, explained below.
3.2.1.2 Scenarios and Scenario Outlines A scenario captures a behavioural aspect of a feature—a single path of execution, its assumptions, what it does and what the result should be. A scenario outline is a generalized
26 Feature: Ticketing As a business owner, I want to effectively manage customers waiting to be served in a "first come, first served" manner.
Scenario: Start_Machine Given Machine is off When Machine is started Then Machine is on And Display shows 0 And Ticket is 1
Scenario: Stop_Machine Given Machine is on When Machine is stopped Then Machine is off And Display shows 0
Scenario: Customer_Takes_Ticket Given Machine is on When Customer takes
Scenario: Serve Given Machine is on And The difference between
Scenario: Reset_Machine Given Machine is on When Machine is reset Then Display shows 0 And Ticket is 1 Figure 3-3. Gherkin feature file example for a customer queueing device scenario where certain values are substituted in from a table of examples. A scenario con- tains three or more steps, discussed next.
3.2.1.3 Scenario Steps A scenario is comprised of three types of steps, Given, When, Then :
• Given: The Given step describes the required system state for the scenario to execute.
For instance, in the “Serve” scenario in Figure 3-3, the Given step states that in order
for a customer to be served, the machine must be on and a ticket must be outstanding
(the current value displayed must be less than ticket). The Python function associated
with the step will get a supplied test data value of ticket as a parameter (shown as
27
step.
• When: The When step describes what change in the system state the scenario per-
forms. In the “Reset Machine” scenario, the When step describes resetting the queue
counter as the state change.
• Then: The Then step captures expectations of the system state after the When step has
occurred. For example, in the “Customer Takes Ticket” scenario, after a customer has
taken a ticket (the When clause), the specification states that we expect the next ticket
to be greater than the one just dispensed.
Steps may also begin with “And” causing the step to inherit the type (Given, When
Then) from the previous step.
3.2.1.4 Scenario Step Implementation In order to execute a scenario, the BDD tooling package provides a mechanism to create associations between scenario steps and functions coded in the implementation language.
A common practice is the use of Python function “decorators” (with the “@” symbol) that match a function (here, step_impl ) a scenario step text, for example, in Behave (see
Section 3.2.3.3):
@when('Machine is started') def step_impl(context): # step code...
3.2.2 Invoking Tooling
When the tooling package is invoked for a set of feature files, the following happens:
1. All feature files are parsed.
28 2. For each feature file, all scenarios are parsed into Given, When and Then statements
(typically referred to as “steps”).
3. Each scenario is executed. When executed, the tool creates a context for that scenario.
The context is shared between the Given, When and Then statements as they execute.
4. When each step executes, the framework finds an associated code block (typically a
function).
• For the Given, the step sets up the scenario by setting the context according to the
preconditions.
• For the When, the step makes the state change specified by the scenario.
• For the Then, the step checks a number of assertions expected to apply after the sys-
tem has performed the When step.
5. An assertion violated in a scenario step causes that step, its containing scenario and the
scenario’s feature to fail. At the end of execution, all failing steps/scenarios/features
are reported.
The flow described above is common to a number of tooling packages, surveyed next.
3.2.3 Tooling Packages
3.2.3.1 Cucumber Cucumber [Cuc17] is a BDD tooling written in Ruby, but supports additional languages including C++ (Cucumber-cpp [Amb13]) and PHP (Behat [Kud16]). The Gherkin [Cuc16]
DSL originates from the Cucumber project.
29 3.2.3.2 Pyramid [TRdS+10] introduced Pyramid (not to be confused with the web framework). To support
Pyramid a number of additional tools were introduced: PyCukes, a Cucumber-style DSL
with support for Python; Ludíbrio, a test double framework; and Should-DSL, an assertion
library. Pyramid was built to provide full thread BDD tooling for Python, which was not
available with the existing Pyccuracy [HFFC13] framework.
3.2.3.3 Behave Behave 1 [RJE14, Eng16] also provides BDD tooling for Python. It supports an extension
of Gherkin with small modifications to allow lowercase step keywords. Another minor dif-
ference between Pyramid and Behave is the introduction of a variable context to every step
implementation. This eliminates the global variables used by Pyramid.
3.2.3.4 RSpec RSpec [MRP+17], invented by Steven Baker in 2005, is better understood by developers
than by business stakeholders. Whereas Cucumber’s Gherkin is meant to closely resemble
a natural language, RSpec embeds the testing/assertion constructs into the specification
itself.
Gherkin and RSpec are not necessarily in an “either-or” relationship. Chelimsky et
al. [CAD+10] suggest that they be used in concert: Cucumber to describe the behaviour of
the application, and RSpec to describe the behaviour of the objects within. This amalgam
provides the customer-facing benefits of BDD with the discipline of development using
TDD. This relationship is best illustrated by the “concentric cycles” shown in Figure 2-2
1. “Behave” typically appears in lowercase in python literature and documentation. It is capitalized in this thesis to distinguish it from the ordinary verb.
30 on page 14. The outer loop application cycle follows a red-green-refactor flow for applica- tion behaviour. The contained object cycle is akin to classical TDD where we write tests before creating the objects themselves. In detail (with numbers matching Figure 2-2):
1. A cucumber acceptance test (a scenario) is created.
2. All steps for that scenario will fail (application cycle “Red”).
3. To move that step in application cycle to green we jump into the object cycle and write
a failing example (Object Cycle “Red”).
4. Enough code is written to make that example pass (object cycle “Green”). The code
should be the simplest possible—including stubbing using literal values, or using a
mocking framework.
5. We continue the inner process of writing a failing test, followed by the code to make it
pass. As the object cycle develops, we will need to clean up our code and extract com-
monality (object cycle “Refactor”).
6. We break back out to the application cycle when the step passes (application cycle
“Green”).
7. Our attention turns to the next step in scenario, for which we repeat 1-7 and refactor as
necessary (application cycle “Refactor”).
An important distinction between the surveyed methods and RSpec is the increased rigour of the inner loop. RSpec provides a very thorough and exacting mocking and asser- tion system for this purpose. In the next section, methods similar or related to BDD are pre- sented.
31 3.3 Approaches Related to BDD
3.3.1 Specification By Example
The term “specification by example” coined by Martin Fowler [Fow04] refers to the use of examples provided by the client in lieu of a specification created through analysis of the client's business procedures. According to Adzic [Adz11] there is marked improvement in the “signal to noise ratio” of development process communication. Client-furnished exam- ples provide a “single source of truth.” A sufficient set of examples can be used in place of a generalized system specification and serves as acceptance tests. Specification by Exam- ple is not a full-fledged technique in itself, but rather one (similar to Domain-Driven
Design’s notion of “Ubiquitous Language” discussed next) that strengthens other approaches presented below.
3.3.2 Domain-Driven Design (DDD)
Domain-Driven Design (DDD) is a set of model-based practices for system development introduced by Eric Evans [Eva03]. He asserts that for software projects there are typically two spheres of complexity, technological complexity and complexity in the problem
domain, and problems typically tend toward one of these types. Technological complexity
refers to problems whose crux is solving a technological limitation, such as a creating a
hardware/software system that can handle millions of requests per second. The second type
of complexity is one which is rooted in understanding the problem itself, and this latter
complexity is what DDD strives to address.
At the core of DDD is an evolving model built to facilitate understanding of the
project's domain and the logic that governs it. Evans defines “domain” as a sphere of
32 knowledge, influence or activity. The subject area to which the user applies a program is
the domain of the software. A “model” is a system of abstractions that describes selected
aspects of a domain that can be used to solve problems related to that domain. A model is
not designed to be as realistic as possible; rather, it is meant to loosely represent reality so
it becomes tractable for all stakeholders. It provides a structured representation of an over-
whelming amount of information and complexity outside the mind of a domain expert.
According to Evans, DDD serves to take this information from the sole guardianship of the
domain expert, “rigorously organize” it, and carefully apply a “selective abstraction of that
knowledge.”
Evans asserts that the problems associated with ambiguity in software can be helped by the use of a common, rigorous language between developers and stakeholders. The lan- guage itself should be based on the same “domain model” used in the software and hence use the same terminology and constructs. Furthermore, this language should be used “per- vasively” and should evolve such that the language’s usage is natural and flowing. This model becomes a “ubiquitous language” for all project stakeholders that connects all devel- opment activities.
The model allows software to enter the problem domain. When a team jumps right to implementation details, they lose the chance to create artifacts that cultivate a shared understanding and serve as a basis for the complexity of the problem domain. The collab- orative manner in which the model is constructed becomes the source of value in the model.
Thus, in his preface Evans describes two characteristics of teams capable of leveraging
DDD for maximum benefit:
1. Their development is iterative.
33 2. Developers and domain experts have a close relationship.
During the collaborative sessions between domain experts and other project stake-
holders in which a model will be built, Evans suggests using concrete scenarios as a means
for elicitation of requirements. In addition, other “holdout” scenarios can be used to test if
the model is sufficient. The discussion of related work continues with additional
approaches that consume customer examples.
3.3.3 Example-Driven Modeling (EDM)
Example-Driven Modeling (EDM) [ABC+13] uses concrete, real examples to inform and validate a model constructed in concert with subject matter experts. The authors summarize the definition of a model as “Model = Abstractions + Examples.” EDM applies a process similar to DDD with an emphasis on validating the model in progress using examples. This is an iterative process [ABC+13]:
Going back and forth between examples and abstractions is unavoidable. It is virtually impossible to construct and understand a non-trivial model without validating it on examples. Also, understanding a large set of examples with- out abstractions is equally hard. The process of going back and forth is incremental: abstractions and examples evolve together.
Of particular interest is their assertion that significant model refinement occurs from the use of “near miss” examples rather than examples in stark contrast to each other.
In using a set of examples with subtle differences, the development team is able to gain greater insight into the intended behaviour of the system. Their hypotheses were validated using a quantitative study with UML models and store rewards schemes [ZAC14], and the authors applied their approach to modelling using the Clafer language [ABC+13].
34 3.3.4 Acceptance Test Driven Development (ATDD)
Acceptance Test Driven Development (ATDD) [Gär12], also referred to as Story Test
Driven Development (STDD) [PM10], captures system requirements as a set of acceptance tests authored in concert with the project customer. It differs from BDD in that it is more
“white box” and “developer facing.” Acceptance tests are often created using the Given-
When-Then scheme associated with BDD, but are limited purely to a functional specifica- tion of the system.
The set of acceptance tests created form an “executable specification.” The Given-
When-Then construct used by ATDD and BDD is semantically equivalent to the “Hoare
Triple” [Hoa69], discussed as part of the following section of formalisms related to this work.
3.4 Formalisms
3.4.1 B-Method
B-Method, or “classical B,” descends from another specification method Z, both created by
J.R. Abrial [Abr96]. It is a state-based formalism that decomposes a software system into a number of “B machines” using the Abstract Machine Notation (AMN). In B, a machine contains a set of variables (its state), invariants on those variables, an initialization proce- dure that must establish all invariants for the machine’s state, and a set of operations that mutate the machine’s state. Operations are expressed in terms of pre- and postconditions.
All constructs in B are specified using a combination of set theory and first-order logic.
For example, the machine described using Gherkin in Figure 3-3 on page 27 was recreated using B’s AMN:
35 MACHINE TakeANumber
VARIABLES running, ticket, display
INVARIANT running : BOOL & ticket : INT & display : INT & display < ticket
INITIALISATION running := FALSE || ticket := 1 || display := 0
OPERATIONS
Customer_Takes_Ticket = PRE running = TRUE THEN ticket := ticket + 1 END;
Reset_Machine = PRE running = TRUE THEN display := 0 || ticket := 1 END;
Start_Machine = PRE running = FALSE THEN running := TRUE || display := 0 || ticket := 1 END;
Serve = PRE running = TRUE & ticket - display > 1 THEN display := display + 1 END;
Stop_Machine = PRE running = TRUE THEN running := FALSE || display := 0 END END
With B, a software system is produced through a series of “refinements.” A speci- fier begins in the abstract, and incrementally refines their specification away from using abstract data representations to more concrete structures that map closely to atomic types in conventional programming languages. Successive refinements are proven to satisfy invariants of the machines from which they are descended through the means of “linking
36 invariants” that relate refined state and operations to their more abstract predecessors. Clas-
sic B is supported by a number of tools including Atelier B, B4Free, and ProB [LB08].
3.4.2 Event-B
Event-B [Abr10] is an evolution of classical B. Whereas classical B focused on the trans- lation to software, Event-B is focused solely on system specification but maintains a similar syntax, machine structure and the concept of refinement. Event-B is realized in an Eclipse- based tool, Rodin.
3.4.3 Hoare Logic
Hoare Logic [Hoa69], or Floyd-Hoare Logic, is a formal system devised for reasoning about computer software, influenced by a similar flow-charting system devised by Floyd.
At its core is a simple building block:
P {Q} R where Q is a “command”, or something to be performed by the software; P is the precon- dition required for Q to happen; and R is the state after Q has happened. More succinctly:
“If P is true before Q, then R will be true after Q.”
Hoare provides a small set of axioms that allow for reasoning upon systems speci- fied using this logic:
• The skip statement—the axiom for empty statements
• Assignment axiom for substitution
• Composition of rules
• Conditional rules
37 • Consequent rules that provide axioms to strengthen and weaken pre/postconditions
• The “While” rule, which provides the means for reasoning about loops
3.5 Combining agile and formal methods
This section surveys existing work toward combining agile processes with formal method based development approaches.
3.5.1 Specification-Driven Development
Presented by Ostroff et al. [OMP04] in 2004, Specification-Driven Development (SDD) bridges the formal world of Meyer’s “design by contract” (DbC) [Mey92] with a test- driven development approach. The authors acknowledge that the two methodologies are often regarded as “absolute extreme opposites with no combination possible or desirable,” but are able to show the gap between them is not as wide as initially thought, and the two different approaches to specification combine to give a richer, complementary expression of system requirements. They succinctly describe a specification as a “bridge” between the physical world and the system under design, connected by a description of the constraints shared by both. DbC and its use of assertions matches well with their explanation of the nature of specifications.
The authors argue that DbC, despite being an up-front design activity, is palatable to Agile purists who eschew a “big design up front” because the specification is validated code and not purely documentation. The inclusion of TDD makes the development process amenable to refactoring with the set of unit tests confirming that improvements to the code base did not introduce regressions. Shortcomings of TDD, such as the lack of expressing a big picture architecture, are strengthened by DbC’s capabilities, and TDD adds utility by
38 allowing the capture of valid system traces, something not easily expressed using DbC.
Relevant to BHive, DbC uses a schema (Figure 3-4) of preconditions, body and postcondi-
tions, which align respectively to P, C, and Q present in a Hoare Triple [Hoa69].
routine_name (argument declarations) is -- comments require precondition do routine body ensure postconditions end Figure 3-4. Design by Contract schema sample [Mey92]
SDD uses DbC for describing logical transformations performed by the system.
TDD is used for capturing and testing collaboration between system subcomponents at the
same level of abstraction as UML’s collaboration diagrams.
The authors present a flexible method for applying their approach, noting that a
team may decide to undertake writing of TDD tests or DbC contracts first. They recom-
mend beginning with TDD as it provides “closure” for what is being designed (develop-
ment is complete, or can pause, when a feature passes its tests) and TDD tests are more
appropriate for collaboration. Their approach is inherently synergistic, as noted in the con-
clusion:
TDD and DbC are complementary technologies and can be used synergistically, but also to supplement limitations: con- tracts make design decisions explicit that may only be implicit in tests; and tests can better capture requirements (such as the LIFO property on stacks) than contracts.
SDD shares a number of the goals of BHive, but the integration of formal and agile differs considerably. With SDD, both the TDD tests and DbC artifacts coexist and it is highly dependent on the project team to create a meaningful link between the two. With
39 BHive, the formal specification is derived from the agile method (BDD), creating a tighter
coupling, and formal activities are “driven” from the agile activities.
3.5.2 Formal Specification-Driven Development
Rutledge et al. [RDLT14] propose Formal Specification-Driven Development (FSDD) which builds upon TDD and BDD through the introduction of an additional artifact, “a formal design specification expressed in a behavioral specification language.” The authors identify the following shortcomings of BDD:
• As the BDD specification is comprised solely of scenarios, the creators and maintainers
are limited in what they can express, which they argue is contrary to Evan’s notion of a
“ubiquitous language” (UL) and prone to “knowledge transfer errors” as found without
using a UL.
• When considering the difference between BDD and TDD, the authors assert, “Without
the expansion into the analysis and test realms, BDD devolves into TDD with a specific
vocabulary.”
A study by Erdogmus [EMT05] showed no gains to code quality and statistically
insignificant productivity boost through the application of TDD. Its authors explain the
finding as being related to the scope of the project used in the study and the inexperience
of its participants. However, Rutledge et al. offer an additional explanation: TDD does not
address knowledge transfer issues which contribute to overall quality of the system. They
acknowledge that any process designed to supplant or augment TDD and BDD must pro-
vide tool support that allows frequent and automated execution of the created tests and
specifications. Their approach is comprised of the following steps and artifacts:
40 • A designer creates a state-based formal specification from a set of requirements, aided
by a “design specification analyzer.”
• From the formal specification artifact, the following artifacts are created:
1. A coder creates the software manually. He or she is able to begin with set of stub
functions generated automatically from the preceding formal specification.
2. The developer is aided by a set of unit tests automatically generated from the for-
mal specification using an FSDD specific unit testing framework.
3. Quality assurance analysts test the system using a manually-created test plan that
has been analyzed by an FSDD supported test case analyzer.
The work described is ongoing at the time of writing, but does provide a sound methodological base on which many formal methods could be integrated into a more tradi- tional plan-based development approach. It differs from BHive in that it is merely a frame- work for integrating TDD and BDD methods with a formal specification medium, and assertions derived from the formal specification are synthesized into resulting system code.
BHive offers an advantage in that the development team is able to select the appropriate level of formalism and synthesize the specification via BHive directives in a BDD imple- mentation.
In the next chapter, the BHive approach is presented, which seeks to integrate the agile world of BDD with the formalism of B-Method.
41 Chapter 4 Approach
This chapter presents the BHive approach, its principal components and a discussion of integrations required to situate BHive into a contemporary BDD approach. This work was prototyped using Python's “Behave” [RJE14] module and although discussion does refer- ence specific elements of Behave, the approach is generalizable to any BDD tooling. Inte- gration components unique to Behave are noted.
As has been stated above, BHive is another attempt to integrate formal methods within the world of software engineering as commonly practiced by industry. In order to situate BHive in that conceptual realm, Section 4.1 examines BHive’s philosophical ances- try, as it were, exposing the influences on its evolution. In this way, readers may better appreciate the goals that BHive is trying to achieve. Section 4.2 presents the theoretical approach for mapping BHive to B machines. This is all in preparation for a full-fledged case study in the next chapter. Finally, Section 4.3 details the specific application of the the- oretical approach to the Behave tooling, which includes a worked example.
4.1 The Search for a Third Way
Integrated approaches to software development such as those reviewed in Section 3.5 are basically trying to create a decent compromise: a methodology that allows conventional developers to enjoy at least some of the benefits of formal methods without their having to adopt full formal development, which they won’t do (for reasons of cost, schedule, and lack
42 of special expertise, among others). This “search for a Third Way” is driven by researchers
who appreciate the value of formal specifications, but recognize the practical barriers to full
formal development in the commercial software industry. Rather than letting either formal
or non-formal techniques drive the other, the hope is to meet in the middle, combining
something of both sides of the spectrum.
In terms of the author’s involvement at the University of Guelph in the Design
Automation Research Group since 2004, two strands of influence have been important and can even be characterized as trying to “dig the tunnel from both ends,” where the centre point, at least for now, is BHive. The first strand is Gardner’s CSP++ [Gar05] which com- bines the formalism Communicating Sequential Processes (CSP) [Hoa78] with conven- tional C++ coding. The CSP++ approach starts with a CSP specification that has formally verified properties, and proceeds from there to automated code generation. The synthesized software runs under control of an object-oriented application framework (OOAF) written in C++ that supplies CSP semantics at run time. The compromise in the approach is dubbed
“selective formalism,” and the sense is that developers choose which portions of the target system will receive formal treatment (and thus code generation), leaving the rest of the soft- ware to be coded conventionally. This technique applies to the domain of control-domi- nated systems, and the assumption is that system elements warranting formal treatment are exactly the system’s control backbone: the site of concurrent execution and interprocess communication and synchronization. That portion is fully specified using CSP; the remain- ing software modules, called user-coded functions (UCFs), are plugged into the synthe- sized backbone where they are invoked at run time by CSP channels. UCFs are not allowed to communicate “behind the back” of the CSP control backbone, but must use specified
43 channels for that purpose. Using this “specification first” technique, the generated code is
“correct by construction” and retains the specification’s formally verified properties. A positive feature of this approach is that CSP, as formalisms go, is fairly approachable to stakeholders, not being overly laden with mathematical notation. Drawbacks are that (a) a formal methods guru is still needed at the front end of system design to write the specifica- tion, write assertions, and operate the verification tools; and (b) conventional developers can subvert the verified properties if they violate the “no communication apart from chan- nels” rule, for example, by coding global variables. The most significant influence to come from CSP++ is the notion of selective formalism.
CSP++ was the “formal end” of the tunnel. The other end is represented by the
R2D2C project of NASA [CGRH07] in which the author participated. As with CSP++, the problem domain was control-dominated systems. Standing for Requirements to Design to
Code, the aim was to start, not with a formal specification, but with a requirements docu- ment that is understandable to stakeholders and not needing a formal methods guru to com- pose. To enable automated processing, a structured requirements medium was devised by the author [Car07, CG08] that could be converted, first, to CSP traces, and from there a CSP specification—the “design”—would be extracted. The latter would be equivalent to the requirements document, have formally verified properties, and be amenable to automated generation of correct-by-construction code. This project was extremely ambitious, and did not proceed past the prototype stage, although some of its techniques were patented. From the standpoint of selective formalism, there was probably not enough of that to apply the approach to arbitrary systems.
44 BHive inherits (spiritually) from both CSP++ and R2D2C: On the one hand, BHive
allows for a high degree of selective formalism since there is no code generation (except
for skeletons produced by Behave) and developers are responsible for expanding and refin-
ing the mocks to produce a full implementation. (In future work, it is envisioned that BHive
will be integrated into a full-fledged mocking approach that will automate a substantial por-
tion of developers’ mocking chores.) On the other hand, BHive features R2D2C’s approach
of “The tool will extract the formal specification”—in the form of B machines—“the devel-
oper does not have to write it.”
With the above background, we may critique the “injecting agility into formal” strategy (Section 3.5.2): This strategy suffers from the same drawback as CSP++, that of requiring a formal methods guru, which would tend to hamper adoption. BHive, on the other hand, dispenses with the formal expert, making it adoptable by typical agile develop- ment shops. It supports the concept of a ubiquitous language, with good visibility by all stakeholders. BHive’s generated system model, which uses the same language and schema of its BDD examples, will enhance the understanding of the entire system, since it can be animated and explored.
Unlike the CSP++ and R2D2C strategies, there is no plan to refine B machines into code. That would be full formal development which we are trying to avoid due to the requirement for B-Method gurus; it would throw up barriers to adoption. However, under the rubric of selective formalism, if a particular shop needs or wants to do that, there is noth- ing stopping them from selectively implementing certain critical or sensitive portions of the code via B machine refinement. In contrast, expanding the initial mocks is what agile devel- opers are normally accustomed to doing, and the continual testing of BDD will help keep
45 the formally verified properties from disintegrating. Similar to CSP++’s “no communica-
tion apart from the control backbone,” we also lay down some guidelines for coders to obey
in order to make effective use of BHive and produce a useful model.
From the above, it will be seen how BHive’s “Third Way” has evolved. The basic ingredients of the theoretical approach are described next.
4.2 Theoretical Underpinnings of BHive
This section outlines the theoretical mappings from BHive to B-Method, and describes the current state of BHive’s translation process.
A development team working under the BDD paradigm follows an iterative process of creating and updating feature files, and the scenarios contained within, before passing those feature files through a tooling package that supports the programming language for the project. The BDD tooling allows for the execution of scenarios, serves to drive testing and allows the team to keep a running tally of pass/fail status while writing implementation code to be invoked by the tooling from the scenarios. The purpose of BHive is to augment this customary development process by giving developers the ability to generate a B machine that is representative of key relationships and behaviours of the system.
The BHive process augments an existing BDD tooling. The project team authors scenarios normally, with BHive providing a few general guidelines on naming and struc- turing of features and scenarios. Developers select the critical interactions of the system and use BHive directives to embed formalism into their step function code. BHive directives are designed to resemble mocking tools commonly used by team practising BDD. These directives serve to define the state space of the feature under development and specify valid
46 transformations of said state. A BDD tooling package augmented with BHive uses these
directives to generate B Machine specifications for the system automatically. These speci-
fications can be verified, animated and explored by the development team.
The B Machines generated by BHive capture behaviour and data relationships that
the development team has deemed critical. Secondary and tertiary elements need not be
modelled formally, allowing for a compact model ready to be examined for unwanted
behaviour. The synthesis process employed by BHive is based on translating the Given-
When-Then (Hoare triples) pattern to representative operations in a B Machine. BHive
directives provide a natural way to embed information required for the synthesis of precon-
ditions and state changes associated with operations into step functions invoked by a tradi-
tional BDD tooling.
Current work has resorted to a “lower level” syntax for proof-of-concept purposes,
with more knowledge of B-Method required of the user. Moving beyond the present proto-
type, an additional layer of abstraction could propel adoption of the approach. Such a layer
could be modelled after a widely used matching library like Hamcrest [RR16] or Should-
DSL [Tav15], thus providing a familiarity to existing BDD-inclined developers currently
using Hamcrest, or similar, in their step implementations. The syntax of Hamcrest also
results in highly readable step code containing “expressions of intent” [RR16]. A closer
integration with Hamcrest or similar tool is planned for future work (see Chapter 6).
In the subsections below, the theoretical underpinnings of BHive are described, beginning with basics such as naming, typing and constants. This is followed by the intro- duction of scenarios, step functions and BHive directives used in step functions. Lastly, testing-related provisions are discussed.
47 4.2.1 Naming
A. Machine Naming Figure 4-1 shows the typical file layout of a BHive project. The name of the synthesized B machine comes from the name of the containing folder of its feature file(s). Here, two machines will be synthesized, “MachineA” and “MachineB”, using their respective con- tained feature files. If the folder name contains spaces or other characters not available in
B’s machine name syntax, they will be replaced with the underscore character. Note that in the case of “MachineA” in Figure 4-1, the machine will be synthesized from the combina- tion of the two feature files, “Feature File A1” and “Feature File A2”.
B. Operation Naming When a B machine is synthesized, the names of its operations originate from the name of
Gherkin scenario. In Figure 4-1, MachineA will contain three operations: Scenario_A1.1,
Scenario_A1.2 and Scenario_A2.1, with spaces having been replaced by underscores to ensure compliance with B-Method syntax.
4.2.2 Types and Constants
BHive provides the means for specifying constant values and enumerated sets for reference in step implementations. As depicted in Figure 4-1, the project may contain an optional
“definitions” folder containing a JSON [Bra14] file. An example of JSON input and syn- thesized output are provided in “Appendix A: Sample BHive Custom Types File” on page 132.
With the custom types file, the development team can specify any or all of the fol- lowing:
48 Figure 4-1. File system layout of a BHive Project
• Constant values, often used as sentinels (e.g., NO_BILLING_ID = 0)
• Sets of constant literals including numbers and strings (e.g., NAMES = {“Alice”,
“Bob”, “Carol”} or SMALL_PRIMES = {2, 3, 5, 7, 11})
• Sets of symbols containing literal elements (e.g., JOBS = {Baker, Butcher, Candlestick-
Maker})
During the invocation of BHive’s synthesis step, a CustomTypes.mch file will be generated. This machine will then be included in a read-only manner in all synthesized B machines. Inclusion is accomplished via B-Method’s “SEES” mechanism that includes the
“static aspect” of one machine within another. The static context of a B machine is the set of all definitions that cannot vary during the execution of a machine, often used to define
49 the state space of the machine’s types. The custom types file generates the CONSTANTS,
PROPERTIES and SETS section of all synthesized machines. At present, BHive does not
support B-Method’s higher level types such as records and trees. Future support for such
types is discussed in Section 6.2. The relationship between BHive and the “dynamic
aspect” of synthesized B machines is discussed next.
4.2.3 State
At its core is B-Method’s ability to describe the possible states of a machine and the rela- tionships of these states to one another. The set of all variables within a machine comprises its state.
In BHive a variable can be declared anywhere inside the implementation of a step.
The syntax for declaring a variable specifies the variable name, type and initial value: declare_variable(context, 'variable_name', 'VARIABLE_TYPE', 'INITIAL_VALUE')
The first parameter, context, is a key-value store used by Behave. It is passed to all Behave invocations (see Section 4.3.6) and is used by BHive for storage of its internal data struc- tures.
During synthesis of each machine, the following happens for all variables declared in any steps associated with the machine:
• The variable name is added to the VARIABLE section of the machine.
• The type associated with the variable (i.e., variable_name : VARIABLE_TYPE, repre-
senting set membership, meaning “variable_name is an element of the set
VARIABLE_TYPE”) is added to the INVARIANT of the machine. During model
50 checking, the B tooling determines if any operations can occur that violate the invari-
ant, in the case of variables by assigning them a value outside of their specified type.
• In the INITIALISATION section of the machine, the variable is assigned its initial
value as specified as the last parameter of declare_variable. When a Machine is model-
checked, B will ensure that the INITIALISATION section of the machine establishes
the INVARIANT.
Variables and the initialisation sections of the machine are considered part of the
“dynamic aspect” of a machine, whenever the invariant section is considered a part of the
“static aspect.” Continuing with the constructs that contribute to the dynamic aspect of syn-
thesized machines, the means to specify and reason about valid changes of state are dis-
cussed next.
4.2.4 Scenarios
As BHive is built upon a Gherkin [Cuc16] based approach to BDD, the system is specified as a collection of scenarios, each containing a set of steps. Each step is associated with a
Python function that can contain both BHive directives and BDD-supporting code. The sce- nario itself captures a description of a state change in the system and requirements of the system for this state change to occur. A Gherkin scenario may be generalized as depicted in Figure 4-2 where g 0...g n are predicates joined using logical conjunction (AND), w 0...w n are either substitutions (i.e., variable assignments), or statements that do not change the state of the system, and t 0...t n are predicates joined using logical conjunction (AND).
51 Scenario: Example Given: g0 ... And: gn
When: w0 ... And: wn
Then: t0 ... And: tn Figure 4-2. Generalized Gherkin scenario
Shifting to B-Method, an operation contained in a B machine is generalized as
having an a precondition P, and a body S. The precondition, composed of one or more pred-
icates, must hold if the machine is to perform the body S. This is shown in Figure 4-3:
OPERATION(x) = PRE P THEN S END; Figure 4-3. Generalized B machine operation
This machine is decomposed in Figure 4-4. The precondition P has been replaced
with the conjunction of the given steps, and the body S has been replaced with parallel
assignments of the when steps.
OPERATION(x) = PRE p0 && ... && pn THEN w0 || ... || wn END;
Figure 4-4. Rewriting B machine
Now the relation between Figure 4-2’s g 0...g n and Figure 4-4’s p 0...p n (the same for w0...w n and s 0...s n) becomes evident, showing that formally underpinning Gherkin is pos- sible, and by extension, BDD.
52 We observe that the scenario’s Then step (t 0...t n), akin to the postcondition in Hoare triples, is absent from the machine. In Gherkin, the Then step is used to interrogate the system after the When step is applied. The Then steps typically contain a number of asser- tions against the system under development that a developer must create to verify that the state change occurred and is (informally) correct. This is not necessary in B-Method since, by definition, the body of an operation (and its associated state change) has occurred if an operation is invoked, with invocation implying the precondition held to enable that opera- tion. As with Hoare triples, the body of an operation may also contain statements that do not modify the state of machine (the SKIP statement). The Then steps are used by BHive for linking testing activities performed on the system under development with constraint- based model checking, discussed in Section 4.3.8. To achieve the semantic linking from the
Given and When steps, two BHive directives are introduced, discussed next.
4.2.5 BHive Directives
Directives are coded in the associated Python step functions. After describing the first two in their basic forms, other syntactic variations are mentioned and the means of declaring parameters for operations via another directive is explained.
A. ensure_that The “ensure_that” directive is used to specify a predicate that BHive must ensure is true for the state change associated with the scenario to be valid. Every predicate specified in a
Given step becomes part of the precondition for the operation synthesized from the sce- nario. Note that a single step may include multiple ensure_that statements; the predicate for each will be added to the precondition of the machine. This is correct given the associative property of the logical conjunction (AND) operation. If a given predicate g i (Figure 4-2) 53 can be decomposed into (gi' AND gi'') then a machine predicate of g 0 ... AND gi AND ...
gn is equivalent to g0 ... AND gi' AND gi'' AND ... g n.
B. assign The “assign” directive is used to specify the state change occurring. It accepts a 3-tuple con- taining, in order: the variable into which to substitute a new value, the operation used for the substitution (assignment, union, intersection) and the new value. BHive applies these substitutions to an internal dictionary, and then synthesizes all substitutions contained in the dictionary as a series of parallel assignments.
In the present implementation of BHive, assign has been constrained to parallel assignment to avoid side effects in assignments. Consider the following statements: a := b b := a+1
With sequential assignment, if a has a value of 0, and b has a value of 5, executing the above two lines will result in a = 5, b = 6. With parallel assignment, a = 5, b = 1. The latter fits with the notion that the When step is happening at once, not as a series of smaller incre- mental state changes. In the current implementation of BHive, multiple invocations of the assign directive for the same variable will result in an error condition and termination of the synthesis process. This is to avoid a developer’s assigning the variable in multiple When steps.
C. Syntax Considerations for ensure_that and assign Both ensure_that and assign offer syntactic variations designed to allow BHive to deter- mine necessary typing information during the processing of Gherkin steps. For instance, assign_filtered_domain(variable, list of predicates) and assign_filtered_range(variable, list
54 of predicates) accept a function and a series of predicates which are then applied to the
domain or range of a function, respectively, and all values for which the predicate holds are
assigned to the specified variable.
It is the intention to replace these relatively cumbersome directives with “higher level” directives designed to be familiar to frequent users of matching libraries such as
Hamcrest [RR16]. For example, the following Hamcrest statements were used during development of the IPTV Data Layer to ensure that after the profile fetch operation has completed, the set of profiles is not empty and returns the correct number of profiles (i.e., count): assert_that(has_length(context.data_layer.profiles), greater_than(0)) assert_that(has_length(context.data_layer.profiles), count)
This is syntactically similar to the current implementation of BHive’s ensure_that and test_that directives, but presently BHive lacks the ability to compose multiple matchers into a higher-level mocking. That would both add elegance and enable richer specification using fewer directives.
D. Operation Parameters Any type of step may “declare” that the corresponding operation accepts a parameter using the declare_parameter() directive which accepts a name for the parameter and a type to enforce on the parameter. Typing on the parameter is enforced through the addition of a predicate to the operation’s precondition.
Operation parameters need not only correspond to parameters passed from a sce- nario outline; instead, they are best employed to represent external influences on the oper- ation, such as the result of an HTTP request (used in the next chapter’s case study).
55 Operation parameters can be syntactically aligned with scenario outline parameter types.
BHive provides the means to specify that scenario outline parameters must match the def- inition used in the B machine. B’s built-in types (INT, NAT, NAT1, STRING, BOOL) are also registered with Behave’s scenario parser when BHive is initialized.
E. Testing Directives The last directive discussed is test_that() which is used in the scenario’s Then steps. This directive accepts a predicate, and associates that predicate with the operation to be synthe- sized. During synthesis, all test predicates for the operation will be outputted to allow for input into ProB’s test generation tool. Test case generation allows the development team to specify a predicate, or predicates, for a state they would like the system to reach and the operations the machine may use to reach that state. If that state can be reached, traces to the state satisfying the predicate will be generated. These traces can then be applied by means of either manual or automated testing to the system under development to determine if the predicate holds for the implementation.
At present, scenario outline parameters may not be included in a test_that statement, as ProB does not support the ability to specify operation parameters as part of the coverage criteria (i.e., execute the operation foo(x,y) using only the value of 5 for the parameter x).
At the time of writing, the possibility of such a feature has been confirmed with the devel- oper of the ProB tool and that BHive is not the first application for such an addition.
56 4.2.6 Guidelines and Practices for Applying BHive
Throughout the course of the development of this approach, iterations of BHive have been applied to a handful of case studies. As a result of these experiments, the following set of guidelines was created.
• BHive variables and parameters need only be created to capture data types and their
respective state space that needs to be understood and explored.
• As a starting point for deciding what to declare formally in a Gherkin step function, it is
recommended to consider the interactions with the system and its environment, specifi-
cally what information passes between the boundary of the environment and the system
informs the parameters chosen, and what the system is responsible for “remembering”
following an operation informs variables.
• With CSP++ (see Section 4.1) a rule was that a developer must not “go behind the
back” of the specification by passing data in code instead of via the formal model. With
BHive, a similar guideline applies: The developer is free to pass state outside of BHive
directives, but doing so is informal and creates a partition between the behaviour of the
system under development and its representational model.
Finally, BHive was designed to be an agile, iterative approach containing a tight
feedback loop. As with the TDD notion of writing just enough code to make a unit test pass,
the development team should employ just enough BHive to synthesize a specification, ani-
mate it, model check it and then iterate again. The goal is to avoid a big formal specification
up front, but not skip immediately to source code. The following section outlines integra-
tion between BHive and Behave [RJE14], a Python module for BDD.
57 4.3 Joining BHive and Behave
BHive's approach and integration are designed to augment an existing BDD methodology and its tooling. Modifications to syntax are avoided to ensure the approach is usable by existing teams applying BDD, and to minimize efforts to port to other BDD toolings.
A system specified behaviourally using Gherkin makes use of three components:
Features, Scenarios, and Steps. The relationship between these components is illustrated in
Figure 4-4:
Figure 4-4. Diagram showing the relationship between Gherkin components
A system is described behaviourally as a collection of features, authored in feature files. A feature will contain a number of scenarios or parameterized scenarios called out- lines. Each scenario contains Given, When, and Then steps that specify when the scenario can occur, the effect of the scenario and the outcome of the scenario.
Figure 4-5 shows a high-level view of the inputs (feature files, step definitions, and environment file) and the output (the system itself) of a typical BDD flow using Behave.
58 Figure 4-5. Block Diagram of Behave
The dark grey arrows show flow into and out of Behave, and the arrow from output to input
represents the iterative nature of the BDD process.
Figure 4-6 shows a view of BHive, at the same level of generalization as Figure 4-
5. BHive accepts the same feature files, step specifications and environment files that make use of BHive's integration functions and produces B machines represented in Abstract
Machine Notation (AMN). In addition to the generation of B specifications, test cases are generated using the ProB tool [LB08]. It should be noted that BHive is not meant to replace
Behave. The BHive development process is overlaid on top of the Behave development process. BHive may be used with an existing Behave project, and a BHive project can con- tinue as a Behave project when the development no longer has need of BHive’s additional assurance.
59 Figure 4-6. Block Diagram of BHive
BDD with Behave will be introduced by way of a small example of a ticketing
machine typically found at banks and service counters. BHive’s augmentations to the
Behave process follow the introduction of each Behave component where applicable. The
example system is designed to dispense ordered tickets, and display which customer is cur-
rently being served. The discussion begins with the highest level component, the feature.
4.3.1 Features
Features, captured in feature files, describe the desired behaviour of the system being spec- ified through a number of scenarios. Behave makes use of the domain specific language
Gherkin [Cuc16], providing a number of scenarios specifying the externally observable behaviour of the feature, shown in Figure 4-7. BHive does not augment the Gherkin syntax used by Behave. The contained scenarios are discussed next.
4.3.2 Scenarios
The “Ticketing” feature file specifies five scenarios, each of which is explained briefly:
60 Feature: Ticketing As a business owner, I want to effectively manage customers waiting to be served in a "first come, first served" manner.
Scenario: Start_Machine Given Machine is off When Machine is started Then Machine is on And Display shows 0 And Ticket is 1
Scenario: Stop_Machine Given Machine is on When Machine is stopped Then Machine is off And Display shows 0
Scenario: Customer_Takes_Ticket Given Machine is on When Customer takes
Scenario: Serve Given Machine is on And Display is less than
Scenario: Reset_Machine Given Machine is on When Machine is reset Then Display shows 0 And Ticket is 1 Figure 4-7. Feature file for a “Now Serving” ticketing machine
• Start_Machine - When the machine is off, it can be powered on, resulting in the ticket
and display being initialized to the first ticket (1).
• Stop_Machine - When the machine is on, it can be powered off, resulting in the display
showing “0”.
• Customer_Takes_Ticket - If the machine is on, a customer can take a ticket. The
ticket will be greater than or equal to the number currently on the display.
61 • Serve - If the machine is on and the number displayed is less than the current ticket (a
ticket is outstanding) a customer can be called to the wicket and the display will then be
less than or equal to the next ticket.
• Reset_Machine - If the machine is on, its ticket count and display can be reset back to
initial values.
As with features, scenarios in BHive are unchanged from the Gherkin scenarios used by Behave. The major points of integration, the step and its step function, are dis- cussed next.
4.3.3 Steps
Within Gherkin, a step is a single clause of one of three types: Given, When, Then. Gener- ally a scenario will contain at least one of each type:
Scenario: Reset_Machine Given Machine is on When Machine is reset Then Display shows 1 And Ticket is 1
The “And” designator is used to conjunctively specify multiple clauses of a given type. For instance, in the Reset_Machine scenario, the Then step states that after the machine is reset, the Display shows 1 and the Ticket is 1. When Behave executes a feature and its containing scenarios, the tooling looks for a single step function that will be called for each step in a scenario.
4.3.4 Step Functions
A step is realized in code as a step function by the development team. For example, in
Behave the Python code for the Reset_Machine Given step is:
62 @given('Machine is on') def step_impl(context): # establish the precondition.
In the case of Behave, step functions are linked to steps in the scenario using
Python’s decorator 1 mechanism, here, “@given”. When the tooling is invoked, Behave will execute the function with the matching step type (given, when, then) and matching step text
(here, ‘Machine is on’). The body of the function is implemented to satisfy the requirements of the Given, When and Then steps (discussed next).
If the Behave tooling is invoked and a matching step function is not found, Behave will generate a stub:
@given('Machine is on') def step_impl(context): raise NotImplementedError('STEP: Given Machine is on')
This stub will raise an exception when executed, alerting the developer of the need to implement it. Following the prescribed Behave development process, a development team would then implement each of these steps to exercise and interrogate the system under development.
From this feature file, BHive will synthesize a machine “Ticketing”. This machine will become the formal specification of the machine’s behaviour when viewed as a series of state changes. It should be noted that the text that follows the feature name (i.e., “As a
1. A decorator in Python is related, but not equivalent, to the “decorator” design pattern [GHJV95]. In Python, a decorator (via the @ syntax) allows the behaviour of a function or method to be altered without changing its source code. In the case of Behave, the decorators provide a linkage from a particular scenario step, identified by a keyword (like Given) and piece of text, to a corresponding step function @given(”text”). Strictly speaking, the linkage will work even if the target function is coded without the matching keyword, i.e., @step(“text”), but internal documentation is obviously strengthened by using Given, When, or Then.
63 business owner,... at any given time.”) is not used for any automatic processing and is
included only to provide context and motivation for the feature.
Each scenario contained in the feature specifies a portion of the desired behaviour
of the system. Referring back to Floyd-Hoare triples (Section 3.4.3), BHive conceptualizes
each scenario as being comprised of a precondition, a command and a postcondition that
must hold after the command is executed with the required precondition. BHive (and B-
Method which it uses as its formal underpinning) conceptualizes all system actions as being
changes in state. From a scenario contained in a feature file, BHive allows us to specify the
precondition, the command (change in state), and the postcondition using the Given, When,
Then clauses found in a Gherkin scenario. More precisely, BHive using Behave allows to
us to link the Given, When, Then clauses to Python code (i.e. the steps specification) to
specify the state changes and assertions on state.
BHive will use these descriptions of state change to synthesize operations in gener- ated machine. This mapping is depicted in Figure 4-8.
Figure 4-8. Mapping Feature file and scenarios to B machine and operations.
64 Consider, for example, the “Start Machine” scenario contained in the “Ticketing”
feature:
Scenario: Start_Machine Given Machine is off When Machine is started Then Machine is on And Display shows 0 And Ticket is 1
This describes the situation where the system is being switched from being off to on. The precondition is that the machine was off previously, and the postcondition is that the machine reports being on, the display should display “0” and the next ticket should be “1”.
In reading this scenario, one notices a lack of precision in the specification. The specifics of what is meant for the machine to be “off” or “on”, the process involved in “starting” the machine, and details of the display mechanism do not appear—the scenario is described completely externally and behaviourally—as a user of the system would interact with it.
BHive adds state specification using the steps specification file. With a traditional
BDD approach using Behave, a developer would initially provide empty steps 1 for each of the steps in the scenario, as shown below:
@given('Machine is off') def step_impl(context): pass 2
@when('Machine is started') def step_impl(context): pass
@then('Machine is on') def step_impl(context): pass
1. The empty step functions can be generated by Behave (and by extension BHive) using the “--snippets” argument, thus minimizing typographical errors when linking steps and step functions. 2. For readers unfamiliar with Python, “pass” is a “no op”—nothing happens during its execution.
65 @then('Display should show 0') def step_impl(context): pass
These steps would then be fleshed out as follows:
• The Given step should place the system into the state required. In the case of “Machine
is off”, it would likely reset the machine into the powered off state.
• The When step should invoke the functionality to accomplish the scenario “Start
Machine.” In the case of our example, this would likely include procedure(s) performed
when the machine is started such as resetting the ticket counter, and initializing the dis-
play.
• The Then step should make assertions or test the system to determine if the When step
was successful. In the case of “Start Machine”, the Then step might check that the
machine reports being powered on, and the current ticket is the correct initial value.
Note that in our example, we have two Then steps. Each one tests a separate success
criterion.
When developing a system using BDD, or BDD with TDD approach, it is generally
recommended to follow a three-step process:
1. Write a failing test. The developer creates a test such that when it passes, a feature (or
subfeature) can be considered complete. Writing the test first serves to force the devel-
oper to design an interface before getting invested in implementation details. The fail-
ing test should be able to compile, so the developer may need to “stub” some or all of
the implementation to get it to compile and then fail when invoked.
66 2. Make the test pass. The developer is free to do whatever is necessary to make the test
pass. This might include replacing empty stubs with more sophisticated mocks, or
simply hardcoding expected values. The emphasis is on getting the test to pass, not on
producing robust or generalized code, that is handled in the following step.
3. Refactor. The developer performs “clean up” arising from previous two steps. This
could include replacing “canned” data with real data, moving away from stubs/mocks,
or making the implementation more general. The refactor can be tested by rerunning
the test itself to verify the test still passes.
This process repeats with the system taking shape with each iteration, and the refac- tor step being used to improve the overall code quality once new functionality is intro- duced. The levels of abstraction may vary between BDD, BDD+TDD, or pure TDD but the overarching principles are the same.
The use of stubs and mocks in this type of development process is significant. Aside from forcing the development team to consider their interface before implementation, mocking injects a degree of specification detail into the testing domain. In specifying the types, range of values and expected outputs, constraints on the data in the system begin to take shape. In the previous “Start Machine” example, the mocked interface might tell us that the on/off state of the machine is a Boolean value and the value displayed is a natural number. Assertions in the Then step also serve to form a rough specification of acceptable and non-acceptable values. In the “Start Machine” example, assertions may be included to ensure that the value does not exceed a particular value. Additionally, BDD frameworks allow for “scenario outlines”, or parameterized scenarios. Working “outside-in” as illus- trated in Figure 3-4 on page 39, the steps toward implementation would be as follows:
67 1. Run a failing scenario.
2. Write a failing step definition.
3. Get the step definition to pass.
4. Refactor the step definition if necessary.
5. Once all steps are complete, refactor the scenario.
The next section provides a short example of working through this process for a
single scenario.
4.3.5 Implementing with Behave
Using the example scenario “Start Machine” we begin with the “Machine is off” step:
@given('Machine is off') def step_impl(context): context.domain_model.running = False
Here, we’ve made a design decision that we want to provide a Boolean property “running” in the model of the machine. This code will not compile, thus providing an initial failing step. We can then go about righting this by creating a class 1 and giving it a property “run- ning”: class TicketingModel: def __init__(self): self._is_running = False def __get_running_state(self): return self._is_running def __set_running_state(self, running_state): self._is_running = running_state running = property(__get_running_state, __set_running_state)
1. For the purposes of discussion, we will only implement the model (from the model-view-controller pat- tern) for the system under design.
68 In this case, implementing the step was fairly trivial and did not depend on additional
system components. In cases where getting the step to pass is more involved, stubbing or
mocking may be appropriate. Behave provides the means to pass data by way of the context
variable, and an additional environment file may be provided to set up the infrastructure to
inspect and exercise the system. Here, we are initializing the domain model for each sce-
nario, before it executes: def before_scenario(context, scenario): context.domain_model = NowServingModel.TicketingModel()
This will create an instance of TicketingModel and store it in the domain_model variable passed around with context. The use of before_scenario will result in a fresh instance being stored in domain_model before each scenario executes. Next, we move onto the When step “Machine is started”. This step should cause the machine to be started. We will represent this with a start() function:
@when('Machine is started') def step_impl(context): context.domain_model.start()
Next, we want to get this step to pass, so we add this to our class: def start(self): self.running = True
After implementing or stubbing the start() function for the domain model, we write a Then step to check the postcondition of starting the machine:
@then('Machine is on') def step_impl(context): assert(context.domain_model.running is True)
69 Here we want to ensure that after the machine is started, the running property is true.
That does not require any additional changes to our domain model. If we run our feature
now, we will see a report (Figure 4-9) which tells us that three of the five steps are passing.
Figure 4-9. Feature test results, taken from the PyCharm IDE
The last two still need to be implemented, so we will create those next. We will write a Then
step that asserts that the display should read “0” when the machine is started:
@then('Display shows 0') def step_impl(context): assert(context.domain_model.display == 0)
Once again, the system was developed with an eye to the interface desired from our domain_model—a integer property called “display”. As it is relatively simple, this can be implemented immediately: def __get_display(self): return self._display def __set_display(self, display): self._display = display display = property(__get_display, __set_display)
Now it is time to backtrack and revisit the start function 1. The feature states that if
the machine is started, display should be 0: def start(self):
1. Strictly adhering to BDD+TDD, the developer(s) would re-run any unit tests after making any changes to the model.
70 self.running = True self.display = 0
Finally, we do the same with the “Ticket is 1” step:
@then('Ticket is 1') def step_impl(context): assert(context.domain_model.ticket == 1)
Following by adding the implementation to the domain model: def __get_ticket(self): return self._ticket def __set_ticket(self, ticket): self._ticket = ticket ticket = property(__get_ticket, __set_ticket) and updating start() once more: def start(self): self.running = True self.display = 0 self.ticket = 1
Running the feature, the scenario and its steps have “gone green.”
Figure 4-10. Feature tests results, showing all steps passing for the “Start_Machine” scenario
At this point in the development process a developer will refactor any repeated code, replace any suitable stubs or mocks with implementations, and verify all changes by re-running all unit tests and completed features. Instead of immediately moving on to the next feature, the BHive augmentation to Behave allows further exploration of the domain model and offers the possibility of additional verification.
71 4.3.6 Integrating BHive
BHive can be invoked by an existing Behave code through the environment file. The envi- ronment file, discussed during the previous example of the Behave work flow, provides developer “hooks” for the start and end of each BDD scope: global, feature, scenario, and step. Behave also provides the ability to tag scenarios, and provides a before and after hook for each tag. Invoking BHive during Behave execution is done by calling each of BHive’s scope hooks in its Behave counterpart. For example, for the Ticketing machine the envi- ronment.py file is very straightforward: def before_all(context): bhive.integration.before_all(context) def after_all(context): bhive.integration.after_all(context) def before_feature(context, feature): bhive.integration.before_feature(context, feature) def after_feature(context, feature): bhive.integration.after_feature(context, feature) def before_scenario(context, scenario): bhive.integration.before_scenario(context, scenario) context.domain_model = NowServingModel.TicketingModel() def after_scenario(context, scenario): bhive.integration.after_scenario(context, scenario) def before_step(context, step): bhive.integration.before_step(context, step) def after_step(context, step): bhive.integration.after_step(context, step)
In the above, the only divergence from the boilerplate inclusion process is the initialization of the domain model before each scenario invocation as shown in the previous Behave example.
72 Returning to the Start_Machine scenario, formal semantics can be introduced
alongside the implementation created in Section 4.3.5 on page 68. We introduce two addi-
tional statements, shown in bold:
@given('Machine is off') def step_impl(context): declare_variable(context, 'running', 'BOOL', 'FALSE') context.state.ensure_that('running','=','FALSE') context.domain_model.running = False
The “declare_variable” function introduces a variable in the specification that will be syn- thesized. In this case the variable’s name is “running”, it is a Boolean value and should be initialized in the machine as false. Note that not every variable used in the implementation will require a BHive analogue. Only variables that affect state from outside the scenario need be included. Additionally, if formal verification is of little use for a particular part of the system it may be omitted entirely. This is akin to Gardner’s notion of “selective formal- ism” (Section 4.1) where one can specify the critical “backbone” of a system formally but avoid the extra work of specifying non-critical components or parts of the system where formal verification offers no advantage and/or is deemed impractical.
Returning to the example, this statement specifies the precondition that the Given step establishes, specifically that the newly introduced running variable must be false for the Start_Machine scenario to be invoked: context.state.ensure_that('running','=','FALSE')
This is achieved by calling the ensure_that function on the context’s state property. The state property is automatically added to the context of a scenario. BHive provides means to modify the state (its previously declared variables), introduce restrictions on the state (such
73 as the ensure_that function) and highlight conditions developers wish to use to generate test
cases.
The When step makes three modifications to the state for the Start_Machine sce- nario, shown in bold:
@when('Machine is started') def step_impl(context): declare_variable(context, 'display', 'INT', '0') declare_variable(context, 'ticket', 'INT', '0') context.state.assign([('running', ':=', 'TRUE')]) context.state.assign([('display', ':=', '0')]) context.state.assign([('ticket', ':=', '1')]) context.domain_model.start()
Here, the developer has specified that the effect of starting the machine is the running vari- able becomes true, and the newly introduced display and ticket variables become 0 and 1, respectively. The variables display and ticket are declared here for the sake of readability, but can be declared anywhere, and even multiple times. In the case of multiple declarations, any contradictory typing information (e.g., declaring a type as both INT and BOOL) will be flagged as an error and synthesis halted.
The final step type (Then) is explained next:
@then('Machine is on') def step_impl(context): context.state.test_that(context, 'running', '=', 'TRUE') assert(context.domain_model.running is True)
@then('Display shows 0') def step_impl(context): context.state.test_that(context, 'display', '=', '0') assert(context.domain_model.display == 0)
@then('Ticket is 1') def step_impl(context): context.state.test_that(context, 'ticket', '=', '1') assert(context.domain_model.ticket == 1)
74 These three Then steps contained in the start_machine scenario are shown. Each step spec-
ifies a condition that holds in the system’s state after the scenario has executed. This is
slightly differently from the non-bolded implementation code that checks the modified
values in the model. The test_that function will form the input to the test generation capa-
bilities provided by ProB. Test generation is discussed in a later section, with the synthesis
process outlined next.
4.3.7 Synthesis of “Start_Machine”
Invoking Behave with the BHive augmentation will result in a B machine synthesized for the TakeANumber Machine and the single Start_Machine scenario outlined above. This code will be written to a file in the project’s output directory (see Figure 4-1 on page 49) when Behave invokes the bhive.integration.after_all() hook:
MACHINE
TakeANumber
VARIABLES running, ticket, display
INVARIANT running : BOOL & ticket : INT & display : INT
INITIALISATION running := FALSE || ticket := 0 || display := 0
OPERATIONS
Start_Machine = PRE running = FALSE THEN running := TRUE || display := 0 || ticket := 1 END END
75 Using declare_variable in a BHive-enabled step results in:
• variables added to the machine, in this case: running, ticket, display
• the typing restrictions for the variables added to the machine’s invariant, here, Boolean,
integer and integer respectively
• the variables’ initial values added to the initialization section
In looking at the Start_Machine operation synthesized from the Start_Machine sce- nario, we see the precondition is given by the ensure_that() function and all changes made to the context.state property, form the state change for the synthesized operation, i.e.: context.state.assign([('running', ':=', 'TRUE')]) context.state.assign([('display', ':=', '0')]) context.state.assign([('ticket', ':=', '1')])
It should be noted that the context.state.test_that statements do not have an effect on the synthesized specification. The BHive constructs used in the Then steps are used for test- case generation, discussed next.
4.3.8 Test Case Generation
The Then steps in Behave are used to inspect and verify the system after a When step has occurred. Such checks are of no use in B because by definition the postcondition will hold after executing an operation as it is what was specified. Comparing:
@when('Machine is started') def step_impl(context): context.domain_model.start() to:
Start_Machine = PRE running = FALSE THEN running := TRUE || display := 0 || ticket := 1
76 END it is apparent that there is little interpretation possible with B’s Start_Machine when com- pared to calling the start() function of the domain model.
BHive leverages the Then step to specify parameters used for automatic test case generation. For a set of Then steps containing one or more test_that() statements, a predi- cate will be generated that can be fed into a test generation tool. For the early work, BHive makes use of ProB’s model-checking-based test generation. When the B specification is synthesized, the contents of test_that are synthesized into a predicate which is used identify target states in the test case generation tool. The test case generation tool will then create test cases that:
• put the machine into a state that satisfies the predicate
• cover all operations
This does not provide additional formal assurance, but instead generates test cases derived from known states in the model. These test cases are intended to be executed on the system under development, exposing any variance from the formal model. Furthermore, in having the specification part of manual testing, this moves developers away from a tradi- tional sequential development flow, and allows for test case generation to happen itera- tively as both the model and system evolve.
4.3.8.1 Verification using B-Method B-Method offers additional verification possibilities, not specific to BHive, but that can serve to inform a manual test plan. These include:
77 • Invariant Violation Checks. The B tooling provides the means to search the machine’s
state space for invariant violations and provide a trace to that unwanted state. Such a
situation could arise as a result of incorrect specification, and the system under devel-
opment can follow the same trace to ensure that the unwanted behaviour is not present.
• Deadlock Checking. ProB allows a developer to search the state space for deadlock
conditions. In the context of state-based formalisms, this is a state where no operation’s
preconditions are true, so no further state change is possible. As with invariant viola-
tions, traces provided by the deadlock checker can form the basis for manual checks for
deadlocks in the system under development.
• Vacuous Guards. ProB provides the means to check the preconditions of operations for
predicates that will always be satisfied. Such a discovery can prove useful in under-
standing inter-system dependencies, or simply discovering incorrect assumptions made
during development.
• Using “Goal” predicates. In B, a specifier can craft a predicate and ask the model
checker whether a state can be reached satisfying such predicate and, if so, to provide a
trace to that state. Such a task requires slightly more knowledge of B specification tech-
niques than invariant and deadlock checks, but allows the development team to “work
backwards” from certain types of bug reports. Using the TakeANumber example: Sup-
pose a customer reported that the “now serving” number was negative. A novice B user
could specify a predicate where the display variable is negative, have the model
checker find that state, and then attempt to reproduce it in the system under develop-
ment using generated traces. This is not pure formal verification, but rather a formal
model informing testing procedures.
78 4.3.9 Scenario Outlines
A powerful construct in Behave is the use of parameterized scenarios, or scenario outlines.
Consider this example taken from the Behave tutorial [Eng16]:
Scenario Outline: Blenders Given I put
Examples: Amphibians | thing | other thing | | Red Tree Frog | mush |
Examples: Consumer Electronics | thing | other thing | | iPhone | toxic waste | | Galaxy Nexus | toxic waste |
Behave allows us to pass tables of example data to our scenarios, and will execute perform- ing the substitution of the column into the corresponding variable in the scenario outline.
The ability to make examples provided by a project stakeholder into first class artifacts con- tained within the behavioural specification serves to guide BDD development by allowing the development team a convenient way to focus on making a single example pass. It builds the bridge of a “ubiquitous language” as Evans stated [Eva03] that North highlighted in his work introducing BDD. BHive integrates with scenario outlines by synthesizing set types for the table columns contained in a scenario outline linking the behavioural and synthe- sized formal specification.
4.3.10 Typing in BHive
Aside from scenario outlines, BHive integrates with Behave’s typing and parsing system.
For instance, the Then steps from our running example could be parameterized as follows:
@then('Display shows 0') def step_impl(context):
79 to this:
@then('Display shows {x: DisplayNumber}') def step_impl(context):
When the updated step is executed by Behave, values for x will be substituted from the examples table. The addition of the type “DisplayNumber” provides rudimentary typing in Behave. A developer can register a parse function for a given type: def parse_display_number(text): # just convert a string to an int: return int(text) register_type(DisplayNumber=parse_display_number)
When the Behave tooling encounters a scenario parameterized by the type “Dis- playNumber”, it will then invoke the parse_display_number function to attempt to convert it from a string, causing the test to fail if parsing cannot be performed. BHive integrates with this typing mechanism by wrapping calls to register_type() with register_bhive_type() and by allowing Behave types to be specified using B-Method primitives.
4.4 Complete Example
Closing this chapter is the complete example from above, showing all operations, the domain model created and the B specification synthesized from the Gherkin specification.
A. Complete Steps File from behave import given, when, then from bhive.integration import log_info, declare_variable
@given('Machine is off') def step_impl(context): # start bhive declare_variable(context, 'running', 'BOOL', 'FALSE') context.state.ensure_that('running','=','FALSE') # end bhive context.domain_model.running = False
80 @when('Machine is started') def step_impl(context): # start bhive declare_variable(context, 'display', 'INT', '0') declare_variable(context, 'ticket', 'INT', '0') context.state.assign([('running', ':=', 'TRUE')]) context.state.assign([('display', ':=', '0')]) context.state.assign([('ticket', ':=', '1')]) # end bhive context.domain_model.start()
@then('Machine is on') def step_impl(context): # start bhive context.state.test_that(context, 'running', '=', 'TRUE') # end bhive assert(context.domain_model.running is True)
@then('Display shows 0') def step_impl(context): # start bhive context.state.test_that(context, 'display', '=', '0') # end bhive assert(context.domain_model.display == 0)
@then('Ticket is 1') def step_impl(context): # start bhive context.state.test_that(context, 'ticket', '=', '1') # end bhive assert(context.domain_model.ticket == 1)
@given('Machine is on') def step_impl(context): # start bhive context.state.ensure_that('running', '=', 'TRUE') # end bhive context.domain_model.running = True
@when('Machine is stopped') def step_impl(context): # start bhive context.state.assign([('running', ':=', 'FALSE')]) context.state.assign([('display', ':=', '0')]) # end bhive context.domain_model.stop()
@then('Machine is off') def step_impl(context):
81 # start bhive context.state.test_that(context, 'running', '=', 'FALSE') # end bhive assert(context.domain_model.running is False)
@when('Customer takes
@then('Display should be less than or equal to the ticket') def step_impl(context): # start bhive context.state.test_that(context, 'display', '<=', 'ticket') context.state.test_that_always(context, 'display', '<=', 'ticket') # end bhive assert(context.domain_model.display <= context.domain_model.ticket)
@given('Display is less than
@when('A customer is called') def step_impl(context): # start bhive context.state.assign([('display', ':=', 'display + 1')]) # end bhive context.domain_model.serve_customer()
@when('Machine is reset') def step_impl(context): # start bhive context.state.assign([('display', ':=', '0')]) context.state.assign([('ticket', ':=', '1')]) # end bhive context.domain_model.reset()
B. Completed Domain Model class TicketingModel: def __init__(self): self._display = 0
82 self._is_running = False self._ticket = 0
# model properties def __get_display(self): return self._display def __set_display(self, display): self._display = display def __get_ticket(self): return self._ticket def __set_ticket(self, ticket): self._ticket = ticket def __get_running_state(self): return self._is_running def __set_running_state(self, running_state): self._is_running = running_state display = property(__get_display, __set_display) ticket = property(__get_ticket, __set_ticket) running = property(__get_running_state, __set_running_state)
# model operations def start(self): self.running = True self.reset() def stop(self): self.running = False self.display = 0 def reset(self): self.display = 0 self.ticket = 1 def take_ticket(self): if not self.running: return None current = self.ticket self.ticket = (self.ticket + 1) return current def serve_customer(self): if self.ticket - 1 == self.display: return None self.display += 1
83 return self.display
C. Complete Synthesized B Specification MACHINE NowServing(max_display)
CONSTRAINTS max_display : INT & max_display > 0 & max_display < 100
VARIABLES running, display, ticket
INVARIANT running : BOOL & display : NAT & ticket : NAT1 & display < max_display
INITIALISATION running := FALSE || display := 0 || ticket := 1
OPERATIONS
Start_Machine = PRE running = FALSE THEN running := TRUE || display := 0 END;
Stop_Machine = PRE running = TRUE THEN running := FALSE || display := 0 END;
Customer_Takes_Ticket = PRE running = TRUE THEN ticket := ticket + 1 END;
Reset_Machine = PRE running = TRUE THEN display := 1 || ticket := 0 END;
Serve = PRE running = TRUE & display < ticket THEN display := display + 1 END
END
The following chapter presents the application of BHive to the development of a data layer to be used in IPTV applications.
84 Chapter 5 Case Study
This chapter presents a case study showing the development of a system using the BHive approach. It focuses on the construction of a data layer for an internet television (IPTV) application—video content delivered using TCP/IP packets instead of traditional radio fre- quency or satellite methods. The IPTV market has seen tremendous growth in the past decade as the proliferation of high-speed internet connectivity has brought with it new pos- sibilities for content distribution from service providers. Broadly speaking, IPTV refers to the following services:
• Linear programming, content that airs on a specific channel at a provided time (e.g.,
live television)
• A variation of linear programming called “catch up TV” where a piece of programming
is available for replay for a duration of time after it has aired
• Subscription video-on-demand or SVOD, where a subscriber is free to watch all titles
provided by the subscription at any time (e.g., Netflix)
• Transactional video-on-demand or TVOD, where a subscriber purchases an entitlement
for a particular piece of programming, often with a window of expiry (e.g., pay-per-
view movies or sporting events)
85 IPTV is not limited to traditional cable “set-top boxes.” Consumers watch content
on “smart” televisions, mobile and tablet devices, and gaming consoles. Developers of
applications for these platforms typically integrate with one or more backend systems using
RESTful 1 interfaces in an efficient manner to provide a responsive and compelling user experience (UX). This case study focuses on building a software system to provide a single data model for IPTV UX developers. The data model will provide a single API for UX developers to access authentication, metadata, entitlement, purchasing and search/recom- mendation services.
Figure 5-1. IPTV Components
Figure 5-1 shows the data model in relation to the UX on the left and to backend components on the right. The multi-platform UX accesses a diverse set of backend services via the IPTV Data Layer. Note that the data layer loosely follows the facade design pattern
[GHJV95]. The data layer system must provide this functionality to the consuming appli- cation in a responsive manner, likely initiating several calls to various backend systems and marshalling the results to assemble a single response.
1. REST and RESTful stand for Representational State Transfer. This simply means that the service pro- vider at some internet address is accessed via HTTP commands (GET, PUT, etc.) as if it were a web server.
86 The project detailed in the case study involves collaboration with three specific
groups: the development team creating the IPTV data layer, the vendor providing the REST
interface to the backend and the team of UX developers. Henceforth we will use the unqual-
ified term “development team” to refer to the IPTV data layer team. The other teams will
be referred to specifically.
This chapter proceeds with an overview of the UX specification of the IPTV appli- cation (Section 5.1), followed by a summary of the REST interface (Section 5.2) that the
IPTV data layer component will consume. Following these overviews, various Gherkin scenarios will be introduced, specified and implemented via the BHive methodology
(Section 5.3). The chapter closes with a summary of benefits BHive made to the develop- ment of this theoretical data layer system.
5.1 Introducing the IPTV Application
The IPTV application is a multi-screen, multi-platform project that allows customers of its associated service to access a catalogue of movies pulled from many sources including:
• Recently aired movies shown on a cable television channel to which the user subscribes
(linear TV, catch-up TV)
• Films currently available from one of the user's monthly subscription services (SVOD)
• Content that has been rented or purchased (TVOD)
The application should present content in such a way that customers can easily access their paid content and discover additional content of interest to them based on their viewing habits. The next sections present wireframe diagrams of the IPTV application’s
87 UX, or “screens,” overlaid with a simple summary of backend interactions. We specify the
application here, even though the case study does not involve building it, because the data
layer’s purpose is to make implementation of the application easier, thus we need to define
the latter’s functionality. Therefore, the following five screen descriptions—login, home,
lander, media details, and player—refer to the data layer intermittently.
To reduce the scope and complexity of the case study, digital rights management
(DRM) features are omitted although it typically falls under the purview of systems like the
data layer. A production grade system would include a number of mechanisms to ensure
that content is not made available to unauthorized third parties and cannot be extracted from
the application or back end.
5.1.1 Launching IPTV
The application launches to the “Login Screen.” In launching, whether it be a set box or mobile application, it authenticates with the back end using the hardware address of the device to obtain a billing ID 1. A billing ID maps to a corresponding household which may
own multiple hardware devices and contain several users.
Once authenticated, the application will present the user with a list of profiles asso- ciated with the billing ID. Profiles correspond to persons living at the billing address. Each profile has a name, PIN code and parental restriction code associated with it. Viewing his- tory and content recommendations are tied to this profile. As depicted in Figure 5-2, the
1. Note on author’s identifier spelling conventions in this chapter: The spelling varies by context, thus “bill- ing ID” is for prose use, and “billingId” (camel case) is for specification use. In a Gherkin scenario, the name is “Billing_Id” which also becomes its B machine operation name, while in prose the scenario is called “Billing Id”. Step text is free form, “billing id”, but is subject to exact string matching. Example parameters become Python variables, “billing_id”. Finally, as a data type, “BILLING_ID”. The only genuine constraints are that identifiers which are processed by a language parser—Behave, Python, or ProB—must follow the rules for that language.
88 user will select their named profile, enter their PIN code and begin using the application.
In practice, “adult profile” and “child profile” would be replaced with actual profile names.
Figure 5-2. Wireframe for IPTV login screen
Once a profile is logged in, the application will fetch the list of subscriptions for the
customer associated with the given billing ID. Subscriptions contain a list of movies
(assets) that the customer is entitled to access during a given window provided as start and
end dates.
5.1.2 Home Screen
After profile selection, the IPTV application transitions to the “Home Screen” depicted in
Figure 5-3. The home screen displays a “scrabble” (tiled) list of movie genres, as well as
“Continue Watching”, “Recommendations” and “Purchased and Rented” categories. The data for the home screen is fetched from the back end, sorted by genre and filtered for parental restrictions, if applicable.
5.1.3 Lander Screen
Selecting a genre or category will take the user to a “Lander Screen” shown in Figure 5-4 and described below. The “Continue Watching” category displays all content that a user has started watching, but has not completed. The “Recommendations” category offers a list of
89 Figure 5-3. Wireframe for IPTV home screen movies that the current profile may be interested in based on criteria including their viewing
history and the popularity of the movies themselves. Lastly, the “Purchased and Rented”
category shows a list of movies that have been bought or rented by current billing ID, fil-
tered by parental restriction.
Figure 5-4. Wireframe for an IPTV lander screen
On the lander screen, assets are displayed to match a given set of criteria, such as genre, or in the case of Continue Watching, assets that have not been watched completely.
These “in progress” movie titles display with a bar showing the unwatched duration. As the user navigates landers within the application, the data layer communicates with the back
90 end to populate the screens, filtering the set of assets for the particular screen and caching
data to minimize the load time of screens. Selecting a movie launches the “Media Details
Screen”, discussed next.
5.1.4 Media Details Screen
The “Media Details” screen (not shown) displays the poster for the movie, a bar showing the watch progress, information about the content itself, and buttons that may be enabled or disabled depending on ownership and availability. The “Purchase” or “Rent” buttons add the respective entitlement to the back end accounting, “Watch now” starts movie playback, and “Watch later” adds the movie to the “Continue Watching” category on the home screen. The data layer is responsible for facilitating these transactions and updating its local cache to reflect the new list of entitlements.
5.1.5 Player Screen
When movie playback begins, the “Player Screen” is displayed (shown in Figure 5-5), the application communicates with the data layer to obtain the playback URL and the video starts. If a user exits playback before watching the movie in its entirety, the data layer cre- ates a “bookmark” associating the current progress in the asset with the current profile. This bookmark is used to populate the progress bar on a movie item and add it to the “Continue
Watching” category on the home screen.
5.2 IPTV Backend System
The case study requires an API specification for a notional IPTV backend system. The API is a set of REST calls, accessed via the HTTP protocol and will be described using an
OpenAPI specification (formerly “Swagger specification”) represented in YAML syntax 1.
91 Figure 5-5. Wireframe for the IPTV Player Screen
The API is listed briefly in terms of its GET operations in Table 5-1. The full specification can be found in Appendix B: Specification for IPTV Back End.
Table 5-1. HTTP GET Operations for the IPTV Backend API Endpoint Description /asset/{assetId} Returns the short form metadata for the given asset. Short form metadata is used to minimize network transport times when populating screens that do not require all the informa- tion about a specific asset. /assetDetails/{assetId} Returns the full metadata for the given asset. /bid/{hardwareId} Returns the billing ID associated with the given hardware ID. /bookmarks/{profileId} Returns the list of bookmarks for the given profile. /entitlements/ Returns the list of entitlements for the given subscription. {subscriptionId} /offers/{assetId} Returns the list of entitlements available for purchase for the given asset. /profiles/{billingId} Returns the list of profiles for the given billing ID. /subscriptions/{billingId} Returns the list of subscriptions for the given billing ID.
A data layer is not strictly required for the application, as the various screens and landers could make their own direct calls to the REST API, but this is inadvisable for a number of reasons:
1. YAML [BKEdN09] is described as as “a human friendly data serialization standard for all programming languages.”
92 • Abstracting data management to a specific “layer” decouples it from the application.
Various implementations of the data layer conforming to a single abstract interface
could exist for different vendors and backend configurations, thus minimizing the work
needed for new integrations.
• Containing all data-specific operations is sound object-oriented programming practice,
following the classic Model-View-Controller [GHJV95] paradigm.
• Centralizing the data layer allows it to encapsulate complicated data queries composed
of multiple requests and provide a convenient interface for UX developers. Requiring
individual screens and landers to manage and cache data requests is inviting trouble by
injecting additional state space into the UX code. Furthermore, sharing a cache across
screens is impractical without a central data layer.
For the purpose of simplifying the case study for this work, query “paging” is not supported; a request will return all matching data. The design of a full-fledged system would normally include the ability to specify the offset and maximum result size for a query, and return only the requested window. Another simplification is that the relationship between the back end and data layer is “read-only.” In a real world application, the back end would also include an API for purchases, writing bookmarks, and other operations making use of HTTP PUT and POST operations.
Figure 5-6 shows all the entities contained in the IPTV data layer, and the relation- ships between them. In this iteration of the case study, development focuses primarily on the hardware device, the billing (associated with a household), the profile (many of which are associated with a household, and each of which is associated with a single parental con-
93 trol restriction), and assets holding watchable content with a genre and parental rating. The
detailed fields for each entity that exists in the REST API for the hypothetical backend
system can be found in Appendix B: Specification for IPTV Back End.
Figure 5-6. Entity-relationship diagram of IPTV data entities
5.3 First Development Iteration with BHive
The BHive prototype, as it is currently implemented, builds upon the Python Behave mod- ule. Incorporating BHive into an existing Behave project is a matter of calling the BHive
“hooks” from the environment.py file set up for one’s Behave project, as was shown in
Section 4.3.6. No additional dependencies are required. BHive is invoked automatically when one runs Behave on the project.
To illustrate the BHive development process we will walk through a number of fea-
tures for the IPTV data layer. We will then implement an initial version of each one suffi-
cient to get the BDD feature to pass, making heavy use of mocks and stubs. BHive
94 constructs will be introduced separately, allowing the automatic synthesis of a B-Method
model that provides further insight into the system.
Before the data layer may request data from the back end, it must authenticate by means of a hardware ID, which is then used to determine the authorization available to the particular subscriber. To represent and ultimately implement this functionality, we will create a new feature file named “authentication.feature.” Within the “features” folder of our
BHive project we will create an “IPTVDataLayer” folder that will contain all features we wish to synthesize into the IPTVDataLayer specification. For larger projects, it is likely useful to structure feature files into multiple folders that will result in separate specification outputs. At present, support by BHive for B machine structuring (similar to object-oriented inheritance) is minimal, so synthesis will be restricted to a single B machine.
The first step of the data layer’s authorizing itself to the backend system is for the data layer to obtain the billing ID associated with the hardware ID of the host system. If the request for the billing ID is successful, the data layer will store the billing ID to be used in subsequent requests. The first scenario in the first feature file is shown in Figure 5-7. Pro- vided the device has an associated hardware ID, the data layer will make a request contain- ing the hardware ID. If the response to the request indicates success (HTTP code 200), the data layer should store the billing ID for later use. A table of examples is provided showing sample data and sample responses.
Following a typical BDD development flow, the initial task is to get the first exam- ple of this first scenario to pass. To allow the feature file to “drive” the implementation, five empty steps are created (say, by running Behave with the “--snippets” argument) as shown in Figure 5-8. Each step implementation function called step_impl uses a Python deco-
95 Feature: Authentication As a UX developer, I want a way to authenticate and authorize users of my application.
Scenario Outline: Obtain_Billing_Id Given The device has a hardware id
Examples: | hardware_id | billing_id | status_code | | V8A20u7w | 93773072 | 200 | | DyfcMh1S | 45968589 | 200 | | DXfxMx1X | 45968589 | 200 | | TT7X3qFI | 0 | 403 | | i80yCLRQ | 0 | 404 | | PeLB1TW3 | 0 | 402 | | kMWE9KbF | 0 | 204 |
Figure 5-7. Obtain Billing Id scenario rator to associate it with a step in the feature file. Each one has a single pass statement as its entire body in order to allow the step functions to compile and execute. Invoking Behave on this specification will result in the Obtain Id Scenario and all its examples passing. They do not yet perform any tasks or raise any assertions on the behaviour of the system under development.
At this stage, a BHive user has the choice of introducing BHive primitives to syn- thesize a B specification that may be animated and model-checked, or of performing imple- mentation tasks. For the purposes of illustration, we will begin with a mock implementation and introduce BHive constructs afterward. The first step can be implemented as follows:
@given("The device has a hardware id {hardware_id:HARDWARE_ID}.") def step_impl(context, hardware_id): context.data_layer = DataLayer(hardware_id)
96 @given("The device has a hardware id {hardware_id:HARDWARE_ID}.") def step_impl(context, hardware_id): pass
@when("The data layer requests the billing id for the device.") def step_impl(context): pass
@when("The data layer request completes with status {status_code:STATUS_CODE} and {billing_id:BILLING_ID}.") def step_impl(context, status_code, billing_id): pass
@then("The data layer should have the billing id {billing_id:BILLING_ID}.") def step_impl(context, billing_id): pass Figure 5-8. Empty steps created for the Obtain Billing Id Scenario. Here, an instance of the DataLayer class is initialized with the provided hardware ID. Since
this class does not yet exist, it is an example of writing a test before the implementation,
allowing an implementor to discover the necessary interfaces rather than guessing.
Figure 5-9 shows the first version of the DataLayer class. This initial version accepts a
hardware ID and stores it in an instance variable.
class DataLayer(object): def __init__(self, hardware_id): self.hardware_id = hardware_id Figure 5-9. First version of the DataLayer class.
It should be noted that the Given step creates and stores the instance of the data layer inside the context variable. The context variable is provided by Behave and provides multi-level scoping for use in step functions. In Behave, when a new scope (“Top Level”
(All), Feature, Scenario, Step) is encountered, a fresh key-value pair named context is pushed on the stack. When a scope ends, it is popped from the stack, leaving only the outer
97 scopes. This provision allows a deep integration of BHive with Behave without being
overly invasive.
At this point in development, the team can invoke Behave and observe that all tests are still passing. With the first step of the first scenario passing, the developers turn their attention to implementing the When step of the Obtain Billing Id scenario.
That step contains two separate steps joined with “And”. To code the implementa- tion for these steps, we simply write two @when decorators (or one could use @step for the second, with the latter step obtaining its step type from the previous). This kind of
“And” logic can be used for any of the Given, When and Then scenario steps.
The development team implements these two When steps, once again adding to the interface of the DataLayer class first shown in Figure 5-9, enhanced in Figure 5-10 on page 100. The request for the billing ID is initiated using a new initialize_billing_id member function, and two additional functions are added so the second step can check if the request is complete and get the status code of the last request.
@when("The data layer requests the billing id for the device.") def step_impl(context): context.data_layer.initialize_billing_id()
@when("The data layer request completes with status {status_code} and {billing_id}.") def step_impl(context, status_code, billing_id): assert_that(context.data_layer.is_request_complete(), is_(True)) assert_that(context.data_layer.get_status_code(), equal_to(int(status_code)))
The second When step also contains assertions on the two new member functions to ensure the request completed and the status code matches that provided to the step as a
98 parameter. The assertions use assert_that from the Python Hamcrest [RR16] module, which aims to provide highly readable assertions. A number of BHive primitives were designed to emulate the syntax of Hamcrest to make them familiar to users of Hamcrest and syntac- tically meaningful. Invoking Behave at this point will cause the scenario and its examples to fail, so additional code must be added to the DataLayer class.
Figure 5-10 on page 100 shows the updated DataLayer class. This class can now initiate an HTTP request to the backend system and store the result of the request, as well as provide methods to interrogate the status of the request. The DataLayer class has also been enhanced to accept as the second parameter to its constructor a DataLayerNet- workHandler object, defined in Figure 5-11 on page 101.
By injecting the network handler as a dependency, the development team allows for a mock network handler to be used in place of the real one. This removes a number of com- plications such as propagating network connectivity and latency issues across tests, removes the dependency on the backend system, and allows for the actual network handler to be injected at later stages of development and testing. The mock network handler created for this case study makes use of Python’s requests module [Rei17] and requests-mock
[Len14] that allows us to make the requests as we would in a production system, but instead return predetermined responses. Here, the MockDataLayerNetworkHandler takes as a parameter the path to a JSON file containing a map of URLs and responses to return when those URLs are accessed through the requests module.
With the Given and When steps mocked and the scenario passing once more during invocation of Behave, the development team can set about fleshing out the Then step, as shown below using the get_billing_id member function that was added in Figure 5-10.
99 class DataLayer(object): base_url = 'mock://iptv'
def __init__(self, hardware_id, network_handler=DataLayerNetworkHandler()): self.hardware_id = hardware_id if DataLayer.is_hardware_id_well_formed(hardware_id) else None self.billing_id = 0 self.last_status_code = None self.request_complete = False self.network_handler = network_handler
def initialize_billing_id(self): self.get_billing_id()
def start_request(self): self.last_status_code = None self.request_complete = False
def complete_request(self): self.request_complete = True
def is_request_complete(self): return self.request_complete
def get_status_code(self): return self.last_status_code
def get_billing_id(self): self.start_request() status_code, response_text = self.network_handler.perform_get_request( DataLayer.base_url + '/bid/' + str(self.hardware_id)) self.last_status_code = status_code result_dict = json.loads(response_text) # bug: for a 204, billing is not updated. if self.last_status_code == 200: self.billing_id = result_dict['billingId']['id'] elif self.last_status_code >= 400: self.billing_id = 0 self.complete_request() return self.billing_id Figure 5-10. Second version of the DataLayer class
@then("The data layer should have the billing id {billing_id:BILLING_ID}.") def step_impl(context, billing_id): # here we check that the billing id stored is what we expect
100 class DataLayerNetworkHandler(object): def perform_get_request(self, url): response = requests.get(url) return response.status_code, response.text
class MockDataLayerNetworkHandler(object): def __init__(self, response_filename): # set up mocks here self.adapter = requests_mock.Adapter() self.session = requests.Session() self.session.mount('mock', self.adapter)
f = open(response_filename, 'r') mock_list = MockDataLayerNetworkHandler.get_mock_response_list_from_json(f .read()) f.close()
for mock in mock_list: self.adapter.register_uri(**mock)
def perform_get_request(self, url): response = self.session.get(url) return response.status_code, response.text
@staticmethod def get_mock_response_list_from_json(json_text): return json.loads(json_text)
Figure 5-11. Basic network handler and mock network handler.
assert_that(context.data_layer.get_billing_id(), equal_to(int(billing_id)))
After invoking Behave and verifying that the scenario and its examples are once
again passing, the developers would typically set about refactoring the newly created code
and possibly removing some of the mocks used to make the scenario pass. For the purposes
of this case study refactoring will be omitted, instead using this opportunity to introduce
some formalism into the system we have specified thus far.
Figure 5-12 on page 102 shows the Given step presented before, but now with
BHive directives inserted. In synthesizing a B machine from the Gherkin-based BDD spec-
101 ification, the Given steps form the basis for an operation’s precondition. Here we are spec- ifying the Obtain_Billing_Id scenario for the IPTVDataLayer machine. The name of the operation is taken from the scenario name, and the name of the machine comes from the name of the folder containing the feature file synthesized, here, IPTVDataLayer. @given("The device has a hardware id {hardware_id:HARDWARE_ID}.") def step_impl(context, hardware_id): # bhive declare_parameter(context, 'hardware_id', 'HARDWARE_ID') declare_parameter(context, 'bid', 'BILLING_ID') declare_parameter(context, 'status_code', 'STATUS_CODE') context.state.ensure_that(context, 'status_code', '/=', 'NO_STATUS')
# this is really the only "state" we care about for this operation declare_variable(context, 'billing_id', 'BILLING_ID', 'NO_BILLING_ID') # bhive
context.data_layer = DataLayer(hardware_id, MockDataLayerNetworkHandler( 'features/IPTVDataLayer/implementation/ obtain_billing_id.json'))
Figure 5-12. Revised Given step for Obtain Billing Id scenario
The first BHive invocation shown is declare_parameter. (In the code samples,
“# bhive ” is used to indicate where BHive directives were manually inserted in the imple- mentation.) This is used to specify parameters to the operation, or any external inputs that affect the machine’s state. Here we have specified that the operation is dependent on the hardware ID of the device, the billing ID we receive in our response and the HTTP status code accompanying the response. Each invocation of declare_parameter takes the Behave context as its first argument, the parameter name as its second argument and a type as the third argument. Here, the types HARDWARE_ID, BILLING_ID and STATUS_CODE are
102 used. In B-Method types may be composed of numbers, constants, sets and relations map-
ping one set to another. BHive allows for the definition of types as sets, constants, or in
terms of BHive primitives (integers, natural numbers, etc.). Sets take on two forms: enu-
merated sets where all elements are known and listed exhaustively, or deferred sets whose
elements are not provided and left to the model checker. During development of BHive,
enumerated sets have proven most useful for understanding the specification since ele-
ments may be given recognizable names, making model animation similar to interfacing
with the scenarios via their example tables.
Appendix A: Sample BHive Custom Types File shows how a development team
can enumerate sets by populating a file, as well as the resulting synthesized CustomTypes
machine. That machine is synthesized separately, and included in all synthesized BHive
machines via B-Method’s SEES structuring mechanism, thus these types may be used
across the BHive project.
This line inserted in the Figure 5-12 implementation specifies the precondition of
the operation—the requirements on system state for the Obtain Billing Id scenario to be
invoked and a valid state change to take place: context.state.ensure_that(context, 'status_code', '/=', 'NO_STATUS') 1
Similarly, to declare variables in the synthesized machine, the follow invocation was inserted: declare_variable(context, 'billing_id', 'BILLING_ID', 'NO_BILLING_ID')
1. “/=” is B-Method syntax for the negation of the equality operator (“not equal to”).
103 Here, a variable named “billing_id” is declared and its type is specified as BILLING_ID, as defined in previously created custom types.
Skipping to the synthesized B machine shown in Figure 5-13 on page 104, the fol- lowing connections are apparent:
MACHINE IPTVDataLayer
SEES LibraryStrings, CustomTypes
VARIABLES billing_id
INVARIANT billing_id : BILLING_ID
INITIALISATION billing_id := NO_BILLING_ID
OPERATIONS
Obtain_Billing_Id(hardware_id,bid,status_code) = PRE hardware_id : HARDWARE_ID & bid : BILLING_ID & status_code : STATUS_CODE & status_code /= NO_STATUS THEN billing_id := bid END END Figure 5-13. Synthesis of B machine after completing Obtain Billing Id Scenario.
• The “SEES” clause is used to include both the synthesized CustomTypes machine and
LibraryStrings which enables full-fledged support for strings within the ProB tool
[LB08].
• billing_id appears in the variable list for the machine. Its type is specified as part of the
machine’s invariant and it is initialized to a constant value NO_BILLING_ID.
• The parameters hardware_id, bid and status_code are generated for the
Obtain_Billing_Id scenario. The types for each of these parameters are included in the
104 precondition (PRE) for the operation, specifying that each of the parameters must be an
element of the sets defined in CustomTypes .
• The synthesized state change: THEN billing_id := bid is discussed next.
Figure 5-14 on page 106 shows the two When steps for the “Obtain Billing Id” sce-
nario, the first of which does not include any additional BHive constructs, but the second
step— @when("The data layer request completes with status...") — shows the state change. It should be noted the mock implementation of DataLayer contains additional state, but the development team has made a selection of what state is useful for modelling activities. This is in line with Gardner’s notion of selective formalism described in Section 4.1. Modelling the billing_id allows the development team to observe and make conjectures about billing_id in relation to the operation’s parameters and additional opera- tions that will be added later.
Figure 5-14 also includes the Then step— @then("The data layer should
have the billing id...") —containing: context.state.test_that(context,'billing_id','/=', 'NO_BILLING_ID') The Then step is used by traditional BDD to verify a state change after the When step.
BHive re-purposes this slightly in that it allows predicates to be added to test case genera- tion for this operation. In the case of Obtain_Billing_Id, we specify that we want to test that billing_id is not equal to the constant NO_BILLING_ID after the Obtain_Billing_Id oper- ation has occurred. ProB’s test generation tooling employs a constraint solver that will pro- duce test cases containing parameter values if this state is reachable. These test cases can then be manually applied against the implemented system to see if invalid states can be reached. Testing for the case study is discussed in further detail in Section 5.3.5.
105 @when("The data layer requests the billing id for the device.") def step_impl(context): # bhive # we're external to the data layer, so we don't care about network state during this iteration, i.e. no state change # /bhive # this performs the request to the data layer to get the billing id from the hardware id context.data_layer.initialize_billing_id()
@when("The data layer request completes with status {status_code:STATUS_CODE} and {billing_id:BILLING_ID}.") def step_impl(context, status_code, billing_id): # bhive context.state.assign([('billing_id', ':=', 'bid')]) # /bhive # here we check that the request completed, we got the expected status code assert_that(context.data_layer.is_request_complete(), is_(True)) assert_that(context.data_layer.get_status_code(), equal_to(int(status_code)))
@then("The data layer should have the billing id {billing_id:BILLING_ID}.") def step_impl(context, billing_id): # bhive context.state.test_that(context, 'billing_id', '/=', 'NO_BILLING_ID') # /bhive # here we check that the billing id stored is what we expect assert_that(context.data_layer.get_billing_id(), equal_to(int(billing_id)))
Figure 5-14. Completed When and Then steps for the Obtain Billing ID scenario.
Thus far, we have looked at the specification of a single scenario in a single feature file and synthesized a single operation and its associated types. In the following sections, the presentation will continue with additional authentication and authorization related sce- narios, and finally with test case generation. The ongoing code samples will show only
BHive directives, and will not include the implementation code for the sake of space.
106 5.3.1 Fetch Profiles Scenario
After the data layer has determined the billing ID for the given hardware device, the data layer will fetch the list of profiles for that billing ID from the backend. This scenario is given in Figure 5-15. Profiles correspond to people within the household with their own viewing history and parental control settings. The single row in the examples table in
Figure 5-15 shows a sample result having five profiles.
The Given step for Fetch_Profiles is shown below:
@given("The device is associated with a billing_id {billing_id:BILLING_ID}.") def step_impl(context, billing_id): context.state.ensure_that(context, 'billing_id', '/=', 'NO_BILLING_ID')
It requires only that the data layer has an assigned billing ID. Both the variable billing_id and the constant NO_BILLING_ID were defined in the previous step, and can simply be referenced here. The resulting B machine operation is given in Figure 5-16.
The When and Then steps are shown next:
Scenario Outline: Fetch_Profiles Given The device is associated with a billing_id
Examples: | billing_id | status_code | count | names | | 93773072 | 200 | 5 | Abe, Anne, Huck, John, Willie | Figure 5-15. Fetch Profiles scenario
Fetch_Profiles(profiles_list, status_code) = PRE profiles_list <: PROFILE_NAMES & status_code : STATUS_CODE & status_code /= NO_STATUS & billing_id /= NO_BILLING_ID THEN profiles := profiles_list END; Figure 5-16. Synthesized operation for Fetch Profiles
107 @when("The data layer request completes with status {status_code:STATUS_CODE} and the profiles {names:STRING}.") def step_impl(context, status_code): # assigning {} makes it a set. declare_parameter(context, 'profiles_list', '{PROFILE_NAMES}') declare_parameter(context, 'status_code', 'STATUS_CODE') declare_variable(context, 'profiles', 'PROFILE_NAMES', '{}') context.state.assign([('profiles', ':=', 'profiles_list')])
@then("The data should have {count:INT} profiles.") def step_impl(context, count, names): # simplification here... context.state.test_that(context, 'card(profiles)', '/=', '0')
The When step declares two parameters for fetching profiles, that is, values which the developers want to include in their formal model, typically values that can vary. In the case of Fetch_Profiles, the state change is an assignment of the fetched profiles in profiles_list to the machine’s profiles variable. The When step labeled “The data layer requests the sub- scription profiles.” is omitted, since in the current iteration the development team has chosen not to model a state change for the act of requesting, instead tying the state change to its completion. In the Then step we see a single predicate checking that the cardinality of profiles is non-zero after the When step. At present, the B tools do not support the ability to pass parameters to the operations being tested, so test predicates must be written more generally—providing a predicate that should be true for all invocations of the operation, regardless of parameter values. The synthesized B machine is not intended to be used as an
“oracle” to check input parameters against expected outputs; instead, it is intended to be used to look for problems of correctness and consistency in the state space of said inputs and outputs.
In animating the B model presented in Figure 5-16, the development team would observe that the Fetch_Profiles operation can be called multiple times, saving the list of
108 profiles with each invocation. They would determine that this is unwanted behaviour: once
the list of profiles has been fetched it should remain unchanged until the user logs out. In
allowing the list of profiles to change there is the potential for a user to be using a profile
that gets overwritten on a subsequent call to Fetch_Profiles. The development team does
not know if fetching profiles multiple times will be possible in the final system, but through
the exercise of animating the model (likely with the customer watching) they have
improved their understanding of how the system works, and will introduce additional cor-
rectness provisions to the Gherkin feature file as shown in Figure 5-17 in bold.
Scenario Outline: Fetch_Profiles Given The device is associated with a billing_id
Figure 5-17. Amended Fetch Profiles scenario
The change will be added to the synthesized specification with the addition of the following
step:
@given("The set of profiles for the device is empty.") def step_impl(context): context.state.ensure_that(context, 'profiles', '=', '{}')
The updated B specification for the operation is given in Figure 5-18, noting the addition of profiles = {} to the precondition for the operation. Fetch_Profiles(profiles_list, status_code) = PRE profiles_list <: PROFILE_NAMES & status_code : STATUS_CODE & status_code /= NO_STATUS & billing_id /= NO_BILLING_ID & profiles = {} THEN profiles := profiles_list END; Figure 5-18. Re-synthesized Fetch Profiles operation
109 In this instance, the ability to “click through a model” and trace possible execution
paths has informed the development process and diagnosed a potentially unwanted state
change.
5.3.2 Populating Content Restrictions
During the login process the set of content restrictions for the profiles associated with the billing ID are fetched. The content restrictions allow parents of children to control the con- tent they consume. In the REST API for the IPTV backend system (provided in Appendix
B: Specification for IPTV Back End), content restriction is represented as an integer prop- erty within the profile object. For the sake of space, this operation will not be discussed in detail as it is functionally similar to the two scenarios just presented. It appears in the fully synthesized specification shown in Figure 5-23 on page 116.
Of note is the inclusion of the variable profile_restrictions which would be defined using a BHive directive: declare_variable_function(context,'profile_restrictions' 'PROFILE_NAMES --> RESTRICTIONS', '{}')
This statement declares a variable that is a function from the domain of PROFILE_NAMES to the range of RESTRICTIONS. The use of the --> denotes a total function (B-Method uses +-> to specify a partial function). Both of these function types are supported by BHive.
B-Method does offer additional function types (surjective, injective, bijective) of both the total and partial variety, but are not supported by the current implementation of BHive in order to allow immediate implementation using Python’s basic types. Such an enhance- ment can be readily achieved using a custom data structure that allows for the mapping of multiple values in the domain of a relation to values in its range. It should be noted that the
110 Populate_Restrictions scenario could be rolled into the Fetch_Profiles scenario, but was
broken out for the sake of readability.
5.3.3 Select Profiles Scenario
Following the successful completion of the data layer’s obtaining the list of profiles and content restrictions from the back end, the data layer will allow the selection of a profile from those fetched. A possible UX design for the presentation of profile selection was given in Figure 5-2 on page 89. The scenario for profile selection is shown in Figure 5-19. Scenario Outline: Select_Profile Given The data layer has fetched the profiles associated with
A user’s selecting a profile has the side effect of applying that profile’s parental control restriction level, hence there is a second Given step that requires the parental restric- tions to have been fetched prior to this scenario. The steps for this scenario are provided in
Figure 5-20.
Two inclusions to highlight are in the When step. First, we see two assignments per- formed 1. Aligning with “parallel assignment” in B-Method, all assignments in a BHive
When step are performed simultaneously. Secondly, one assignment includes func- tion(profile_restrictions, profile_name) on the right hand side of the assignment. Currently, this is the means in BHive for obtaining a value from a total or partial function variable.
Here, the expression returns the value to which profile_name maps in the function
1. These assignments could been combined into a single statement using a list of tuples in Python. Separate assignments were coded for purposes of readability.
111 @given("The data layer has fetched the profiles associated with {billing_id}.") def step_impl(context, billing_id): declare_variable(context, 'active_profile', 'PROFILE_NAMES', 'NO_PROFILE') declare_variable(context, 'active_restriction', 'RESTRICTIONS', '0') declare_parameter(context, 'profile_name', 'PROFILE_NAMES') context.state.ensure_that(context, 'profiles', '/=', '{}') context.state.ensure_that(context, 'profile_name', ':', 'profiles') context.state.ensure_that(context, 'profile_name', '/=', 'NO_PROFILE') context.state.ensure_that(context, 'profile_restrictions', '/=', {})
@given("The data layer has fetched parental restrictions for all profiles.") def step_impl(context): context.state.ensure_that(context, 'profile_restrictions', '/=', '{}')
@when("A profile with {profile} is selected with {restriction}.") def step_impl(context, profile, restriction): context.state.assign([('active_profile', ':=', 'profile_name')]) context.state.assign([('active_restriction', ':=', 'function(profile_restrictions, profile_name)')])
@then("The parental control restriction should be {restriction}.") def step_impl(context, restriction): pass Figure 5-20. Steps for Select_Profile Scenario. profile_restrictions, which is a partial function from the set of profile names to parental control restrictions.
Also of note is that the inclusion in the Given step of this line: context.state.ensure_that(context, 'profile_restrictions', '/=', {})
It ensures that profile_restrictions is not empty, implicitly requiring that
Populate_Restrictions() had been invoked earlier. This is another example of model explo- ration informing the non-formal development process. In examining this model, the devel- opment team would see this dependency clearly, and decide whether checking for a non- empty list of profiles was sufficient to ensure this requirement, or if additional implemen- tation logic is needed to prevent a profile from being selected without parental control restrictions having been populated. This is also another example of a possible error intro- duced in the final implementation as a result of shared state. It is not hard to imagine that a
112 lack of diligence in implementation could result in a situation where a child was able to
select a profile before the profile_restrictions set had been completely fetched from the
back end, potentially allowing them access to inappropriate content.
5.3.4 Populate Lander Scenario
The final scenario presented in the case study is the means of populating lander screens with assets. A possible UX design for the home screen was given in Figure 5-3 on page 90, where a user may select a specific genre (or relation) of content which will then present it using a lander as depicted by the wireframe in Figure 5-4 on page 90.
In practice, landers typically hold a subset of all the content available in the system, filtered according to one or more associations. On the aforementioned home screen, we see a list of landers related to movie genres, but also for other groups such as movies to be rec- ommended to the user, movies the user has watched partially or movies the user has rented or purchased.
In the author’s experience with IPTV systems, there are two common alternative paradigms for fetching data to populate landers:
• The entire catalog of data is fetched and stored in the data layer. The set of assets for
each lander is filtered given criteria such as genre, or other data properties such as a
“purchased” flag.
• As the application transitions to a lander, the lander initiates a query to the backend sys-
tem to obtain the list of relevant assets.
There are advantages and disadvantages to each paradigm, and the interaction
design of the application itself may tend toward one or the other. Additionally, if a long-
113 lived caching strategy is used, the second paradigm may begin to resemble the first. In this
case study, the first paradigm will be used.
Figure 5-21 shows the scenario for populating landers with assets.
Scenario: Populate_Lander Given The data layer asset fetch completed with status
To simplify the case study presentation, it is assumed that the entire asset catalog
was fetched previously along with parental control and genre information for assets con-
tained in the catalog 1. When a lander is selected, the entire asset catalog will be filtered based on the lander type and age rating of the currently selected profile.
1. Since it is a trivial operation that assigns one set to another, the scenario for the “Populate_Catalog” oper- ation was omitted for space.
114 The scenario steps for the Populate_Lander scenario are given in Figure 5-22 and the final synthesized operation appears in Figure 5-23 on page 116. @given("The data layer asset fetch completed with status
@given("A profile
@when("The active lander is set to
@then("The asset list should contain assets of type
@then("The asset list should contain assets at or under the current parental restriction level.") def step_impl(context): context.state.test_pass_raw(context, 'bool(card({x | x:active_asset_list & catalog(x) > active_restriction}) = 0)')
Figure 5-22. Steps for Populate Lander scenario
The Populate_Lander steps make use of constructs introduced previously, with two notable additions. The following statement will apply a “filter” across the domain of a func- tion—accepting only elements in the domain of the function that satisfy the given predi- cates—here specifying that the rating for an asset must be less than or equal to the current parental restriction level, and the genre for the asset must match the parameter lander_type:
115 MACHINE IPTVDataLayer SEES LibraryStrings, CustomTypes
VARIABLES catalog, billing_id, profiles, active_profile, subscriptions, active_asset_list, active_lander, profile_restrictions, active_restriction, genre
INVARIANT catalog : NAT1 +-> RESTRICTIONS & billing_id : BILLING_ID & profiles <: PROFILE_NAMES & active_profile : PROFILE_NAMES \/ {NO_PROFILE} & subscriptions <: SUBSCRIPTIONS & active_asset_list <: NAT1 & active_lander : LANDER_CONFIGS & profile_restrictions : PROFILE_NAMES +-> RESTRICTIONS & active_restriction : RESTRICTIONS & genre : NAT1 +-> STRING
INITIALISATION catalog := {} || billing_id := NO_BILLING_ID || profiles := {} || active_profile := NO_PROFILE || subscriptions := {} || active_asset_list := {} || active_lander := "NONE" || profile_restrictions := {} || active_restriction := 0 || genre := {}
OPERATIONS Obtain_Billing_Id(hardware_id,bid,status_code) = PRE hardware_id : HARDWARE_ID & bid : BILLING_ID & status_code : STATUS_CODE & status_code /= NO_STATUS & billing_id = NO_BILLING_ID THEN billing_id := bid END;
Fetch_Profiles(profiles_list, status_code) = PRE profiles_list <: PROFILE_NAMES & status_code : STATUS_CODE & status_code /= NO_STATUS & billing_id /= NO_BILLING_ID & profiles = {} THEN profiles := profiles_list END;
Select_Profile(profile_name) = PRE profile_name : profiles & profiles /= {} & profile_name /= NO_PROFILE & profile_restrictions /= {} THEN active_profile := profile_name || active_restriction := profile_restrictions(profile_name) END;
Populate_Restrictions(restriction_list) = PRE restriction_list : PROFILE_NAMES --> RESTRICTIONS & profile_restrictions = {} THEN profile_restrictions := restriction_list END;
Populate_Catalog(aa) = PRE aa : NAT1 +-> RESTRICTIONS & card(aa) = 10 THEN catalog := aa END;
Populate_Lander(lander_type, status_code) = PRE lander_type : LANDER_CONFIGS & status_code : STATUS_CODE & active_profile /= NO_PROFILE THEN active_asset_list := {x | x:dom(catalog) & catalog(x) <= active_restriction & genre(x) = lander_type} END
END
Figure 5-23. Synthesized IPTVDataLayer machine context.state.assign_filtered_domain('active_asset_list', ['catalog(x) <= active_restriction', 'genre(x) = lander_type'])
In examining the synthesized output, this construct maps directly to a set comprehension on the domain of the function.
Secondly, the second Then step (“The asset list should contain assets at or under the current parental restriction level.”) includes a test predicate. The test_pass_raw method passes the predicate directly due to the synthesis step in order to sidestep parsing limitations
116 in the current testing support with BHive. The predicate here insures that the set of assets
in the active_asset_list (current lander) that exceed the current parental rating is empty (car-
dinality of 0).
In the next section, case study testing is described.
5.3.5 Generating Test Cases for the IPTV Data Layer
In the development of the case study, the Then steps shown in several of the listings included the use of the “test_that” construct. This construct is used as a trigger for generat- ing test cases via the ProB tool [LB08]. An invocation of test_that with a predicate adds an association of that predicate and the operation synthesized. Following the synthesis of the
B machine, the BHive tool will print a list of operations and predicates for test case gener- ation. These predicates, and the operations that should be covered in the generated test cases, can be manually input into ProB (as shown in Figure 5-24), or input by BHive via the ProB command line interface.
Test case generation support in BHive at present is limited, but does provide addi- tional value to the BHive approach. ProB supports both constraint-based-checking (CBC) and model-based-checking. CBC does not construct the entire state space, instead using
ProB's constraint solver to find relevant traces of operations. In applying CBC test case generation, a development team is able to synthesize traces in XML format for a given predicate that can be executed against the final system, either manually or using a test auto- mation tool such as Appium [Han15] which is frequently used by BDD or TDD develop- ment teams.
117 Figure 5-24. Generating Test Cases in ProB Tool
Test case generation in BHive is not intended to replace traditional manual testing, but instead augment the development process with a formal model. The ability to synthe- size test cases allows the development team to work more effectively with their ever-evolv- ing understanding of the system and its requirements.
118 5.3.6 Application of BHive Guidelines
In Section 4.2.6, three guidelines for BHive users were stated. Let us return to those (in ital- ics) and show how they were followed in the context of this case study.
• BHive variables and parameters need only be created to capture data types and their
respective state space that needs to be understood and explored. In this instance the rel-
evant data was primarily identifiers and relationships between identifiers. Secondary
and tertiary data such as the title of assets or the URLs of asset posters were not
included as there was no need for verifying them formally.
• As a starting point for deciding what to declare formally in a Gherkin step function, it is
recommended to consider the interactions with the system and its environment, specifi-
cally what information passes between the boundary of the environment and the system
informs the parameters chosen, and what the system is responsible for “remembering”
following an operation informs variables. In the case of the IPTVDataLayer, the param-
eters were responses containing query data and HTTP status codes, and variables were
introduced that corresponded to state required for subsequent operations and/or to pop-
ulate the UX elements.
• The developer is free to pass state outside of BHive directives, but doing so is informal
and creates a partition between the behaviour of the system under development and its
representational model. In this case study, data was passed strictly via BHive direc-
tives.
119 5.3.7 Benefits
In this chapter a short worked example of applying BHive to a larger, complicated system was presented. Through a small set of scenarios, a useful model was synthesized and uti- lized to provide a hypothetical development team with feedback and a better shared under- standing of the system being created. Additionally, BHive served to highlight potential correctness problems with ordering of cascading requests in the login system, and invariant violations where variables were used in error cases without first being assigned. Several unreachable states were also discovered during both model exploration and test generation that resulted in modifications to Given step functions. Expanding testing support and the introduction of documentation generation are major areas of future work, discussed in the next chapter.
120 Chapter 6 Conclusions and Future Work
This work puts forward a novel approach to the integration of a contemporary agile devel- opment process and a rigorous formal specification medium for the purpose of creating cus- tomer value through correctness. Correctness in this sense has two distinct interpretations:
Are we building the system the client has asked for and is it one that will create value for them?
Does the system implementation do what we intend it to?
BHive, in combining an iterative, outside-in development method with a rigorous formal- ism possessing a proven track record, provides the means for development teams to be cor- rect in both senses.
The iterative nature of BDD allows project teams to achieve and show regular progress to clients. BDD provides transparency through the Gherkin specification lan- guage, offering the chance for short-term course correction (“a little rudder”) by “speaking the language” of the customer and making the language of the client ubiquitous across the project team. The introduction of B-Method via BHive into the BDD process allows for the project team to gain deep insights into the system under development through providing a model that can be explored, be verified and inform testing activities. As does the use with the customer of a common parlance provided by BDD, deficiencies found through the application of formalism also provide fine-grained course correction. Inconsistencies found
121 in the model inform the understanding of the development team, and BHive allows them to
select “just enough” formalism to get value in their development iteration without the
inherent risk of the “big design up front” often associated with formal methods.
A selling point of agile methods is the notion of a system being ready to be delivered
“at any time.” 1 This means that if a client decides the project team has completed enough
of the system so that it provides value to the client’s customers, then it may ship. This is not
possible with traditional plan-based approaches associated with formal methods where
specification activities are front-loaded. BHive allows just enough formalism to be intro-
duced within a development sprint, and for the newly injected formalism to be related to
the task at hand in the current sprint.
As explained in Section 4.1, this work was inspired by two antecedents, Gardner's
selective formalism and Mise en Scene for the R2D2C project. It is the author’s hope that
bridging the realms of these projects with modern day agile practices creates a method that
results in better systems, empowered developers and delighted customers.
6.1 Contributions
This list summarizes the research contributions embodied in the BHive approach:
1. BHive's major theoretical contribution is a reinterpretation of the Given-When-Then
steps (the Hoare triple as discussed in Section 3.4.3) used in behavioural-driven devel-
opment processes to form the basis of an approach that maps Gherkin scenarios to
operations in a synthesized B machine. This constitutes exploiting a latent formalism
1. Generally at the end of a sprint when a block of work is complete and verification activities have been performed.
122 within a non-formal method and leveraging it in the interests of integrating a popular
agile process with B-Method.
2. The process by which a B machine’s code is generated from a Gherkin feature file is
presented in Section 4.2. While the worked example is specific to Gherkin, BHive is
nonetheless a general approach and can be applied to any flavor of BDD and the
respective tooling(s) for that variation.
3. An initial realization of the BHive approach has been written in Python that demon-
strates both its feasibility and the ease by which BHive may be “bolted on” to an exist-
ing tooling approach. A demonstration of this is provided in Section 4.3. The current
BHive software is available at: https://github.com/mrjohndcarter/bhive.
4. A set of guidelines for the effective application of BHive to a system being developed
using a Behaviour-driven approach is given in Section 4.2.6. These guidelines build
upon antecedent work under the rubric of “selective formalism” where development
teams are able to inject custom levels of formalism into a traditional development pro-
cess to apply additional rigour to critical system components.
5. Basic support for test case generation from a synthesized B machine using widely
available B-Method tools was shown in Section 4.3.8.
6. Finally, a detailed case study is presented in Chapter 5 that applies BHive to the devel-
opment of an IPTV application’s DataLayer. The application contains a significant
degree of complexity, and BHive was shown to be compatible with the project and its
iterative approach to development.
Contributions numbered 2 and 3 have been published in the International Workshop
on Formal Methods Integration (FMi) of the International Conference on Information
123 Reuse and Integration (IEEE IRI) [CG16], and in extended version along with numbers 1
and 5 as an invited paper in the journal Advances in Intelligent Systems and Computing
[CG17].
6.2 Future Work
The current realization of BHive is a result of intense experimentation in combining Gher- kin and B-Method. The present syntax for BHive directives is not likely palatable for teams wary of injecting set theory and first order logic into step functions. The remedy for this is through the provision of a more abstract, higher-level interface, wrapping the directives presented previously with an interface that bears a resemblance to a widely-used matching library such as Hamcrest [RR16] or Python's mocking provisions. Furthermore, the existing
BHive parser is admittedly naive and offers no shortage of syntactic pitfalls due to its reli- ance on strings as parameters. There is significant room for improvement in the area of detecting syntax problems and providing meaningful feedback about error conditions.
In addition to the enhancement of BHive directives, support for B-Method’s com- posite type (“record”) and hierarchical type (“tree”) is planned. Such an integration would allow for these types to be parsed from the text content of a Gherkin file, exist as python objects in step functions and be synthesized as their respective B-Method type.
Due to B-Method's lack of concurrency, work to date has applied the view that sce- narios are single-threaded and executed sequentially. It is desirable to allow for concur- rency and its associated complexity to be introduced behaviorally (likely within a Gherkin scenario) and for synthesized specifications to contain concurrency constructs as found with an event-based formalism such as CSP.
124 As mentioned in the introduction, the same set of semantics about the system used for B-Method specification synthesis can form the basis for documentation generation.
Information such as typing, pre- and postconditions for operations and potential machine flows can be outputted in a format palatable to nontechnical team members. Additionally, this documentation can provide traceability of the system by linking feature files, scenarios, step functions, types and machine operations in an easy to navigate “hyperlinked” presen- tation.
Following refinement of the syntactic considerations, there is potential for addi- tional research to be carried out by evaluating BHive alongside a “vanilla” BDD develop- ment project and a “big design up front”-style project employing a formal method, particularly with metrics such as bug count and team velocity.
It would also be important to evaluate this approach “at scale.” BHive should be applied to a project larger than the included case study and the work should be performed by a larger, cross-functional team. Furthermore, an evaluation of BHive in the context of a project containing multiple developers should be conducted. Of particular interest is the effect of BHive on intra-team communication, and practical matters such as using BHive with source control. To date, BHive has been used only for single developer projects. Such an evaluation would serve to validate BHive’s value in a collaborative setting.
Test case generation stands to gain from closer integration with tools such as ProB.
The ability to pass parameters directly from scenario examples into traces would create yet another link in the integration of BDD and B-Method.
Lastly, at the time of writing, Abrial's seminal “B Book” [Abr96] is over twenty years old. Event-B [Abr10] represents the evolution of B-Method, offering a simpler nota-
125 tion, and is intended less for the development of “correct-by-construction” software and more for modelling all aspects of a system (including its environment). Moving BHive to
Event-B is practical due to Event-B's ancestry and robust tool support provided by the
Rodin platform [ABHV06]. Following migration to Event-B, the possibility of integrating formalisms other than B-Method and Event-B should be evaluated.
126 Bibliography
References
[ABC+13] Michal Antkiewicz, Kacper Bak, Krzysztof Czarnecki, Zinovy Diskin, Dina Zayan, and Andrzej Wasowski. Example-driven modeling using Clafer. In CEUR Workshop Proceedings , volume 1104, pages 32–41, 2013.
[ABHV06] Jean-Raymond Abrial, Michael Butler, Stefan Hallerstede, and Laurent Voi- sin. An open extensible tool environment for Event-B. In Z. Liu and J. He, editors, Formal Methods and Software Engineering. ICFEM 2006, volume 4260 of Lecture Notes in Computer Science , pages 588–605. Springer Ber- lin Heidelberg, 2006. [Abr96] J.-R. Abrial. The B-book: Assigning Programs to Meanings . Cambridge University Press, New York, NY, USA, 1996. [Abr10] Jean-Raymond Abrial. Modeling in Event-B: System and Software Engi- neering . Cambridge University Press, 2010. [Adz11] Gojko Adzic. Specification by Example: How Successful Teams Deliver the Right Software . Manning Publications Co., Greenwich, CT, USA, 2011. [Amb13] Paolo Ambrosio. Cucumber-cpp wiki [online]. 2013 [cited Mar. 12, 2017]. Available from: https://github.com/cucumber/cucumber-cpp/wiki. [BA04] Kent Beck and Cynthia Andres. Extreme Programming Explained: Embrace Change . Addison-Wesley Professional, 2nd edition, 2004. [BBvB+01] Kent Beck, Mike Beedle, Arie van Bennekum, Alistair Cockburn, Ward Cunningham, Martin Fowler, James Grenning, Jim Highsmith, Andrew Hunt, Ron Jeffries, Jon Kern, Brian Marick, Robert C. Martin, Steve Mellor, Ken Schwaber, Jeff Sutherland, and Dave Thomas. Manifesto for Agile Software Development, 2001. Available from: http://www.agilemani- festo.org/. [Bec02] Beck. Test Driven Development: By Example . Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2002. [BKEdN09] Oren Ben-Kiki, Clark Evans, and Ingy döt Net. YAML ain’t markup lan- guage (YAMLtm) version 1.2, Oct. 2009. Available from: http:// www.yaml.org/spec/1.2/spec.html.
127 [Bra14] T. Bray. The JavaScript Object Notation (JSON) Data Interchange Format. RFC 7159 (Proposed Standard), March 2014. Available from: http:// www.ietf.org/rfc/rfc7159.txt.
[CAD+10] David Chelimsky, Dave Astels, Zach Dennis, Aslak Hellesoy, Bryin Helmkamp, and Dan North. The RSpec Book: Behaviour Driven Develop- ment with RSpec, Cucumber, and Friends . Facets of Ruby. Pragmatic Book- shelf, 2010.
[Car07] John Douglas Carter. Mise en Scene: A scenario-based approach to gener- ating CSP system traces. Master’s thesis, University of Guelph, 2007. Avail- able from: http://www.collectionscanada.gc.ca/obj/thesescanada/vol2/002/ MR25403.PDF.
[CG06] J. Carter and W.B. Gardner. A Formal CSP Framework for Message-Pass- ing HPC Programming. 2006 Canadian Conference on Electrical and Com- puter Engineering , pages 1466–1470, 2006. Available from: http:// ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4054886. [CG08] J. Carter and W.B. Gardner. Converting scenarios to CSP traces with Mise en Scene for requirements-based programming. Innovations in Systems and Software Engineering , 4(1):45–70, Apr 2008. [CG16] J. Carter and W.B. Gardner. BHive: Towards Behaviour Driven Develop- ment supported by B Method. In Proceedings of IEEE 17th International Conference on Information Reuse and Integration (IRI) , pages 249–256, 2016. [CG17] John D. Carter and William B. Gardner. BHive: Behaviour-Driven Devel- opment meets B-Method. Advances in Intelligent Systems and Computing , 2017. To appear. [CGRH07] J. Carter, W.B. Gardner, J.L. Rash, and M.G. Hinchey. Mise en Scene: Sce- nario to CSP trace conversion for the Requirements to Design to Code project. Technical Report TM-2007-214155, National Aeronautics and Space Administration, 2007. Available from: https://ntrs.nasa.gov/ search.jsp?R=20070031562. [Cuc16] Cucumber Limited. Gherkin wiki [online]. 2016 [cited May 1, 2016]. Avail- able from: https://github.com/cucumber/cucumber/wiki/Gherkin. [Cuc17] Cucumber Limited. Cucumber website [online]. 2017 [cited Mar. 12, 2017]. Available from: https://cucumber.io. [Dij72] Edsger W. Dijkstra. The humble programmer. Commun. ACM , 15(10):859– 866, 1972. ACM Turing Lecture.
128 [EMT05] Hakan Erdogmus, Maurizio Morisio, and Marco Torchiano. On the effec- tiveness of the test-first approach to programming. IEEE Transactions on Software Engineering , 31(3):226–237, 2005.
[Eng16] Jens Engel. behave examples and tutorials [online]. 2016 [cited Mar. 11, 2017]. Available from: https://jenisys.github.io/behave.example/. [Eva03] Evans. Domain-Driven Design: Tackling Complexity in the Heart of Soft- ware . Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 2003. [FMPW04] S. Freeman, T. Mackinnon, N. Pryce, and J. Walnes. Mock roles, not objects. Companion to the ACM conference on Object-Oriented Program- ming, Systems, Languages and Applications (OOPSLA) , pages 236–246, 2004. Available from: http://www.jmock.org/oopsla2004.pdf.
[Fow04] Martin Fowler. Specification by example, 2004. Available from: http://mar- tinfowler.com/bliki/SpecificationByExample.html. [Gar05] William B. Gardner. Converging CSP specifications and C++ programming via selective formalism. ACM Transactions on Embedded Computing Sys- tems , 4(2):302–330, May 2005. Available from: http://portal.acm.org/cita- tion.cfm?doid=1067915.1067919. [Gär12] Markus Gärtner. ATDD by Example: A Practical Guide to Acceptance Test- Driven Development . Addison-Wesley Professional, 2012. [GCGS09] W.B. Gardner, J. Carter, A. Gumtie, and Y. Solovyov. CSP++ : An open source tool for building concurrent applications from CSP specifications. Technical Report TR-UG-CIS-2009-001, School of Computer Science, University of Guelph, 2009. [GGC15] William B. Gardner, Alicia Gumtie, and John D. Carter. Supporting selec- tive formalism in CSP++ with process-specific storage. In Proc. 12th IEEE International Conference on Embedded Software and Systems (ICESS) , pages 1057–1065. IEEE, 2015. Available from: http://dx.doi.org/10.1109/ HPCC-CSS-ICESS.2015.265. [GHJV95] Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns: Elements of Reusable Object-oriented Software . Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1995. [Han15] Manoj Hans. Appium Essentials . Packt Publishing, 2015. http://appium.io/. Available from: http://appium.io [cited March 19, 2017]. [HFFC13] Bernardo Heynemann, Claudio Figueiredo, Gabriel Falcão, and Guilherme Chapiewski. Pyccuracy acceptance testing framework wiki [online]. 2013 [cited March 10, 2017]. Available from: https://github.com/heynemann/ pyccuracy/wiki.
129 [Hoa69] C.A R. Hoare. An axiomatic basis for computer programming. Communica- tions of the ACM , 12(10):576–580, 1969.
[Hoa78] C.A.R. Hoare. Communicating Sequential Processes. Communications of the ACM , 1978. Available from: http://dl.acm.org/citation.cfm?id=359585. [Kud16] Konstantin Kudryashov. Behat website [online]. 2016 [cited Mar. 12, 2017]. Available from: http://behat.org/en/latest/.
[LB08] Michael Leuschel and Michael Butler. ProB: An automated analysis toolset for the B method. International Journal on Software Tools for Technology Transfer , 10(2):185–203, 2008. Available from: http://dx.doi.org/10.1007/ s10009-007-0063-9.
[Len14] Jamie Lennox. Welcome to requests-mock’s documentation! [online]. 2014 [cited Mar. 13, 2017]. Available from: http://requests-mock.readthedocs.io/ en/latest/. [Mar13] L. David Marquet. Turn the Ship Around! A True Story of Turning Followers into Leaders . Portfolio, 2013. [Mey92] Bertrand Meyer. Applying "Design by Contract". Computer , 25(10):40–51, October 1992. Available from: http://dx.doi.org/10.1109/2.161279. [MRP+17] Myron Marston, Jon Rowe, Sam Phippen, Xavier Shay, Aaron Kromer, Yuji Nakayama, and Bradley Schaefer. RSpec: Behaviour Driven Development for Ruby [online]. 2017 [cited Mar. 12, 2017]. Available from: http:// rspec.info. [Nor06] Dan North. Behavior modification: The evolution of behavior-driven devel- opment. Better Software Magazine , March 2006. Available online as "Intro- ducing BDD". Available from: http://dannorth.net/introducing-bdd/. [OMP04] Jonathan S. Ostroff, David Makalsky, and Richard F. Paige. Agile specifi- cation-driven development. In Jutta Eckstein and Hubert Baumeister, edi- tors, Extreme Programming and Agile Processes in Software Engineering , volume 3092 of Lecture Notes in Computer Science , pages 104–112. Springer Berlin Heidelberg, 2004. [Ope17] Open API Initiative. Homepage [online]. 2017 [cited Mar. 13, 2017]. Avail- able from: https://www.openapis.org/. [Par94] David Lorge Parnas. Software aging. In Proceedings of the 16th Interna- tional Conference on Software Engineering , ICSE ’94, pages 279–287, Los Alamitos, CA, USA, 1994. IEEE Computer Society Press. Available from: http://dl.acm.org/citation.cfm?id=257734.257788.
130 [PLL11] David Parsons, Ramesh Lal, and Manfred Lange. Test Driven Develop- ment: Advancing Knowledge by Conjecture and Confirmation. Future Internet , 3(4):281–297, 2011. Available from: http://www.mdpi.com/1999- 5903/3/4/281/.
[PM10] Shelly Park and Frank Maurer. A network analysis of stakeholders in tool visioning process for Story Test Driven Development. Proceedings of the IEEE International Conference on Engineering of Complex Computer Sys- tems (ICECCS) , pages 205–214, 2010.
[Pra09] Dhanji R. Prasanna. Dependency Injection . Manning Publications Co., Greenwich, CT, USA, 2009.
[RDLT14] Richard Rutledge, Sheryl Duggins, Dan Lo, and Frank Tsui. Formal speci- fication-driven development. In Proceedings of the International Confer- ence on Software Engineering Research and Practice (SERP) , page 1. The Steering Committee of The World Congress in Computer Science, Com- puter Engineering and Applied Computing (WorldComp), 2014. [Rei17] Kenneth Reitz. Requests: HTTP for humans [online]. 2017 [cited Mar. 13, 2017]. Available from: http://docs.python-requests.org/en/master/. [RJE14] Benno Rice, Richard Jones, and Jens Engel. Welcome to behave! [online]. 2014 [cited Mar. 10, 2017]. Available from: http://pythonhosted.org/ behave/. [RR16] Jon Reid and Chris Rose. PyHamcrest module [online]. 2016. Available from: https://github.com/hamcrest/PyHamcrest. [Som10] Ian Sommerville. Software Engineering . Addison-Wesley Publishing Com- pany, USA, 9th edition, 2010. [Tav15] Hugo Lopes Tavares. Should-DSL: Basic usage [online]. 2015 [cited Mar. 20, 2017]. Available from: https://pythonhosted.org/should_dsl/. [TRdS+10] Hugo Lopes Tavares, Gustavo Guimaraes Rezende, Vanderson Mota dos Santos, Rodrigo Soares Manhaes, and Rogerio Atem de Carvalho. A tool stack for implementing Behaviour-Driven Development in Python lan- guage. Computer Science , 2010. Available from: http://arxiv.org/abs/ 1007.1722v1. [WD96] Jim Woodcock and Jim Davies. Using Z: Specification, Refinement, and Proof . Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1996. [ZAC14] Dina Zayan, Michal Antkiewicz, and Krzysztof Czarnecki. Effects of using examples on structural model comprehension: A controlled experiment. In Proceedings of the 36th International Conference on Software Engineering (ICSE) , pages 955–966, New York, NY, USA, 2014. ACM. Available from: http://doi.acm.org/10.1145/2568225.2568270.
131 Appendix A: Sample BHive Custom Types File
This appendix provides an example of defining custom types to be used in a BHive-enabled development process. In the custom types file (Section A.1) written in JSON, the set
“HARDWARE_ID” defines seven elements that may be used as identifiers with a B machine, where as the sets STATUS_CODE, BILLING_ID, and PROFILE_NAMES are sets of constants. NO_BILLING_ID and NO_STATUS are scalar constants. Section A.2 shows the “CustomTypes.mch” machine generated by BHive during synthesis and included in all synthesized machines.
A.1 Custom Types File
[{ "set": "HARDWARE_ID", "enumeration": [ "V8A20u7w", "DyfcMh1S", "DXfxMx1X", "TT7X3qFI", "i80yCLRQ", "PeLB1TW3", "kMWE9KbF" ] }, { "set": "STATUS_CODE", "enumeration": [ 0, 200, 203, 300, 400, 401, 402, 403, 404,
132 410, 500 ], "constant": true }, { "set": "BILLING_ID", "enumeration": [ 0, 1000, 1010, 2001, 1982, 2321 ], "constant": true }, { "constant": "NO_BILLING_ID", "value": 0 }, { "constant": "NO_STATUS", "value": 0 }, { "set": "PROFILE_NAMES", "enumeration": [ "\"Abe\"", "\"Anne\"", "\"Huck\"", "\"John\"", "\"Willie\"" ], "constant": true } ] A.2 Synthesized CustomTypes Machine
MACHINE CustomTypes /* synthesized from definitions/custom_types.json*/ SETS HARDWARE_ID = {V8A20u7w, DyfcMh1S, DXfxMx1X, TT7X3qFI, i80yCLRQ, PeLB1TW3, kMWE9KbF} CONSTANTS NO_BILLING_ID, NO_STATUS, STATUS_CODE, PROFILE_NAMES, BILLING_ID PROPERTIES NO_BILLING_ID = 0 & NO_STATUS = 0 & STATUS_CODE = {0, 200, 203, 300, 400, 401, 402, 403, 404, 410, 500} & PROFILE_NAMES = {"Abe", "Anne", "Huck", "John", "Willie"} & BILLING_ID = {0, 1000, 1010, 2001, 1982, 2321} END
133 Appendix B: Specification for IPTV Back End
This specification is provided for use with the case study in Chapter 5. It was written using the OpenAPI (formerly “Swagger”) Specification standard for documenting RESTful web services in a “vendor neutral” way [Ope17]. The design of the backend system and its
REST API is based on the author’s experience in integrating with IPTV backend systems.
The level of granularity was deliberately chosen to illustrate multiple requests are typically required for interactions with the application, with the potential for state-related errors seeping into application code to handle multiple requests, cache results and populate data structures required by the application. swagger: '2.0' info: description: Mocks an IPTV VOD Backend version: 1.0.0 title: IPTV Mock API contact: email: [email protected] paths: /asset/{assetId}: get: summary: gets (short) metadata for an asset operationId: getAsset produces: - application/json parameters: - $ref: "#/parameters/AssetIdParameter" responses: 200: description: Successful fetch of asset details schema: $ref: "#/definitions/Asset" 404: description: No such asset
134 /assetDetails/{assetId}: get: summary: gets (long) metadata for an asset operationId: getAssetDetails produces: - application/json parameters: - $ref: "#/parameters/AssetIdParameter" responses: 200: description: An asset (long form) schema: $ref: "#/definitions/AssetDetails" 404: description: No such asset
/bid/{hardwareId}: get: summary: Returns billing id for given hardware id. Multiple hardware ids may map to a single billing id -- e.g. cable service, mobile phone on the same bill. operationId: getBillingId produces: - application/json parameters: - $ref: "#/parameters/HardwareIdParameter" responses: 200: description: BillingInfo associated to a specific hardware id. schema: $ref: '#/definitions/BillingInfo' 404: description: no billing assocaited with that device
/bookmarks/{profileId}: get: summary: returns a list of bookmarks for the given profile id operationId: getBookmarks produces: - application/json parameters: - $ref: "#/parameters/ProfileIdParameter" responses: 200: description: list of bookmarks schema: type: array items: $ref: '#/definitions/Bookmark' 404: description: not found
/entitlements/{subscriptionId}: get: summary: returns the list of entitlements for the given type for the given subscription operationId: getEntitlements produces: - application/json parameters:
135 - $ref: "#/parameters/SubscriptionIdParameter" responses: 200: description: list of entitlement ids for the given type schema: type: array items: $ref: '#/definitions/Entitlement' 400: description: bad type 404: description: subscription not found
/offers/{assetId}: get: summary: return the list of offers for the given asset operationId: getOffers produces: - application/json parameters: - $ref: "#/parameters/AssetIdParameter" responses: 200: description: list of offers schema: type: array items: $ref: '#/definitions/Offer' 404: description: asset not found
/profiles/{billingId}: get: summary: Returns all profiles associated with the given billing id. operationId: getProfiles produces: - application/json parameters: - $ref: "#/parameters/BillingIdParameter" responses: 200: description: Array of profiles. schema: type: array items: $ref: '#/definitions/Profile'
/subscriptions/{billingId}: get: summary: Returns all subscription information with the given billing id. operationId: getSubscriptions produces: - application/json parameters: - $ref: "#/parameters/BillingIdParameter" responses: 200: description: Array of subscriptions. schema:
136 type: array items: $ref: '#/definitions/SubscriptionInfo' 400: description: unable to determine subscription information parameters: AssetIdParameter: name: assetId in: path description: Id for the asset being referred to required: true type: number format: int32
BillingIdParameter: name: billingId in: path description: A user's billing id required: true type: number format: int32
HardwareIdParameter: name: hardwareId in: path description: A hardware ID of a device required: true type: string
ProfileIdParameter: name: profileId in: path description: The id for a user profile. required: true type: number format: int32
SubscriptionIdParameter: name: subscriptionId in: path description: The ID a user's subscription (a user typically has many). required: true type: number format: int32 definitions: Asset: type: object required: - assetId - assetDetailsId - title - posterUrl - rating properties: assetId: $ref: '#/definitions/Id' assetDetailsId:
137 $ref: '#/definitions/Id' title: type: string posterUrl: type: string rating: type: integer format: int32
AssetDetails: type: object required: - assetId - description - playbackUrl - rating - year properties: assetId: $ref: '#/definitions/Id' description: type: string playbackUrl: type: string rating: type: integer format: int32 language: type: string year: type: integer format: int32 duration: type: integer format: int32
BillingInfo: description: Billing information pertaining to a customer and their hardware devices. type: object required: - billingId - hardwareIds - name - address properties: billingId: $ref: '#/definitions/Id' hardwareIds: type: array items: $ref: '#/definitions/HardwareId' name: type: string address: type: string
Bookmark: type: object
138 required: - assetId - profileId - offset properties: assetId: $ref: '#/definitions/Id' offset: type: integer format: int32 profileId: type: integer format: int32
Entitlement: type: object required: - entitlementId - assetId - start - end properties: entitlementId: $ref: '#/definitions/Id' assetId: $ref: '#/definitions/Id' start: type: string format: date-time end: type: string format: date-time
HardwareId: description: A unique identifier for a hardware device. type: object required: - hardwareUUID - deviceInfo properties: hardwareUUID: type: string deviceInfo: type: string
Id: description: A numeric id, used across the system. type: object required: - id properties: id: type: integer format: int32
Offer: type: object required: - offerId
139 - price - subscriptionId properties: offerId: $ref: '#/definitions/Id' price: type: number format: float subscriptionId: type: integer format: int32
Profile: description: A user residing at a billing id. A profile represnts a demographic and a set of viewing preferences. type: object required: - profileId - name - restriction - pin properties: profileId: $ref: '#/definitions/Id' name: type: string restriction: type: integer pin: type: integer format: int32
SubscriptionInfo: description: Describes a subscription associated with a billing id. type: object required: - subscriptionId - type - start - end - entitlements properties: subscriptionId: $ref: '#/definitions/Id' type: type: string start: type: string format: date-time end: type: string format: date-time entitlements: type: array items: $ref: '#/definitions/Entitlement'
# basePath: /iptv/
140 # Added by API Auto Mocking Plugin schemes: - https # Added by API Auto Mocking Plugin host: virtserver.swaggerhub.com basePath: /mrjohndcarter/iptv/1.0.0
141 142