VIABLE ENTERPRISE SERVICE BUS MODEL A Model for Designing a Viable Service Integration Platform Nizami Jafarov Master of Business Administration (Computer Information Systems) Bachelor of Applied Mathematics and Economic Cybernetics (Applied Mathematics) A thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy at the School of Engineering and Information Technology, the University of New South Wales November · 2016 Canberra · Australia This Page is Left Blank Intentionally • ii • To My Beloved Mother Esmiralda • iii • This Page is Left Blank Intentionally • iv • Abstract Enterprise Service Bus (ESB) is a compound pattern of Service Oriented Architecture (SOA) that can provide sophisticated interconnectivity between services. The ESB concepts can be found at the heart of modern Enterprise Resource Planning, Customer Relationship Management and other conceptually similar platforms that integrate services for various purposes. It has evolved from an on-premises middleware to one of the essential elements of the Cloud Computing service models, such as the emerging Integration Platform as a Service. However, there is a problem in the use of ESB for the integration of services that arises from the breadth of technologies that can be incorporated into ESB, resulting in it becoming over-bloated with functions, noticeable in the distinct ESB products, offered by different software vendors, that support the notion of SOA differently. This research uses the Design Science Research (DSR) methodology to address the generic design of ESB. This approach amalgamates Thomas Erl’s SOA principles of service design and David Chappell’s characteristics that influence the ESB design with the design principles that are derived from the cybernetic concepts embedded into Stafford Beer’s Viable System Model (VSM). The result of this approach is the development of a novel artefact, in the form of a Viable Enterprise Service Bus Model (VESBM), which can be used for designing an ESB, independent from the technologies that might underpin it. The VESBM was found to be useful and usable in its application by several organisations that were designing Cloud-based and other integrated systems. • xi • Keywords Cybernetics, Enterprise Service Bus, Service Oriented Architecture, Viable System Model. • xii • This Page is Left Blank Intentionally • xiii • Acknowledgements During this research I was blessed to work with amazing people, who were supporting, assisting and guiding me throughout this journey. I would like to express my sincere gratitude to Dr. Edward Lewis, for his constant help, unlimited patience as well as individual and professional wisdom, which benefited me so much during these years. I am also deeply grateful to my late co-supervisor, Dr. Gary Millar, for challenging, understanding and extending the ideas we were exchanging during our long and joyful conversations. I am thankful to the staff of all companies involved in this research for their interest and support. Special thanks go to the Department of Defence, THALES Australia and the staff of Airservices Australia, for the fruitful cooperation at various stages of the research. I would also like to thank the staff of the School of Engineering and Information Technology at the University of New South Wales, Canberra campus. I am grateful to Mr. Craig Edwards, Mrs. Elvira Berra and Dr. Sherene Suchy for continuously supporting me throughout the challenges of the academic life. Special gratitude also goes to my colleagues and friends, whom I enjoyed working with and shared all the ups and downs during these years. I thank Dr. Evgeny Mironov, Mrs. Natalia Derevyanko, Ms. Anna Skobeleva, Mr. Alexey Balakirev, Dr. George Leu, Dr. Saber Elsayed, Dr. Mohamed Mabrok, Ms. Nastaran Nemati and Dr. Mohammad Esmaeil Zadeh. I also want to thank Prof. Hussein Abbass, Dr. Samir Alam, Prof. Hemanshu Pota, Dr. William Murray Mount, Dr. Deborah Tuček and others, for their collaboration throughout this research. • xiv • I am immensely grateful to my wonderful family, for their love, patience and sacrifices during my PhD journey. I thank them for being able to put up with the absent son, brother and uncle. I thank my beloved mother Esmiralda, my dearest sister Nika, and my wonderful nephews Rika and Medisha. I would also like to thank my late father, Nurik, who became the idol for the family and for me – you are in our hearts forever. • xv • This Page is Left Blank Intentionally • xvi • “Any sufficiently advanced technology is indistinguishable from magic.” – Sir Arthur Charles Clarke • xvii • This Page is Left Blank Intentionally • xviii • Preface Technology is an essential part of my life that defined me as an individual and a professional. From the early ages, I was genuinely interested in how computers work. My journey to the world of computers started in 1991 with the IBM 386SLC, old 5.25-inch floppy diskettes and a couple of programs running on DOS. As I grew, it was clear for me to keep pursuing my further education and future profession in the field close to Information Technology. However, along with technology, my love towards precise science was growing naturally too. I was trying to find a golden mean that could unite the two fields and came to the conclusion to do my Bachelors in Applied Mathematics and Economic Cybernetics. By the time of finishing my Bachelors, I had a couple of years of work experience as a programmer and software developer, in several domestic and international companies. Yet, I was still hungry for more knowledge and thankfully for my early work experience, I had an awareness of which degree to do the next. For me that was the Master of Business Administration (MBA). Taking a business degree benefited me tremendously. MBA provided me with the big picture thinking that I was initially, mistakenly, neglecting. A clear strategic vision on how technology can fit business was steadily getting revealed. This degree, and the experience in the technical and business fields, was also a substantial trigger for C-level executives, in the companies I worked for, to promote me into higher managerial positions. During those years, I grew from a programmer and software developer to a system architect and project manager. • xix • As a professional with expertise and experience in business and technology, I was involved in a wide range of Information Technology projects. The complexity of the projects varied, but most of them were associated with the systems and software integration: obsolete systems had to be integrated; old software had to operate with new software; adaptors had to be programmed for hardware and software interoperability; and so forth. Although, in these projects I was always advocating the use of best practices, industry-wide paradigms and popular technologies, such as the Enterprise Application Integration (EAI), Service Oriented Architecture (SOA), Web Services and others, there was a constant challenge with one of the core elements in most of the integration projects – the Enterprise Service Bus (ESB). ESB can offer a variety of integration options and even though this variety can be beneficial for tactical integrations at the bottom of enterprise IT, it also creates considerable complexity for integration strategies at the top. As such, this complexity can create unnecessary overburden across enterprise IT services, by providing functions that are often too much for an integration initiative, making the design of the system inherently complex. This is noticeable with the dozens of distinct ESB products, from different software vendors, available on the market. Some of them are over-bloated with capabilities that might hardly be needed during integrations, whilst others are over-promising and advertise functionalities they are incapable to deliver. Along with the possible effects on the integration strategies, the variety of functions in ESB also affects the cost of the end product and thus the investment strategies as well. Integration projects tend to be expensive initiatives and therefore, despite of the possible changes in integration requirements, companies diligently avoid the reinvestment of resources in new products or functions. As a result, for a long time, I was • xx • genuinely curious if it is possible to address the ESB from a generic perspective, to avoid possible vendor or technology lock-in. With the rise of Cloud Computing, the ESB, paired with Application Programming Interfaces (API), is getting increasingly used for Cloud integration. Cloud enables the functions, embedded into the ESB, to be provided as individual, on demand, services, which can scale and substantially benefit the integration strategies and subsequently the design of the end systems. This means that the functions, from different ESB vendors, can be mixed together to achieve greater diversity in meeting individual integration requirements. Thus, the identification of functions, that can be embedded into the ESB, becomes not only possible, but also necessary and worth investigation. I started my PhD with complex integration projects of Air Traffic Management Systems, working with Airservices Australia and Boeing and for, the main defence contractor of the Royal Australian Air Force, THALES. Those projects were deeply technical, exceptionally interesting and important for the government, but for me it was still a ‘walking around and about’. Because of the non-disclosure agreement, information on those projects cannot be revealed, but I was clearly far from the actual research area that I wanted to investigate. I decided to update and reinforce my research direction and started looking into the fields of Systems Thinking, Design Science and most importantly the Cybernetics. Whilst investigating the field of Cybernetics, thankfully to my supervisors, I came across of the Viable System Model (VSM), created by Stafford Beer, which could define the communication and control of a viable system. As at the basis of the ESB is the communication and control as well, I saw a potential, in the VSM, in addressing the design of the ESB from a generic perspective.